langsmith-cli 0.1.6

Context-efficient CLI for LangSmith. Built for humans and agents.

🛠️ LangSmith CLI

The Modern CLI for LangSmith Lightning-fast • Context-efficient • Built for humans and AI agents

Features • Installation • Quick Start • Examples • Documentation

---

🎯 Why LangSmith CLI?

Traditional tools are slow, verbose, and waste tokens. LangSmith CLI is different:

Feature	LangSmith CLI	Official MCP Server
Startup Time	< 100ms (lazy loading)	~2s (heavy imports)
Context Usage	96% reduction with --fields	Full objects always
Human UX	Rich tables with colors	JSON only
Agent UX	Strict --json mode	Mixed output
Live Updates	runs watch dashboard	❌
Advanced Filters	Regex, wildcards, smart presets	Basic only
Export Formats	JSON, CSV, YAML	JSON only

100% Feature Parity + Superior Quality of Life 🚀

---

✨ Features

🏎️ Performance First

• <100ms startup via lazy-loaded imports
• Streams large datasets without memory bloat
• Async-ready architecture

🧠 Agent Optimized

# Traditional: Returns 20KB trace object (1000+ tokens)
langsmith-cli runs get abc123

# Agent Mode: Returns only what you need (40 tokens)
langsmith-cli --json runs get abc123 --fields inputs,outputs,error

96% context savings on large traces!

🎨 Human Friendly

• Beautiful Rich tables with syntax highlighting
• Color-coded statuses (🟢 success, 🔴 error, 🟡 pending)
• Smart column truncation for readability
• Export to CSV/YAML for spreadsheets

🔍 Power User Features

# Regex filtering
langsmith-cli runs list --name-regex "^prod-.*-v[0-9]+"

# Wildcard patterns
langsmith-cli runs list --name-pattern "*auth*"

# Smart filters
langsmith-cli runs list --slow --failed --today

# Live dashboard
langsmith-cli runs watch

📦 Complete Coverage

Every LangSmith resource at your fingertips:

• ✅ Projects - List, create, inspect
• ✅ Runs - Search, stats, watch, open in browser
• ✅ Datasets - CRUD + bulk JSONL uploads
• ✅ Examples - Full lifecycle management
• ✅ Prompts - Version control your prompts

---

🚀 Installation

Using uv (Recommended)

uv tool install langsmith-cli

Using pip

pip install langsmith-cli

From Source

git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync
uv run langsmith-cli --help

---

🔑 Quick Start

1️⃣ Authenticate

langsmith-cli auth login
# Creates .env with your LANGSMITH_API_KEY

2️⃣ Explore Your Projects

langsmith-cli projects list

┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project       ┃ Run Count  ┃ Last Run     ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ production    │ 12,450     │ 2 mins ago   │
│ staging       │ 3,241      │ 15 mins ago  │
│ development   │ 892        │ 1 hour ago   │
└───────────────┴────────────┴──────────────┘

3️⃣ Inspect Recent Runs

langsmith-cli runs list --project production --limit 5

4️⃣ Debug Errors Fast

langsmith-cli runs list --failed --recent --fields error

---

💡 Examples

🔍 Advanced Filtering

Find authentication runs that failed in the last hour:

langsmith-cli runs list \
  --name-pattern "*auth*" \
  --failed \
  --recent

Search for specific versioned services:

langsmith-cli runs list \
  --name-regex "^prod-api-v[0-9]+" \
  --min-latency 5s \
  --today

Multi-tag filtering (AND logic):

langsmith-cli runs list \
  --tag production \
  --tag experimental \
  --slow

📊 Aggregated Insights

langsmith-cli runs stats --project production

{
  "run_count": 12450,
  "error_rate": 0.023,
  "latency_p50": 0.234,
  "latency_p99": 1.892,
  "total_cost": 45.67,
  "last_run_time": "2026-01-14T20:15:30Z"
}

🔴 Live Monitoring

langsmith-cli runs watch --project production

🔴 Live Dashboard (Ctrl+C to exit)

┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Run              ┃ Status ┃ Latency  ┃ Time     ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ ChatOpenAI       │ 🟢     │ 0.234s   │ Just now │
│ RetrievalChain   │ 🟢     │ 1.456s   │ 2s ago   │
│ AuthService      │ 🔴     │ 5.123s   │ 5s ago   │
│ WebScraper       │ 🟡     │ -        │ 8s ago   │
└──────────────────┴────────┴──────────┴──────────┘

💾 Bulk Dataset Uploads

# Export examples to JSONL
langsmith-cli --json examples list --dataset my-dataset > examples.jsonl

# Upload to new dataset
langsmith-cli datasets push examples.jsonl --dataset production-eval

🌐 Open in Browser

langsmith-cli runs open <run-id>
# Opens trace in LangSmith UI

📤 Export for Analysis

# Export to CSV for Excel
langsmith-cli runs list --format csv > runs.csv

# Export to YAML for configs
langsmith-cli projects list --format yaml > projects.yml

---

🤖 AI Agent Integration

As a Claude Code Skill

The CLI is optimized for Claude Code agents:

# In Claude Code, agents can run:
langsmith-cli --json runs list --project default --limit 10 --fields inputs,outputs,error

# Returns clean JSON without Rich formatting
# Uses --fields to minimize context usage
# Filters with --failed, --slow, --recent for targeted debugging

Key Agent Patterns:

• Always use --json as first argument
• Use --fields to reduce tokens by 90%+
• Combine smart filters: --failed --recent --slow
• Keep --limit small (default: 10)

Example Agent Usage

import subprocess
import json

# Agent efficiently fetches only errored runs
result = subprocess.run(
    ["langsmith-cli", "--json", "runs", "list",
     "--failed", "--limit", "5", "--fields", "error,inputs"],
    capture_output=True,
    text=True
)

runs = json.loads(result.stdout)
# Analyze errors with minimal context usage

---

📚 Documentation

Command Reference

langsmith-cli --help

# Core Commands
auth login              # Authenticate with LangSmith
projects list           # List all projects
runs list              # Search and filter runs
runs get <id>          # Inspect a specific run
runs stats             # Aggregate statistics
runs watch             # Live run dashboard
runs open <id>         # Open trace in browser
datasets list          # List datasets
datasets create        # Create new dataset
datasets push          # Bulk upload from JSONL
examples list          # List dataset examples
examples create        # Add example to dataset
prompts list           # List prompt repositories
prompts get            # Pull a prompt template
prompts push           # Push local prompt to LangSmith

Global Flags

--json              # Machine-readable JSON output (no Rich formatting)
--format {json|csv|yaml}  # Export format
--help              # Show help
--version           # Show version

Runs Filtering

Option	Description	Example
--project	Filter by project	--project production
--status	Filter by status	--status error
--failed	Only failed runs	--failed
--succeeded	Only successful runs	--succeeded
--slow	Runs >5s latency	--slow
--recent	Last hour	--recent
--today	Today's runs	--today
--min-latency	Min latency	--min-latency 2s
--max-latency	Max latency	--max-latency 10s
--since	Since time	--since "1 hour ago"
--last	Last duration	--last 24h
--tag	Filter by tag (repeatable)	--tag prod --tag beta
--name-pattern	Wildcard match	--name-pattern "*auth*"
--name-regex	Regex match	--name-regex "^prod-.*"
--model	Filter by model	--model gpt-4

---

🏗️ Architecture

Design Principles

1. Lazy Loading Performance

# ❌ Don't import heavy libraries at top level
from langsmith import Client

# ✅ Import inside command functions
@runs.command("list")
def list_runs(...):
    import langsmith  # Only loads when command executes
    client = langsmith.Client()

2. Context Efficiency

• Default JSON output is "sparse" (only essential fields)
• Field pruning with --fields for targeted data extraction
• No full trace blobs unless explicitly requested

3. Dual UX Pattern

# Check JSON flag for output mode
if ctx.obj.get("json"):
    click.echo(json.dumps(data))  # Strict JSON for agents
else:
    console.print(rich_table)      # Beautiful tables for humans

4. DRY Utilities

• Shared helpers in utils.py (100% test coverage)
• Consistent error handling with field context
• Reusable filtering, sorting, and formatting functions

---

🧪 Development

Setup

# Clone and setup
git clone https://github.com/gigaverse-app/langsmith-cli.git
cd langsmith-cli
uv sync

# Install pre-commit hooks
uv run pre-commit install

Testing

# Run all tests
uv run pytest

# With coverage
uv run pytest --cov=src --cov-report=term-missing

# E2E tests (requires LANGSMITH_API_KEY)
export LANGSMITH_API_KEY="lsv2_..."
uv run pytest tests/test_e2e.py -v

Code Quality

# Linting and formatting
uv run ruff check --fix
uv run ruff format

# Type checking
uv run pyright

Project Stats

• 79% Test Coverage (116 tests)
• 100% Utils Coverage (47 tests for helpers)
• Zero Type Errors (Pyright clean)
• 100% MCP Parity (13/13 tools)

---

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/awesome)
3. Make your changes with tests
4. Ensure 100% coverage for new code
5. Run uv run pre-commit run --all-files
6. Submit a pull request

---

📝 License

MIT License - see LICENSE for details.

---

🙏 Acknowledgments

Built with:

• Click - CLI framework
• Rich - Terminal formatting
• LangSmith SDK - Official Python client
• uv - Fast Python package installer

---

Made with ❤️ for the LangChain community

Report Bug • Request Feature • Documentation