wati-astra-cli 0.1.5

Wati Astra CLI: agent optimizer and MCP server (Go binary wrapped for pip)

astra-insight-mcp

A standalone MCP server that provides a self-contained agent optimization loop — read data, update instructions, run tests, evaluate, and iterate — all through the Astra Gateway public API.

CLI (astra)

Install the unified command-line tool (interactive REPL, MCP server, OAuth):

go install github.com/ClareAI/astra-insight-mcp/cmd/astra@latest

pip (Python wrapper)

The PyPI distribution name is wati-astra-cli. Installing runs go build on the machine and ships the astra binary inside the package; the console script is still astra (same as go install).

# From repo root (development)
pip install .

# After install: set up LLM key once, then use the REPL
astra configure
astra

Using pip install wati-astra-cli

That only works after the project is published to PyPI (or to a private index that your pip is configured to use).

Option A — Publish to PyPI (public pip install):

1. Create a pypi.org account and an API token under account settings.

2. From the repository root:

   pip install build twine
   python -m build    # e.g. dist/wati_astra_cli-0.1.0.tar.gz and .whl
   twine upload dist/*
   

   First upload: use username __token__ and the token as the password.

3. On any machine (with Go installed for sdist builds; see below):

   pip install wati-astra-cli
   astra version
   

   Dry-run on TestPyPI: twine upload --repository testpypi dist/*, then
   pip install -i https://test.pypi.org/simple/ wati-astra-cli.

Option B — Install from Git (no PyPI):

pip install "git+https://github.com/ClareAI/astra-insight-mcp.git@main"

Adjust URL/branch for your fork; Go is still required because the package builds from source.

Option C — Private index (DevPI, Artifactory, etc.): Set extra-index-url or index-url in pip.conf and point twine at your registry; flow is otherwise like Option A.

Optional local builds:

pip install build
python -m build
# dist/*.tar.gz (sdist) and *.whl; the wheel embeds a binary built for the current platform

Note: Installing from sdist runs go build on the user machine, so Go must be installed. The default wheel tag is py3-none-any but the embedded binary is platform-specific. To offer “no Go required” installs on PyPI, build and upload platform-specific wheels from CI for linux / macOS / Windows (or publish sdist only).

Then use astra (similar layout to other coding CLIs: default action is the REPL; server is a subcommand).

Command	Description
astra	Interactive agent optimizer (REPL)
astra chat	Same as astra
astra server	Streamable HTTP MCP server (/mcp, /healthz)
astra mcp	Alias for astra server
astra auth …	Browser OAuth or --token (see below)
astra configure	Save LLM API key to ~/.astra/llm.json (like Claude-style setup)
astra version	Print version
astra help	Usage

Global flag before subcommands: -debug / --debug (gateway curl traces). Legacy entrypoints still work: go run ./cmd/main.go (server only) and go run ./cmd/cli (REPL / auth).

Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                    astra-insight-mcp (this repo)                     │
│                                                                      │
│  Streamable HTTP MCP Server (spec 2025-03-26)                        │
│  POST /mcp  — JSON-RPC 2.0 (JSON or SSE response)                   │
│  GET  /mcp  — SSE notification stream                                │
│  DELETE /mcp — Terminate session                                     │
│  GET /healthz — Health check                                         │
│                                                                      │
│  Session management (Mcp-Session-Id header)                          │
│  Per-IP rate limiting (configurable req/min)                         │
│                                                                      │
│  64+ tools across skills:                                            │
│  Read / Write / Test / Integrations / Webhooks / Analytics           │
└───────────────────────────┬──────────────────────────────────────────┘
                            │  HTTP (Bearer token)
                            ▼
                 Astra Gateway API
              astra-gateway.wati.io/v1

Zero changes to any existing service. All data flows through the public Gateway API.

Optimization Loop

 Understand
 ─────────
  1. list_agents                         → discover available agents
  2. get_agent                           → current instruction & rules
  3. get_agent_config                    → goals, style, lead fields
  4. get_runtime_config                  → live vs draft config comparison
  5. get_brandkit                        → brand colors, logos, company info
  6. list_actions / get_action           → what tools the agent can use
  7. list_connections                    → platform integrations
  8. list_knowledge_documents            → what knowledge the agent has
  9. list_annotations / get_annotation   → existing FAQ overrides
 10. retrieve_knowledge                  → test knowledge base coverage
 11. list_templates / get_template       → reference templates
 12. get_voice_config                    → voice agent settings

 Analyze
 ───────
 13. get_conversation_analytics          → conversation volume & trends
 14. get_lead_analytics                  → lead funnel & daily breakdown
 15. get_performance_analytics           → visitor → lead → qualified funnel
 16. get_lead_distribution               → hot/warm/cold/unqualified breakdown
 17. list_conversations / get_conversation → browse real conversations
 18. get_conversation_messages           → deep-dive into specific chats
 19. get_conversation_lead_qualification → lead qualification per conversation
 20. get_contacts_and_leads / get_contact → customer profiles & lead scores
 21. get_contact_conversations           → all conversations for a contact
 22. list_conversation_feedback          → user feedback per conversation
 23. get_lead_qualifications             → lead quality patterns
 24. get_voice_conversation / messages   → voice call transcripts

 Improve
 ───────
 25. update_agent_instruction            → improve instruction
 26. update_agent_config                 → update style, goals, lead fields
 27. create_annotation / update / delete → manage FAQ overrides
 28. create_knowledge_document           → add text to knowledge base
 29. delete_knowledge_document           → remove outdated knowledge
 30. import_knowledge_from_url           → add a web page
 31. import_knowledge_from_website       → crawl & import entire website
 32. clone_agent                         → create safe copy for experiments
 33. publish_agent                       → activate changes

 Verify — Manual
 ───────────────
 34. send_test_message                   → test with sample queries

 Verify — Systematic Testing Pipeline
 ─────────────────────────────────────
 35. start_test_pipeline                 → one-click: generate → execute → evaluate → analyze
 36. create_test_suite                   → auto-generate test cases (6 categories)
 37. add_test_case                       → manually add test cases
 38. create_test_run                     → execute test suite against agent
 39. get_test_run                        → inspect individual results
 40. create_run_analysis                 → AI analyzes failures
 41. analyze_failures                    → Phase 1: diagnose issues
 42. generate_instruction                → Phase 2: generate improved instruction
     → apply instruction and re-run tests

Tools

Read — Agent Config & Setup

Tool	Description
list_agents	List all agents — names, IDs, types, statuses, channels
get_agent	Agent name, instructions, system rules, status
get_agent_config	Communication style, goals, lead fields, qualification criteria
get_runtime_config	Live/draft runtime config — deployed instructions, rules, actions
get_brandkit	Brand kit — colors, logos, company info, appearance settings
list_actions	Configured tools/actions (HubSpot, Calendly, web search, etc.)
get_action	Single action details — config, platform, connection status
list_connections	Platform connections and their status

Read — Knowledge & Annotations

Tool	Description
list_knowledge_documents	Documents in the knowledge base (names, word counts, hit counts)
list_annotations	FAQ annotations (Q&A pairs, hit counts)
get_annotation	Single annotation details
retrieve_knowledge	Search knowledge base with a query

Read — Templates

Tool	Description
list_templates	Pre-built agent templates, filter by category/type
list_template_categories	Available template categories
get_template	Template details — instructions, config, goals

Read — Performance Data

Tool	Description
get_conversation_analytics	Conversation count analytics, daily breakdown
get_lead_analytics	Lead funnel — visitors, leads, hot/warm/cold, daily trends
get_performance_analytics	High-level funnel — visitor → lead → qualified count
get_lead_distribution	Lead score distribution (hot/warm/cold/unqualified)
list_conversations	Browse real user conversations with filters
get_conversation	Single conversation — status, channel, contact, metadata
get_conversation_messages	Full message history for a conversation
get_conversation_lead_qualification	Lead qualification for a specific conversation
get_contacts_and_leads	Contact profiles, lead scores, AI insights
get_contact	Single contact — email, phone, company, lead score, AI summary
get_contact_conversations	All conversations for a specific contact
list_conversation_feedback	User feedback (thumbs up/down, tags, comments)
get_lead_qualifications	Lead level, score, AI summary per lead

Read — Voice

Tool	Description
get_voice_config	Voice agent configuration — model, language, style
get_voice_conversation	Voice conversation details — duration, status, phone
get_voice_messages	Voice conversation transcript

Write — Modify Agent

Tool	Description
update_agent_instruction	Update draft instructions
update_agent_config	Update config — communication style, goals, lead fields
publish_agent	Publish draft to make it live
clone_agent	Clone an agent for safe experimentation

Write — Knowledge & Annotations

Tool	Description
create_annotation	Add FAQ annotation (guaranteed-correct Q&A)
update_annotation	Update an annotation's title, questions, or answer
delete_annotation	Delete an annotation
create_knowledge_document	Add a text document to the knowledge base
delete_knowledge_document	Delete a document from the knowledge base
import_knowledge_from_url	Import a web page into the knowledge base
import_knowledge_from_website	Crawl and import an entire website

Webhooks

Tool	Description
list_webhooks	List registered webhooks and subscribed events
create_webhook	Register a webhook for real-time event notifications
update_webhook	Update webhook active status or events
delete_webhook	Delete a webhook

Utility

Tool	Description
get_import_status	Check knowledge base import job progress

Test — Manual Verification

Tool	Description
send_test_message	Send a message and get agent response (supports multi-turn)

Test — Systematic Pipeline

The testing pipeline provides automated test generation, execution, evaluation, and analysis.

Quick start: Use start_test_pipeline for one-click end-to-end testing. Advanced: Use individual tools for fine-grained control.

Tool	Description
start_test_pipeline	Full pipeline: generate → execute → evaluate → analyze
list_pipeline_jobs	List past/running pipeline jobs
get_pipeline_job	Get pipeline job status and results
create_test_suite	Auto-generate test cases (6 categories)
list_test_suites	List test suites, filter by status
get_test_suite	Get test suite with all test cases
get_test_suite_progress	Lightweight progress polling during generation
add_test_case	Manually add a test case to a suite
create_test_run	Execute and evaluate a test suite
list_test_runs	List test run history
get_test_run	Get full run results (pass/fail, scores, responses)
create_run_analysis	AI-powered analysis of test results
get_run_analysis	Get analysis — suggestions and recommended instruction
analyze_failures	Phase 1: diagnose failures
generate_instruction	Phase 2: generate improved instruction from analysis
get_agent_test_overview	Agent testing summary — suites, runs, pass rate

Running Modes

1. HTTP MCP Server (Streamable HTTP)

Serves /mcp (Streamable HTTP, spec 2025-03-26) and /healthz. Compatible with Cursor Remote MCP, Claude Desktop, and any MCP-compliant client.

astra server
# or: go run ./cmd/main.go

2. Interactive CLI Agent

Chat-style interface with a built-in LLM that orchestrates all 64 tools autonomously.

Recommended first-time setup (pip / go install): save your LLM key once, then start the REPL:

astra configure    # choose OpenRouter / OpenAI / Gemini, then API key (hidden), base URL, model → ~/.astra/llm.json
astra

Non-interactive examples:

astra configure --provider openai --api-key sk-...
astra configure --provider gemini --api-key AIza...    # Google AI Studio key
astra configure --provider openrouter --api-key sk-or-...

Or use environment variables (they override the file when set):

export LLM_API_KEY=sk-...          # or OPENAI_API_KEY
export LLM_MODEL=gpt-4o            # optional
export LLM_BASE_URL=https://...    # optional

astra
# or: go run ./cmd/cli/main.go

astra configure also accepts flags for scripts: astra configure --api-key sk-... --base-url https://openrouter.ai/api/v1 --model anthropic/claude-sonnet-4. Use astra configure --show-path to print the config file path.

If no key is in the file and LLM_API_KEY / OPENAI_API_KEY is unset, a TTY session prompts for provider (1–3) then API key (hidden), and applies the right default base URL and model. Non-interactive use requires astra configure --api-key ... (optional --provider) or env vars.

Default API bases: OpenRouter https://openrouter.ai/api/v1, OpenAI https://api.openai.com/v1, Gemini https://generativelanguage.googleapis.com/v1beta/openai (AI Studio key, OpenAI-compatible).

Commands: exit/quit/q to quit, clear/reset to start a new conversation.

OAuth Login (Browser)

The CLI supports browser-based OAuth login using PKCE (no client secret needed). After authorization, the access token is saved locally and used automatically for all subsequent tool calls.

Quick Login

# Login via browser (default: dev environment)
astra auth --oauth --env dev

# Login to production
astra auth --oauth --env prod

# Print authorize URL without opening browser
astra auth --oauth --env dev --no-open

How It Works

1. The CLI generates a PKCE code challenge (S256) and starts a local HTTP server on 127.0.0.1:8787/callback
2. Your browser opens the Astra authorization page
3. After you log in and approve, the browser redirects to the local server with an authorization code
4. The CLI exchanges the code for an access token (using the PKCE code verifier)
5. The token is saved to ~/.astra/oauth_token.json

Auth Flags

Flag	Default	Description
--oauth	—	Run browser-based OAuth PKCE flow
--token <tok>	—	Manually save an access token
--env	dev	Environment: prod, stage, dev, qa
--scope	chat,agents:read,agents:write	OAuth scopes
--listen-host	127.0.0.1	Callback server bind address
--listen-port	8787	Callback server port
--timeout	120	Seconds to wait for callback
--no-open	false	Don't open browser; print URL only

Auth Token Resolution

When making API calls, tokens are resolved in this order:

1. api_key argument passed explicitly in tool call
2. ASTRA_API_KEY environment variable
3. ASTRA_ACCESS_TOKEN environment variable
4. ASTRA_OAUTH_ACCESS_TOKEN environment variable
5. ~/.astra/oauth_token.json (saved by auth --oauth)

Existing API key workflows are fully compatible -- OAuth login is additive and does not break any existing authentication method.

Quick Start

go run ./cmd/main.go

# List all tools
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# List agents
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":2,"method":"tools/call",
    "params":{"name":"list_agents","arguments":{"api_key":"sk_live_xxx"}}
  }'

# Start full test pipeline
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":3,"method":"tools/call",
    "params":{"name":"start_test_pipeline","arguments":{
      "api_key":"sk_live_xxx","agent_id":"uuid","user_id":"tester"
    }}
  }'

# Generate improved instruction from test failures
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":4,"method":"tools/call",
    "params":{"name":"generate_instruction","arguments":{
      "api_key":"sk_live_xxx",
      "instruction":"You are a helpful sales agent...",
      "improvement_suggestions":["Be more specific about pricing","Handle objections better"]
    }}
  }'

Docker Deployment

# Build
docker build -t astra-insight-mcp .

# Run (image entrypoint: astra server)
docker run -p 8080:8080 \
  -e ASTRA_GATEWAY_BASE_URL=https://astra-gateway.wati.io \
  -e ASTRA_API_KEY=sk_live_... \
  -e RATE_LIMIT_PER_MIN=60 \
  astra-insight-mcp

# Verify
curl http://localhost:8080/healthz
# → ok

# Initialize MCP session
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
# → returns Mcp-Session-Id header

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: astra-insight-mcp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: astra-insight-mcp
  template:
    metadata:
      labels:
        app: astra-insight-mcp
    spec:
      containers:
        - name: mcp
          image: astra-insight-mcp:latest
          ports:
            - containerPort: 8080
          env:
            - name: ASTRA_GATEWAY_BASE_URL
              value: "https://astra-gateway.wati.io"
            - name: ASTRA_API_KEY
              valueFrom:
                secretKeyRef:
                  name: astra-secrets
                  key: api-key
            - name: RATE_LIMIT_PER_MIN
              value: "120"
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 10
          resources:
            requests:
              memory: "64Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: astra-insight-mcp
spec:
  selector:
    app: astra-insight-mcp
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

Authentication

Every tool call requires an auth token. The token is resolved automatically from multiple sources (highest priority first):

Priority	Source	Example
1	api_key argument in tool call	"api_key": "sk_live_xxx"
2	ASTRA_API_KEY env var	export ASTRA_API_KEY=sk_live_xxx
3	ASTRA_ACCESS_TOKEN env var	export ASTRA_ACCESS_TOKEN=astra_at_xxx
4	ASTRA_OAUTH_ACCESS_TOKEN env var	export ASTRA_OAUTH_ACCESS_TOKEN=astra_at_xxx
5	~/.astra/oauth_token.json file	{"access_token": "astra_at_xxx"}

Both API keys (sk_live_*) and OAuth access tokens (astra_at_*) are accepted -- the gateway treats them identically as Bearer tokens.

Using OAuth tokens

If you authenticate via the Astra OAuth flow (e.g. astra auth --oauth), the CLI writes a token file to ~/.astra/oauth_token.json. astra-insight-mcp will pick it up automatically -- no additional configuration needed.

# Option A: set an env var
export ASTRA_ACCESS_TOKEN=astra_at_xxx
astra

# Option B: use the token file (written by astra auth)
# ~/.astra/oauth_token.json -> {"access_token": "astra_at_xxx", ...}
astra
# -> token is auto-resolved, no env var needed

Environment Variables

MCP Server

Variable	Default	Description
PORT	8080	Server listen port
ASTRA_GATEWAY_BASE_URL	https://astra-gateway.wati.io	Gateway API base URL
ASTRA_API_KEY	--	Astra API key (sk_live_*), auto-injected into all tool calls
ASTRA_ACCESS_TOKEN	--	OAuth access token (astra_at_*), used if ASTRA_API_KEY is not set
ASTRA_OAUTH_ACCESS_TOKEN	--	Alternative OAuth env var name
RATE_LIMIT_PER_MIN	60	Max requests per IP per minute
ASTRA_LOG_LEVEL	info	Service JSON log level: debug, info, warn, error. Agent/LLM iteration traces only at debug
ASTRA_LOG_DEBUG	—	Set to 1 to force debug (same as ASTRA_LOG_LEVEL=debug)

CLI Agent (additional)

Variable	Default	Description
~/.astra/llm.json	—	Created by astra configure (api_key, base_url, model, optional fast_model; file mode 0600)
LLM_API_KEY / OPENAI_API_KEY	--	If set, overrides api_key from the file
LLM_BASE_URL	—	If set, overrides base_url from the file; if still empty, the LLM client defaults to OpenAI’s API URL
LLM_MODEL	anthropic/claude-sonnet-4	If LLM_MODEL is unset, the value from the file is used; otherwise this default
LLM_FAST_MODEL	(derived)	Optional; overrides derived fast model from the file or heuristics