wati-astra-cli 0.1.0

Wati Astra CLI: agent optimizer and MCP server (Go binary wrapped for pip)

astra-insight-mcp

A standalone MCP server that provides a self-contained agent optimization loop — read data, update instructions, run tests, evaluate, and iterate — all through the Astra Gateway public API.

CLI (astra)

Install the unified command-line tool (interactive REPL, MCP server, OAuth):

go install github.com/ClareAI/astra-insight-mcp/cmd/astra@latest

pip（Python 包装）

PyPI 发行名是 wati-astra-cli。安装时会在本机用 Go 编译 astra，并把可执行文件打进 Python 包；命令行入口仍是 astra（与 go install 一致）。

# 仓库根目录（开发）
pip install .

用 pip install wati-astra-cli（像装普通库一样）

这个名字只有在 PyPI 上已经有同名发行版 时才能用。你们需要有人把包 上传（publish）一次（或上传到私有索引，并配置 pip 使用该索引）。

方式 A — 公开发到 PyPI（任何人都能 pip install）：

1. 在 pypi.org 注册账号，用户设置里创建 API token。

2. 在本仓库根目录：

   pip install build twine
   python -m build    # 生成 dist/wati_astra_cli-0.1.0.tar.gz 等
   twine upload dist/*
   

   首次上传会提示输入 PyPI 用户名（可用 __token__）和密码（粘贴 token）。

3. 之后任意环境（需已装 Go，见下方说明）：

   pip install wati-astra-cli
   astra version
   

   建议先在 TestPyPI 试传：twine upload --repository testpypi dist/*，安装时用
   pip install -i https://test.pypi.org/simple/ wati-astra-cli。

方式 B — 不发 PyPI，直接从 Git 安装（适合内测 / 私有仓库）：

pip install "git+https://github.com/ClareAI/astra-insight-mcp.git@main"

（把 URL、分支名换成你们真实仓库；同样需要本机有 Go，因为会从源码构建。）

方式 C — 私有 PyPI / DevPI / Artifactory：配置 pip.conf 的 extra-index-url 或 index-url，上传流程与方式 A 类似，只是 twine 指到你们的仓库地址。

构建产物（可选）：

pip install build
python -m build
# dist/ 下生成 .tar.gz（sdist）和 .whl；wheel 内含当前平台编译出的二进制

注意： 当前从 sdist 安装会在用户机器上执行 go build，因此需要 已安装 Go。默认 wheel 标签是 py3-none-any，但内部二进制是平台相关的；若要在 PyPI 上提供「免装 Go」的安装，需要在 CI 里为 linux / macOS / Windows 分别构建并上传带平台标签的 wheel（或只发 sdist）。

Then use astra (similar layout to other coding CLIs: default action is the REPL; server is a subcommand).

Command	Description
astra	Interactive agent optimizer (REPL)
astra chat	Same as astra
astra server	Streamable HTTP MCP server (/mcp, /healthz)
astra mcp	Alias for astra server
astra auth …	Browser OAuth or --token (see below)
astra version	Print version
astra help	Usage

Global flag before subcommands: -debug / --debug (gateway curl traces). Legacy entrypoints still work: go run ./cmd/main.go (server only) and go run ./cmd/cli (REPL / auth).

Architecture

┌──────────────────────────────────────────────────────────────────────┐
│                    astra-insight-mcp (this repo)                     │
│                                                                      │
│  Streamable HTTP MCP Server (spec 2025-03-26)                        │
│  POST /mcp  — JSON-RPC 2.0 (JSON or SSE response)                   │
│  GET  /mcp  — SSE notification stream                                │
│  DELETE /mcp — Terminate session                                     │
│  GET /healthz — Health check                                         │
│                                                                      │
│  Session management (Mcp-Session-Id header)                          │
│  Per-IP rate limiting (configurable req/min)                         │
│                                                                      │
│  64+ tools across skills:                                            │
│  Read / Write / Test / Integrations / Webhooks / Analytics           │
└───────────────────────────┬──────────────────────────────────────────┘
                            │  HTTP (Bearer token)
                            ▼
                 Astra Gateway API
              astra-gateway.wati.io/v1

Zero changes to any existing service. All data flows through the public Gateway API.

Optimization Loop

 Understand
 ─────────
  1. list_agents                         → discover available agents
  2. get_agent                           → current instruction & rules
  3. get_agent_config                    → goals, style, lead fields
  4. get_runtime_config                  → live vs draft config comparison
  5. get_brandkit                        → brand colors, logos, company info
  6. list_actions / get_action           → what tools the agent can use
  7. list_connections                    → platform integrations
  8. list_knowledge_documents            → what knowledge the agent has
  9. list_annotations / get_annotation   → existing FAQ overrides
 10. retrieve_knowledge                  → test knowledge base coverage
 11. list_templates / get_template       → reference templates
 12. get_voice_config                    → voice agent settings

 Analyze
 ───────
 13. get_conversation_analytics          → conversation volume & trends
 14. get_lead_analytics                  → lead funnel & daily breakdown
 15. get_performance_analytics           → visitor → lead → qualified funnel
 16. get_lead_distribution               → hot/warm/cold/unqualified breakdown
 17. list_conversations / get_conversation → browse real conversations
 18. get_conversation_messages           → deep-dive into specific chats
 19. get_conversation_lead_qualification → lead qualification per conversation
 20. get_contacts_and_leads / get_contact → customer profiles & lead scores
 21. get_contact_conversations           → all conversations for a contact
 22. list_conversation_feedback          → user feedback per conversation
 23. get_lead_qualifications             → lead quality patterns
 24. get_voice_conversation / messages   → voice call transcripts

 Improve
 ───────
 25. update_agent_instruction            → improve instruction
 26. update_agent_config                 → update style, goals, lead fields
 27. create_annotation / update / delete → manage FAQ overrides
 28. create_knowledge_document           → add text to knowledge base
 29. delete_knowledge_document           → remove outdated knowledge
 30. import_knowledge_from_url           → add a web page
 31. import_knowledge_from_website       → crawl & import entire website
 32. clone_agent                         → create safe copy for experiments
 33. publish_agent                       → activate changes

 Verify — Manual
 ───────────────
 34. send_test_message                   → test with sample queries

 Verify — Systematic Testing Pipeline
 ─────────────────────────────────────
 35. start_test_pipeline                 → one-click: generate → execute → evaluate → analyze
 36. create_test_suite                   → auto-generate test cases (6 categories)
 37. add_test_case                       → manually add test cases
 38. create_test_run                     → execute test suite against agent
 39. get_test_run                        → inspect individual results
 40. create_run_analysis                 → AI analyzes failures
 41. analyze_failures                    → Phase 1: diagnose issues
 42. generate_instruction                → Phase 2: generate improved instruction
     → apply instruction and re-run tests

Tools

Read — Agent Config & Setup

Tool	Description
list_agents	List all agents — names, IDs, types, statuses, channels
get_agent	Agent name, instructions, system rules, status
get_agent_config	Communication style, goals, lead fields, qualification criteria
get_runtime_config	Live/draft runtime config — deployed instructions, rules, actions
get_brandkit	Brand kit — colors, logos, company info, appearance settings
list_actions	Configured tools/actions (HubSpot, Calendly, web search, etc.)
get_action	Single action details — config, platform, connection status
list_connections	Platform connections and their status

Read — Knowledge & Annotations

Tool	Description
list_knowledge_documents	Documents in the knowledge base (names, word counts, hit counts)
list_annotations	FAQ annotations (Q&A pairs, hit counts)
get_annotation	Single annotation details
retrieve_knowledge	Search knowledge base with a query

Read — Templates

Tool	Description
list_templates	Pre-built agent templates, filter by category/type
list_template_categories	Available template categories
get_template	Template details — instructions, config, goals

Read — Performance Data

Tool	Description
get_conversation_analytics	Conversation count analytics, daily breakdown
get_lead_analytics	Lead funnel — visitors, leads, hot/warm/cold, daily trends
get_performance_analytics	High-level funnel — visitor → lead → qualified count
get_lead_distribution	Lead score distribution (hot/warm/cold/unqualified)
list_conversations	Browse real user conversations with filters
get_conversation	Single conversation — status, channel, contact, metadata
get_conversation_messages	Full message history for a conversation
get_conversation_lead_qualification	Lead qualification for a specific conversation
get_contacts_and_leads	Contact profiles, lead scores, AI insights
get_contact	Single contact — email, phone, company, lead score, AI summary
get_contact_conversations	All conversations for a specific contact
list_conversation_feedback	User feedback (thumbs up/down, tags, comments)
get_lead_qualifications	Lead level, score, AI summary per lead

Read — Voice

Tool	Description
get_voice_config	Voice agent configuration — model, language, style
get_voice_conversation	Voice conversation details — duration, status, phone
get_voice_messages	Voice conversation transcript

Write — Modify Agent

Tool	Description
update_agent_instruction	Update draft instructions
update_agent_config	Update config — communication style, goals, lead fields
publish_agent	Publish draft to make it live
clone_agent	Clone an agent for safe experimentation

Write — Knowledge & Annotations

Tool	Description
create_annotation	Add FAQ annotation (guaranteed-correct Q&A)
update_annotation	Update an annotation's title, questions, or answer
delete_annotation	Delete an annotation
create_knowledge_document	Add a text document to the knowledge base
delete_knowledge_document	Delete a document from the knowledge base
import_knowledge_from_url	Import a web page into the knowledge base
import_knowledge_from_website	Crawl and import an entire website

Webhooks

Tool	Description
list_webhooks	List registered webhooks and subscribed events
create_webhook	Register a webhook for real-time event notifications
update_webhook	Update webhook active status or events
delete_webhook	Delete a webhook

Utility

Tool	Description
get_import_status	Check knowledge base import job progress

Test — Manual Verification

Tool	Description
send_test_message	Send a message and get agent response (supports multi-turn)

Test — Systematic Pipeline

The testing pipeline provides automated test generation, execution, evaluation, and analysis.

Quick start: Use start_test_pipeline for one-click end-to-end testing. Advanced: Use individual tools for fine-grained control.

Tool	Description
start_test_pipeline	Full pipeline: generate → execute → evaluate → analyze
list_pipeline_jobs	List past/running pipeline jobs
get_pipeline_job	Get pipeline job status and results
create_test_suite	Auto-generate test cases (6 categories)
list_test_suites	List test suites, filter by status
get_test_suite	Get test suite with all test cases
get_test_suite_progress	Lightweight progress polling during generation
add_test_case	Manually add a test case to a suite
create_test_run	Execute and evaluate a test suite
list_test_runs	List test run history
get_test_run	Get full run results (pass/fail, scores, responses)
create_run_analysis	AI-powered analysis of test results
get_run_analysis	Get analysis — suggestions and recommended instruction
analyze_failures	Phase 1: diagnose failures
generate_instruction	Phase 2: generate improved instruction from analysis
get_agent_test_overview	Agent testing summary — suites, runs, pass rate

Running Modes

1. HTTP MCP Server (Streamable HTTP)

Serves /mcp (Streamable HTTP, spec 2025-03-26) and /healthz. Compatible with Cursor Remote MCP, Claude Desktop, and any MCP-compliant client.

astra server
# or: go run ./cmd/main.go

2. Interactive CLI Agent

Chat-style interface with a built-in LLM that orchestrates all 64 tools autonomously.

export LLM_API_KEY=sk-...          # or OPENAI_API_KEY
export LLM_MODEL=gpt-4o            # optional, default: gpt-4o
export LLM_BASE_URL=https://...    # optional, for OpenRouter or other providers

astra
# or: go run ./cmd/cli/main.go

Commands: exit/quit/q to quit, clear/reset to start a new conversation.

OAuth Login (Browser)

The CLI supports browser-based OAuth login using PKCE (no client secret needed). After authorization, the access token is saved locally and used automatically for all subsequent tool calls.

Quick Login

# Login via browser (default: dev environment)
astra auth --oauth --env dev

# Login to production
astra auth --oauth --env prod

# Print authorize URL without opening browser
astra auth --oauth --env dev --no-open

How It Works

1. The CLI generates a PKCE code challenge (S256) and starts a local HTTP server on 127.0.0.1:8787/callback
2. Your browser opens the Astra authorization page
3. After you log in and approve, the browser redirects to the local server with an authorization code
4. The CLI exchanges the code for an access token (using the PKCE code verifier)
5. The token is saved to ~/.astra/oauth_token.json

Auth Flags

Flag	Default	Description
--oauth	—	Run browser-based OAuth PKCE flow
--token <tok>	—	Manually save an access token
--env	dev	Environment: prod, stage, dev, qa
--scope	chat,agents:read,agents:write	OAuth scopes
--listen-host	127.0.0.1	Callback server bind address
--listen-port	8787	Callback server port
--timeout	120	Seconds to wait for callback
--no-open	false	Don't open browser; print URL only

Auth Token Resolution

When making API calls, tokens are resolved in this order:

1. api_key argument passed explicitly in tool call
2. ASTRA_API_KEY environment variable
3. ASTRA_ACCESS_TOKEN environment variable
4. ASTRA_OAUTH_ACCESS_TOKEN environment variable
5. ~/.astra/oauth_token.json (saved by auth --oauth)

Existing API key workflows are fully compatible -- OAuth login is additive and does not break any existing authentication method.

Quick Start

go run ./cmd/main.go

# List all tools
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# List agents
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":2,"method":"tools/call",
    "params":{"name":"list_agents","arguments":{"api_key":"sk_live_xxx"}}
  }'

# Start full test pipeline
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":3,"method":"tools/call",
    "params":{"name":"start_test_pipeline","arguments":{
      "api_key":"sk_live_xxx","agent_id":"uuid","user_id":"tester"
    }}
  }'

# Generate improved instruction from test failures
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0","id":4,"method":"tools/call",
    "params":{"name":"generate_instruction","arguments":{
      "api_key":"sk_live_xxx",
      "instruction":"You are a helpful sales agent...",
      "improvement_suggestions":["Be more specific about pricing","Handle objections better"]
    }}
  }'

Docker Deployment

# Build
docker build -t astra-insight-mcp .

# Run (image entrypoint: astra server)
docker run -p 8080:8080 \
  -e ASTRA_GATEWAY_BASE_URL=https://astra-gateway.wati.io \
  -e ASTRA_API_KEY=sk_live_... \
  -e RATE_LIMIT_PER_MIN=60 \
  astra-insight-mcp

# Verify
curl http://localhost:8080/healthz
# → ok

# Initialize MCP session
curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
# → returns Mcp-Session-Id header

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: astra-insight-mcp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: astra-insight-mcp
  template:
    metadata:
      labels:
        app: astra-insight-mcp
    spec:
      containers:
        - name: mcp
          image: astra-insight-mcp:latest
          ports:
            - containerPort: 8080
          env:
            - name: ASTRA_GATEWAY_BASE_URL
              value: "https://astra-gateway.wati.io"
            - name: ASTRA_API_KEY
              valueFrom:
                secretKeyRef:
                  name: astra-secrets
                  key: api-key
            - name: RATE_LIMIT_PER_MIN
              value: "120"
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 30
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 10
          resources:
            requests:
              memory: "64Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: astra-insight-mcp
spec:
  selector:
    app: astra-insight-mcp
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

Authentication

Every tool call requires an auth token. The token is resolved automatically from multiple sources (highest priority first):

Priority	Source	Example
1	api_key argument in tool call	"api_key": "sk_live_xxx"
2	ASTRA_API_KEY env var	export ASTRA_API_KEY=sk_live_xxx
3	ASTRA_ACCESS_TOKEN env var	export ASTRA_ACCESS_TOKEN=astra_at_xxx
4	ASTRA_OAUTH_ACCESS_TOKEN env var	export ASTRA_OAUTH_ACCESS_TOKEN=astra_at_xxx
5	~/.astra/oauth_token.json file	{"access_token": "astra_at_xxx"}

Both API keys (sk_live_*) and OAuth access tokens (astra_at_*) are accepted -- the gateway treats them identically as Bearer tokens.

Using OAuth tokens

If you authenticate via the Astra OAuth flow (e.g. astra auth --oauth), the CLI writes a token file to ~/.astra/oauth_token.json. astra-insight-mcp will pick it up automatically -- no additional configuration needed.

# Option A: set an env var
export ASTRA_ACCESS_TOKEN=astra_at_xxx
astra

# Option B: use the token file (written by astra auth)
# ~/.astra/oauth_token.json -> {"access_token": "astra_at_xxx", ...}
astra
# -> token is auto-resolved, no env var needed

Environment Variables

MCP Server

Variable	Default	Description
PORT	8080	Server listen port
ASTRA_GATEWAY_BASE_URL	https://astra-gateway.wati.io	Gateway API base URL
ASTRA_API_KEY	--	Astra API key (sk_live_*), auto-injected into all tool calls
ASTRA_ACCESS_TOKEN	--	OAuth access token (astra_at_*), used if ASTRA_API_KEY is not set
ASTRA_OAUTH_ACCESS_TOKEN	--	Alternative OAuth env var name
RATE_LIMIT_PER_MIN	60	Max requests per IP per minute

CLI Agent (additional)

Variable	Default	Description
LLM_API_KEY / OPENAI_API_KEY	--	LLM API key (required for CLI)
LLM_BASE_URL	https://api.openai.com/v1	LLM API base URL (OpenAI, OpenRouter, etc.)
LLM_MODEL	anthropic/claude-sonnet-4	Model name