Skip to main content

ayga_parser CLI Tool with Redis and HTTP API support

Project description

ayga-cli

Data access client — fetch data from configured sources via a Redis Wrapper server. For humans, AI agents, and automation scripts.

For AI Agents: Read CONTEXT.md first — it contains Rules of Engagement, output format, exit codes, and usage examples optimised for LLM consumption. There's also a SKILL.md for skill-based agent frameworks.

Python 3.10+ License: MIT

Philosophy

You ask for data by source name and query — the server handles parsers, proxies, and infrastructure. You do not need to know what runs behind the source, and the CLI never exposes that detail: no parser names, no proxy configuration, no backend engine branding in the interface.

Features

  • Source-based interface: get <source> <query> — no backend/parser knowledge required
  • Redis transport: async queue-based communication with the backend server
  • MCP server: 2-tool MCP server (fetch_data, list_sources) for AI agent integration
  • Modern CLI: Typer + Rich for readable output, --json/--stream for machine consumption
  • Secure: OS keyring for password storage
  • Tested: 195 passing tests (5 skipped)

Installation

# Clone and install
cd ayga-cli
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[mcp]"

The [mcp] extra is only needed if you plan to run the MCP server (ayga_parser-mcp).

Authentication

ayga-cli talks to a hosted backend — you need an API key (AYGA_PASSWORD) to use it. Get your API key from https://redis.ayga.tech or contact support@ayga.tech.

Configuration

# Interactive setup
ayga_parser config init

# Or set via environment variables (AYGA_ prefix, case-insensitive)
export AYGA_HTTP_URL="http://127.0.0.1:9091/API"
export AYGA_REDIS_HOST="127.0.0.1"
export AYGA_REDIS_PORT="6379"
export AYGA_PASSWORD="your_password"
  • Config files, presets, and manifest cache share one canonical config directory:
    • Windows: %APPDATA%\ayga-cli
    • macOS: ~/Library/Application Support/ayga-cli
    • Linux: ~/.config/ayga-cli
  • Settings can also be dropped into a .env file inside that config directory — it's loaded automatically.
  • HTTP requests can use both the API password payload and HTTP Basic Auth. Set AYGA_HTTP_BASIC_USERNAME and optionally AYGA_HTTP_BASIC_PASSWORD if your backend requires explicit Basic Auth credentials.
  • View the resolved configuration at any time with ayga_parser config show.

CLI Usage

Quick start — the get/sources interface

This is the primary, backend-agnostic interface. Discover what's available, then fetch data by source name:

# 1. See what data sources are configured on this server
ayga_parser sources list

# 2. Inspect a source's schema (fields, description, examples) before using it
ayga_parser sources info web-search

# 3. Fetch data
ayga_parser get web-search "machine learning 2025"

# 4. Limit the response to specific fields (recommended for agent contexts)
ayga_parser get web-search "machine learning 2025" --fields title,url,snippet

# 5. Stream results as NDJSON — one record per line
ayga_parser get web-search "Python tutorials" --stream --fields title,url

# 6. Get structured JSON output
ayga_parser get ai-answer "What is quantum computing?" --json

# 7. Preview what would be sent, without executing
ayga_parser get web-search "test" --dry-run

# 8. Custom timeout (default: 300s)
ayga_parser get ai-answer "explain entropy" --timeout 120

sources list and sources info cache results locally; pass --no-cache to force a fresh fetch from the server.

Helper commands (+verb)

For multi-step operations that combine more than one source call. Each +verb is a command group with a single subcommand of the same name:

# Fetch a URL and return clean Markdown (uses the 'article' source internally)
ayga_parser +extract extract https://example.com/article

# Combine web-search + ai-answer for a topic
ayga_parser +research research "quantum computing"

Use a helper only when a single get call isn't enough — if one source covers it, call get directly.

Connectivity and diagnostics

# Test HTTP connectivity to the configured backend
ayga_parser ping

# Show the resolved configuration
ayga_parser config show

Lower-level parser/preset management

ayga-cli also ships lower-level command groups (parsers, presets) for managing the underlying parser manifest and reusable option presets. These are for operators configuring the backend, not for the day-to-day get/sources workflow described above:

ayga_parser parsers list
ayga_parser parsers list --category SE
ayga_parser presets list

See ayga_parser parsers --help and ayga_parser presets --help for details.

Exit Codes

Machine consumers should check the exit code rather than assuming success:

Code Meaning Action
0 Success — data in stdout Parse stdout as JSON
1 General/unknown error Check stderr
2 Timeout — server did not respond in time Retry or increase --timeout
3 Source not found Run sources list to see available sources
4 Server unavailable — Redis Wrapper unreachable Report infrastructure issue, do not retry in a loop
5 Invalid input — bad source name or empty query Fix the input

Error details are always written to stderr; stdout is reserved for clean data output.

MCP Server

Run the MCP server for AI agent integration:

# Start MCP server (requires the [mcp] extra)
ayga_parser-mcp

# Or via Python
python -m ayga_cli.mcp.server

MCP Tools

Tool Purpose
list_sources() List available data sources
fetch_data(source, query, timeout=300) Fetch data from a named source

Example MCP usage

# Discover available sources
sources = await mcp.list_sources()

# Fetch data from a source
result = await mcp.fetch_data(source="web-search", query="machine learning 2025")
# Returns: {"success": true, "status": "completed", "result": {...}}

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Human     │     │  AI Agent   │     │    Cron     │
│   (CLI)     │     │   (MCP)     │     │   (Job)     │
└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
       │                   │                   │
       └───────────┬───────┴───────────────────┘
                    │
            ┌───────▼────────┐
            │   ayga-cli      │
            │  ├─ get/sources │
            │  ├─ Redis Client│
            │  └─ MCP Server  │
            └───────┬─────────┘
                    │
            ┌───────▼────────┐
            │  Redis Wrapper │
            │   (backend)    │
            └────────────────┘

ayga-cli only talks to the Redis Wrapper server. What the server does internally (parsers, proxies, presets) is its own concern — not something callers of get/sources need to know.

Testing

# Run all tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=ayga_cli --cov-report=html

# Specific test file
pytest tests/test_commands/test_get.py -v

Current status: 195 passed, 5 skipped.

License

MIT License — see the license field in pyproject.toml.

Credits


Built for AYGA

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ayga_cli-0.1.0.tar.gz (69.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ayga_cli-0.1.0-py3-none-any.whl (68.3 kB view details)

Uploaded Python 3

File details

Details for the file ayga_cli-0.1.0.tar.gz.

File metadata

  • Download URL: ayga_cli-0.1.0.tar.gz
  • Upload date:
  • Size: 69.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.3

File hashes

Hashes for ayga_cli-0.1.0.tar.gz
Algorithm Hash digest
SHA256 15b025ceea5bf2a111a17978433655055dd67d09b100cb00be11de92522fa688
MD5 d6cf3630c0d45d4a5d07c7ea75175640
BLAKE2b-256 1b833e6624104a6d1d6ed2c9206e037799c6d2a8183abf4dcf723a649c821fd9

See more details on using hashes here.

File details

Details for the file ayga_cli-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: ayga_cli-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 68.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.3

File hashes

Hashes for ayga_cli-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0463ab42b92be2ee59f170b90442c349a0a55a96d099bbbc457f8586330369ef
MD5 49e517bef2371d1541e2e8ad9637b525
BLAKE2b-256 72048b2104fcfadbed853e394ea24045105859b6bf80121cc66b5ea1ecf4b894

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page