dacli 0.3.0

dacli is a data-engineering agent for the terminal.

It plans work, runs governed tool calls, and verifies results against the data systems it touches.

Quick start · Architecture · Connectors · Governance · Evaluation

_{And the real terminal, offline — no API key, no network. Crisp vector version: assets/demo.svg.}

Table of contents

• Design
• Highlights
• How it works
• Supported platforms
• Installation
• Configuration
• Quick start
• Command reference
• Reliability: the environment is the oracle
• Project layout
• Extending dacli
• Testing
• Documentation
• Status
• Contributing
• License

_{35-second tour — the thesis, the governed tier ramp, pass^k, and the connectors.}

Design

dacli is not a SQL chatbot. It turns a plain-language data task into a governed plan, runs connector operations or sandboxed code, and accepts a result only after the target system confirms it.

The hard problem with autonomous data agents is not capability, it is reliability. A 95%-reliable DROP guard is a catastrophe waiting for its 1-in-20. dacli is built on a simple thesis, drawn from "From Model Scaling to System Scaling":

Agent reliability is a property of the system around the model, not the model. Stale memory, diluted context, unchecked tool output, and missing governance are system failures that survive every model upgrade.

That thesis shows up as a six-component harness (ℛ Reasoning · ℳ Memory · 𝒞 Context · 𝒮 Skills · 𝒪 Orchestration · 𝒢 Governance). "The environment is the oracle" means verification and rollback are anchored to native platform features (transactions, Time Travel/UNDROP, EXPLAIN/dry_run, zero-copy clones, dbt test, row counts), not the model's own say-so.

dacli does not depend on LangChain or LangGraph. The core is MCP-free: tools are plain Python/CLI calls composed by the agent, with the same governed dispatch path for direct tools and sandboxed code. An opt-in MCP client bridge can proxy one external MCP server through that path; the core itself never speaks MCP.

Highlights

How it works

Every state-changing action flows through one governed dispatch path — so the tool tier and the code-execution sandbox are governed and verified identically. The diagram is rendered from assets/architecture.mmd; see docs/ARCHITECTURE.md for the full design.

Supported platforms

dacli ships 14 platform connectors plus a diagram-as-code skill. Each connector declares its operations, risk tiers, environment-anchored post-conditions, and a native rollback primitive (the "Definition of Done", enforced in CI — see docs/CONNECTORS.md).

CLI-first connectors (BigQuery, Databricks, dbt, S3, GCS, Postgres, MySQL, MongoDB, DynamoDB) shell out to the platform's first-class CLI rather than bundling SDKs — install the CLIs you actually use.

Installation

Requirements: Python 3.10+ (CI runs 3.10–3.12).

End users — an isolated, on-PATH install:

pipx install dacli          # or: uv tool install dacli

Embedding dacli as a library:

pip install dacli

Not on PyPI yet. Until the first release lands, install from source — after that, pipx/uv is the one-line path.

First 2 minutes

pipx install dacli
dacli                                  # bootstraps your LLM key (encrypted, no .env needed)
> ask it something about your data
dacli plan "drop table prod.users"     # see the guardrails — zero risk, no creds

plan (and eval, diff) run fully offline: no API key, no network, no credentials. They're the zero-risk way to see what dacli does before you wire up a single connector.

From source

git clone https://github.com/mouadja02/dacli.git
cd dacli

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

pip install -e .            # one command — dependencies come from pyproject.toml

Optional extras:

pip install -e ".[all]"     # + every Python-SDK connector (snowflake, pinecone)
pip install -e ".[dev]"     # contributors: pytest, ruff, vulture
pip install -e ".[mcp]"     # the opt-in MCP client bridge
pip install -e ".[pty]"     # faithful TTY for the governed terminal

Use the editable install (-e). A plain pip install . copies the sources into site-packages, so the dacli command then runs that frozen copy and silently diverges from your working tree as you edit.

For a byte-for-byte reproducible environment (what CI uses), install the pinned closure instead: pip install -r requirements.lock && pip install -e . --no-deps.

For CLI-first connectors, install the relevant platform CLIs and authenticate them as you normally would:

Something off? run dacli doctor — it reports where config, state, and logs resolve, your LLM key's source (never the value), governance/sandbox posture, and per-connector health.

Configuration

There is nothing to configure by hand. The first dacli run launches the setup wizard: pick a provider and model, paste an API key, choose connectors — secrets are encrypted at rest into .dacli/dacli.json (see the security model). No config.yaml, no .env needed.

dacli        # first run → wizard → chatting

Advanced: file-based configuration (power users & CI)

dacli also reads a config.yaml (searched at the project root, then the per-user config dir) and substitutes secrets from environment variables (${VAR} placeholders), which you can supply via a .env file. Credentials never live in the config file.

dacli init                               # write a commented config.yaml to start from
cp .env.example .env                     # fill in your secrets

# .env
LLM_API_KEY=...
GITHUB_TOKEN=...
SNOWFLAKE_PASSWORD=...
PINECONE_API_KEY=...
OPENAI_API_KEY=...        # used for Pinecone embeddings

config.yaml, .env, and the wizard-generated config/connectors.yaml are git-ignored. Full reference: docs/CONFIGURATION.md.

State durability & sessions. Session state, encrypted secrets, memory, and usage live under .dacli/ and are written atomically (temp file → fsync → os.replace), so a crash or Ctrl-C mid-write can never truncate or wipe a state file — a reader always sees the complete old or new file. There is no cross-process lock, so run one dacli session per project directory at a time; two sessions sharing the same .dacli/ can overwrite each other's last-write-wins state.

Security model

Secrets entered through the wizard or /connect are encrypted at rest with Fernet; the encryption key lives in .dacli/.key (git-ignored), next to the ciphertext in .dacli/dacli.json. Be clear about what that buys you:

• Protects against: accidentally committing secrets to git, and casual inspection (screen-shares, shoulder-surfing, grepping a backup of the repo).
• Does not protect against: a local attacker with filesystem access — the key and the ciphertext are co-located, so anyone who can read one can read both. The chmod 600 on the key file is best-effort and a no-op on Windows.

If you want the key off-disk, set the DACLI_ENCRYPTION_KEY environment variable (e.g. from a secret manager); it takes priority over .dacli/.key and accepts either a raw Fernet key or a password (derived into one via PBKDF2). This is appropriate for a local single-user tool; it is not a substitute for an OS keychain or a vault.

Quick start

dacli                 # first run launches the setup wizard, then drops into chat
dacli setup           # (re)configure which connectors/operations are enabled
dacli validate        # live-test every enabled connector's credentials
dacli eval --quick    # run the offline reliability suite (pass^k) against simulated platforms

# …or without installing the command:
python run.py

On first run the setup wizard walks you through provider, model, and API key, then asks which connectors to enable — validating each with a live health check. Then just describe what you want:

"Stand up a Bronze→Silver pipeline for the CRM source in Snowflake, then run the dbt models and confirm every test passes."

dacli decomposes the goal into an inspectable plan, asks for approval where the blast radius warrants it, executes step by step, and verifies each step against the platform before moving on.

Worked example. examples/warehouse-snowflake/ builds a Bronze → Silver → Gold warehouse in Snowflake from two dirty CSVs — raw load, dedup/conform, analytics marts, a data-quality gate, and a Time Travel rollback. No Snowflake account? The same folder replays the governed plan offline (dacli replay examples/warehouse-snowflake/scenario.json).

Command reference

CLI subcommands

In-chat slash commands

/help · /keys · /init · /status · /doctor · /usage · /context · /audit · /why-failed [dag] · /tools · /connect [tool] · /new-connector · /testmode [tool] · /import-connector · /push-connector · /debug-connector <name> · /setup · /history · /find <text> · /last-error · /expand <id> · /transcript · /sessions · /catalog [connector] · /schema <object> · /load <id> · /export · /config · /theme <name> · /prompt · /clear · /cls · /reset · /exit

Reliability: the environment is the oracle

dacli refuses to ask the model "did that work?" — it asks the platform. This shows up everywhere:

• Pre-conditions — EXPLAIN, BigQuery dry_run, dbt compile validate before anything runs.
• Post-conditions — a CREATE is confirmed by bq show; a put is confirmed by head-object; a dbt run is confirmed by run_results.json. Fluent success is never accepted as proof.
• Rollback — irreversible actions are blocked unless a native undo path is verified to exist (versioning enabled, retention window open, snapshot taken) — not merely assumed.
• Cost as blast radius — set governance.cost_confirm_usd and any action whose connector-estimated cost (e.g. BigQuery dry_run bytes) exceeds it requires a human confirm, with the estimate shown in the approval panel.
• pass^k — reliability is measured as success across k repeated rollouts, not a single lucky run. The destructive-action gate is held to the highest bar.

$ dacli eval --quick
Reliability dashboard — suite: sim
----------------------------------------------------------------------------------------------
connector          tasks  pass@1  pass^k   succ    esc   corr    gov  unguard     tok       ms
----------------------------------------------------------------------------------------------
bigquery               3    1.00    1.00   1.00   0.00   0.00   0.00        0       0      0.1
s3                     3    1.00    1.00   1.00   0.00   0.00   0.00        0       0      0.1
shell                  7    1.00    1.00   1.00   0.00   0.00   0.29        0       0     19.5
spine                  5    1.00    1.00   1.00   0.00   0.20   0.20        0       0      1.0
...
OVERALL               34    1.00    1.00   1.00   0.00   0.03   0.09        0       0     11.0
----------------------------------------------------------------------------------------------
✓ zero unguarded destructive executions.

Details: docs/GOVERNANCE.md and docs/EVALUATION.md.

Project layout

dacli/
├── scripts/cli.py        # CLI entry point (the `dacli` command)
├── core/                 # 𝒪 orchestration: kernel, planner (DAG), plan-act-observe-verify loop,
│                         #   blackboard, sub-agents, memory facade, pricing/usage, store
├── reasoning/            # ℛ multi-provider LLM client + cheap/strong model router
├── context/              # 𝒞 context constructor: assembler, budget, compaction, disclosure, spill
├── memory/               # ℳ trust-aware store, retrieval (staleness), catalog cache, episodic/procedural
├── connectors/           # 𝒮 microkernel plugin layer
│   ├── base.py           #   Connector ABC · OperationSpec · ToolResult · Risk
│   ├── registry.py       #   manifest discovery + tool definitions + resolver
│   ├── dispatcher.py     #   one governed dispatch path (verify + audit)
│   ├── dod.py            #   Definition-of-Done gate (CI-enforced)
│   └── <platform>/       #   connector.py + manifest.yaml + SKILL.md (×14)
├── governance/           # 𝒢 classifier, policy engine, permissions, rollback, shadow, audit ledger
├── sandbox/              # governed code-execution runtime (results stay on disk)
├── skills/               # contracted skills (e.g. diagram_mermaid) with mandatory post-conditions
├── eval/                 # pass^k harness, simulated platforms, golden suites, regression, dashboard
├── prompts/              # system message + guidelines + loaders
├── config/               # typed pydantic settings + env-var substitution
└── tui/                  # themed terminal UI

Extending dacli

Adding a platform never touches core/, reasoning/, or governance/. Drop a folder:

connectors/myplatform/
├── connector.py     # subclass Connector: operations(), invoke(), health() (+ verify_rollback for irreversible ops)
├── manifest.yaml    # id, class, required_config, default_scope, golden_task
└── SKILL.md         # progressive-disclosure doc

A connector only ships when it passes the Definition of Done (enforced by CI): operations with JSON schemas, ≥1 environment-anchored post-condition per mutating op, a registered rollback strategy, an introspection op, a least-privilege scope, and a verifiable golden task. Full guide: docs/CONNECTORS.md.

Ecosystem surfaces

• Install shared connectors — dacli connector install <name> --index <path-or-url> fetches a connector from a community index, validates it in a sandboxed subprocess, and registers it disabled; enable it with /connect <name> and a restart.
• Bridge an MCP server — the opt-in mcp_bridge connector (pip install -e ".[mcp]" + an mcp: config section) proxies one external MCP server's tools through the same classify → policy → audit path as every native tool; proxied tools default to the conservative risky tier.
• Gate your own CI — the .github/actions/dacli-gate composite action replays a scenario file headlessly (dacli replay) and fails the build on a non-zero exit (2 = governance block). See scenarios/ci_governance_gate.json for a scripted, secret-free example.
• Export a run for compliance — dacli export-run zips a session's transcript, audit-ledger slice and usage summary, with secret-keyed values redacted.

Testing

# Full suite (pytest — `unittest discover` silently skips the bare-function tests)
pytest tests -q

# Connector Definition-of-Done gate (governance debt guard)
python -m unittest tests.test_connector_dod

# Offline reliability suite (pass^k) against simulated platforms
python -m dacli.eval --quick

# Docs drift gate (test badge / eval sample / command reference vs. reality)
python tools/check_docs.py

CI runs ruff, the DoD gate, the full suite (Python 3.10–3.12), the pass^k sim suite, a headless end-to-end smoke, and the docs drift gate on every pull request.

Documentation

Status

dacli is actively developed, with all six harness components implemented and exercised by the test suite:

Contributing

Contributions are welcome — see CONTRIBUTING.md. The one rule that is non-negotiable: scale skills and governance together. Every new capability ships with its post-conditions, rollback strategy, permission scope, and golden task, or it does not ship — and CI enforces it.

License

This project was created by Mouad Jaouhari. If a LICENSE file is not yet present in the repository, please contact the author before reuse or redistribution.