DataMind

Store at inference time. Retrieve with evidence.
A shared data plane for agents — writable during the conversation, useful on the very next question.

Quick start · Tutorial · API · 中文

_{StoreAgent writes on the warm path. RetrieveAgent reads across the shared data plane and returns evidence.}

v1.0.0 — stable native backend + local profile storage. SDK/CCR and remote database adapters are supported integration paths; validate them in your own environment.

The idea

Most agent systems can retrieve knowledge, but they have nowhere to put the new fact they just learned. DataMind gives the runtime two explicit roles:

message / file / CSV / relationship
                 │
                 ▼
           StoreAgent  ─────── write receipt ───────▶  KB · DB · Graph · Skills · Memory
                 │
                 │  next question
                 ▼
           RetrieveAgent  ◀──── evidence + answer ────  shared data plane

This is inference-time data: not model training data, not a batch ETL pipeline, and not an unbounded chat transcript. It is scoped, inspectable state that can change while an agent is running.

Two agents. One hard boundary.

The split is enforced in code, before tools reach the model. RetrieveAgent sees 19 read/utility tools; StoreAgent sees 11 write tools. Every call also passes through PathAllowlistHook, DestructiveSqlHook, and AuditLogHook.

Five surfaces, one answer

• KB / RAG — documents, notes, policies, semantic search
• Database — exact numbers, filters, joins, aggregations
• Knowledge Graph — entities, relationships, multi-hop facts
• Skills — reusable procedures and safe utilities
• Memory — preferences and durable facts, scoped to global, profile, or session

Default providers are Chroma + BM25, SQLAlchemy (SQLite / MySQL / PostgreSQL), NetworkX, profile-scoped SKILL.md, and SQLite memory.

Quick start

pip install datamind

export DATAMIND__LLM__API_BASE=https://your-gateway.example.com
export DATAMIND__LLM__API_KEY=sk-...
export DATAMIND__LLM__PROTOCOL=anthropic   # or openai_chat_completions
export DATAMIND__LLM__MODEL=claude-sonnet-4-6

datamind chat

Or open the local UI:

python -m uvicorn datamind.server:app --port 8000
# http://127.0.0.1:8000

The protocol is explicit and shared by the outer loop and internal generation (NL2SQL, multi-query retrieval, Memory, and graph extraction).

pip install 'datamind[mysql]'
pip install 'datamind[postgres]'
pip install 'datamind[voyage]'
pip install 'datamind[huggingface]'
pip install 'datamind[dev]'

A 60-second end-to-end demo

git clone https://github.com/OpenDCAI/DataMind.git
cd DataMind
python -m venv .venv && source .venv/bin/activate
pip install -e .

cp .env.datamind.example .env.datamind
$EDITOR .env.datamind              # set DATAMIND__LLM__API_KEY

python -m datamind.scripts.hello_sdk
python -m datamind.scripts.seed_enterprise_demo
DATAMIND__DATA__PROFILE=enterprise_demo \
  python -m datamind.scripts.hello_enterprise

The bundled dataset contains 17 documents, 64 graph nodes, 6 tables, and 101 rows. To use the browser UI, run:

DATAMIND__DATA__PROFILE=enterprise_demo \
  python -m uvicorn datamind.server:app --port 8000

Drop in .md, .csv, or .txt, ask a question, and watch the role-scoped tools work. The full walkthrough is in GETTING_STARTED.md.

Data can change during the conversation

you            → "Import sales-q2.csv as table q2_sales"
StoreAgent     → db_import_csv(...) → write receipt
you            → "Which sales rep has the largest Q2 pipeline?"
RetrieveAgent  → db_query_sql(...)  → answer + table evidence

The same flow works for a document, a graph fact, or a profile skill.

Choose your runtime

The built-in native loop is the stable default. The optional sdk loop adds Claude Agent SDK features such as Subagents and Compaction.

Set both switches explicitly:

DATAMIND__AGENT__BACKEND=native
DATAMIND__LLM__PROTOCOL=anthropic

Read the complete native / SDK support matrix. For SDK + OpenAI-compatible gateways, CCR is the local Anthropic ↔ OpenAI protocol bridge.

Python and HTTP APIs

from datamind.agent import build_datamind
from datamind.config import Settings

async def answer() -> str:
    system = await build_datamind(Settings())
    try:
        await system.ingest("Remember that weekly reports use Chinese.")
        result = await system.query("What language should weekly reports use?")
        return result["answer"]
    finally:
        await system.aclose()

The bundled FastAPI server exposes GET /api/health, GET /api/tools, POST /api/ask, POST /api/store, POST /api/chat (SSE), and POST /api/upload. See the stable API contract.

Safe to embed, not safe to expose naked

DataMind expects your authentication and authorization layer at the edge. Before deploying publicly:

• bind local deployments to loopback;
• add authentication, authorization, TLS, and rate limits;
• isolate profile/storage directories and upload paths;
• treat evidence provenance as metadata, never as a permission grant.

See public deployment security boundaries.

Verify locally

pytest
python -m datamind.scripts.verify_sqlite_demo

The repository's CI runs the no-network test suite and the deterministic SQLite demo. Benchmark and checkpoint/resume details live in docs/BENCHMARK_RUNNER.md.

Explore the docs

More architecture notes and tutorials are available in DataMind-Doc. The supported v1.x package lives under datamind/; the original v0.1 prototype remains in-tree for comparison.

License

DataMind is released under the Apache License 2.0.