zleap-sag 0.7.0

Local-first memory & knowledge engine for AI agents

zleap-sag

Local-first memory & knowledge engine for AI agents. Ingest documents, extract an event/entity graph with an LLM, and retrieve over it — fully local by default, progressive to production databases.

Distribution zleap-sag · import zleap.sag · Python ≥ 3.11 · MIT

Highlights

• Zero-infra by default — embedded SQLite + LanceDB (built-in BM25), no services to run. Data lives in ./.zleap/.
• Progressive to production — swap in MySQL / PostgreSQL / OceanBase (relational) and Elasticsearch / pgvector (vector) by changing config only; pipeline code is unchanged.
• Single-database option — run the whole engine on one PostgreSQL (pgvector) or one OceanBase instance (SQL + vector).
• Tiny API — DataEngine orchestrates ingest → extract → search; injected EngineConfig; typed results; one SagError base.
• Multiple retrieval strategies — vector, atomic, and multi-hop (multi) over the event-entity graph.

Quick Start

1. Install

pip install zleap-sag        # runs as-is: embedded SQLite + LanceDB, no services needed

Add an extra only when you switch that backend on (quote the brackets in shells):

pip install "zleap-sag[es]"        # Elasticsearch            (vector_provider="es")
pip install "zleap-sag[mysql]"     # MySQL or OceanBase       (aiomysql driver)
pip install "zleap-sag[postgres]"  # PostgreSQL, incl. pgvector single-DB
pip install "zleap-sag[all]"       # all backends + Alembic migrations + litellm

2. Run

Point it at any OpenAI-compatible LLM + embedding endpoint. Storage defaults to ./.zleap/ (SQLite + LanceDB); the schema is created automatically on first run.

import asyncio
from zleap.sag import DataEngine, EngineConfig
from zleap.sag.config import LLMConfig, EmbeddingConfig


async def main():
    config = EngineConfig(
        llm=LLMConfig(api_key="sk-...", base_url="https://your-gateway/v1", model="qwen3.6-flash"),
        embedding=EmbeddingConfig(model="bge-large-en-v1.5"),
    )
    async with DataEngine(config) as engine:      # start(): create local tables on first run
        await engine.ingest("your_document.md")   # parse → chunk → embed
        await engine.extract()                     # LLM → event/entity graph
        result = await engine.search("Who founded Acme?", strategy="multi", top_k=5)
        for section in result.sections:
            print(section.get("content", "")[:200])


asyncio.run(main())

Storage lands in ./.zleap/ — add it to your .gitignore. Runnable scripts: examples/.

Configuration

Two ways to configure — pick one

zleap-sag always builds a single EngineConfig. Provide it by parameter injection or environment variables — they are alternatives, not layered.

Aspect	Parameter injection	Environment variables
Call	EngineConfig(llm=LLMConfig(...), ...)	EngineConfig.from_env()
Values come from	explicit Python arguments	OPENAI_API_KEY, LLM_MODEL, … (or a .env file)
Pass keys in code?	yes — every value	no — read from the environment
Best for	notebooks, embedding in an app	containers / 12-factor deployments

Does setting env vars remove the need to pass keys? Only if you call EngineConfig.from_env(). Plain EngineConfig(...) never reads the environment — you pass keys there explicitly. Don't mix the two: use from_env() or inject params.

Environment-variable path, minimal set (zero-infra):

export OPENAI_API_KEY=sk-...
export OPENAI_BASE_URL=https://your-gateway/v1     # optional; defaults to OpenAI
export LLM_MODEL=qwen3.6-flash
export EMBEDDING_MODEL=bge-large-en-v1.5

config = EngineConfig.from_env()          # or EngineConfig.from_env(env_file=".env")

Required vs optional

• Required: llm and embedding — each an OpenAI-compatible endpoint.
• Optional (all default to the local stack): storage backend, data_dir (./.zleap), rerank, language, log_level.

Two gotchas

• Separate embedding endpoint? Set EmbeddingConfig(base_url=..., api_key=...) (or EMBEDDING_BASE_URL / EMBEDDING_API_KEY). If omitted, embedding reuses the LLM's key and URL.
• Don't set EmbeddingConfig(dimensions=...) unless your model supports it — many embedding models reject a dimension override.

Full variable reference (storage, backends, rerank): .env.example.

Storage backends

Switch backends by changing EngineConfig only — the ingest/extract/search code is identical. Spin up local backends for testing with make up (docker compose).

Deployment	Relational	Vector	Extra	Schema init
Local (default)	SQLite	LanceDB	—	automatic on start()
Production	MySQL / PostgreSQL / OceanBase	Elasticsearch	[mysql] / [postgres] / [es]	init_schema() once
Single DB	PostgreSQL	pgvector	[postgres]	init_schema() once
Single DB	OceanBase ≥ 4.3.3	oceanbase	[mysql]	init_schema() once

Initialization

• Local SQLite — start() creates the schema and seeds default entity types automatically. Nothing to call.
• Production backends — call await engine.init_schema() once (idempotent: adds missing tables, never drops). For versioned migrations use the [migrations] extra (Alembic).

Example — MySQL + Elasticsearch

from zleap.sag import DataEngine
from zleap.sag.config import EngineConfig, RelationalConfig, ESConfig, LLMConfig, EmbeddingConfig

config = EngineConfig(
    relational=RelationalConfig(
        provider="mysql", host="localhost", user="sag2", password="sag2", database="sag2"
    ),
    vector_provider="es",
    es=ESConfig(hosts=["http://localhost:9200"]),
    llm=LLMConfig(api_key="sk-...", base_url="https://your-gateway/v1", model="qwen3.6-flash"),
    embedding=EmbeddingConfig(model="bge-large-en-v1.5"),
)

async with DataEngine(config) as engine:
    await engine.init_schema()          # once, for production backends
    await engine.ingest("your_document.md")
    await engine.extract()
    result = await engine.search("Who founded Acme?", strategy="multi", top_k=5)

Example — single database

# One PostgreSQL for relational + vector (pip install "zleap-sag[postgres]")
config = EngineConfig(
    relational=RelationalConfig(provider="postgres", host="localhost", port=5432,
                                user="sag2", password="sag2", database="sag2"),
    vector_provider="pgvector",
    llm=LLMConfig(...), embedding=EmbeddingConfig(...),
)

# One OceanBase for SQL + vector (pip install "zleap-sag[mysql]", OceanBase ≥ 4.3.3)
config = EngineConfig(
    relational=RelationalConfig(provider="oceanbase", host="localhost", port=2881,
                                user="root", password="", database="sag2"),
    vector_provider="oceanbase",
    llm=LLMConfig(...), embedding=EmbeddingConfig(...),
)

OceanBase ANN indexes need the tenant setting ob_vector_memory_limit_percentage > 0; if unset, the engine falls back to exact vector search automatically.

How it works

One DataEngine instance binds one data source. The three steps chain automatically — the source_config_id returned by ingest() is reused by extract() and search():

• ingest(path) — parse a document → chunk → embed → persist chunks and vectors. Takes a file path (single document).
• extract() — an LLM extracts an event/entity graph from the last ingest's chunks.
• search(query, strategy=, top_k=) — retrieve. Strategies: vector, atomic, multi (default), multi1, hopllm, multi_es.

DataEngine is an async context manager: async with DataEngine(config) as engine runs start() on enter and aclose() on exit. Result types: from zleap.sag.results import IngestResult, ExtractResult, SearchResult, ChunkResult. To chunk a raw string without persisting (no start() / DB needed), use DataEngine.chunk().

Notes

• One config per process for now — core connections are process-global; run one EngineConfig per process.
• multi_es needs a real-BM25 vector backend (LanceDB or Elasticsearch); on pgvector it raises MissingCapabilityError rather than degrading silently.
• All engine errors derive from SagError — catch it at the boundary.

Links

Examples · Changelog · Contributing · Config reference