memFrame

memFrame brings a pandas-like DataFrame API to DuckDB, PostgreSQL, and ClickHouse — async-first, with an optional AI agent for natural-language data work.

Features

• Database-backed DataFrame API across DuckDB, PostgreSQL, and ClickHouse.
• Compiles every pandas-style call to backend-native SQL (DuckDB / PostgreSQL / ClickHouse) and runs it in-engine — your data never leaves the database.
• Async-first surface with sync equivalents for every operation.
• Upload from CSV, Parquet, or pandas DataFrame.
• Inspection, selection, cleaning, statistics, arithmetic, Plotly charts.
• Two-level cache: lineage audit + replayable result tables.
• Optional AI agent layer (memframe_ai) for chatting with your CSV.

Installation

pip install memframe              # core
uv add memframe                   # alt: uv
pip install "memframe[ai]"        # + Pydantic AI agent layer
uv add "memframe[ai]"             # alt: uv

Local development from this repository:

git clone https://github.com/Debojit95/memFrame.git
cd memFrame
pip install -e ".[dev,ai]"

Quick Start

import asyncio
import pandas as pd

from memframe import MemFrame


async def main():
    mf = MemFrame(
        connection_type="local",
        connection_params={"db_path": "memFrame.duckdb"},
    )
    await mf.aconnect()

    customers = await mf.aupload_df(
        pd.DataFrame(
            {
                "id": [101, 102, 103],
                "name": ["Alice", "Bob", "Charlie"],
                "score": [95.5, 82.0, None],
                "active": [True, False, True],
            }
        ),
        filename="customers",
    )

    preview = await customers.ahead(n=5)
    print(preview["result"])

    await mf.aclose()


asyncio.run(main())

Each upload returns a dataset context; chain inspection, selection, cleaning, arithmetic, statistics, and plotting on it. Sync methods drop the a prefix (head, iloc, fillna, …).

AI Agent

memframe_ai adds a Pydantic AI agent fleet on top of memFrame. After enabling it on the MemFrame instance, any dataset context can run natural-language queries that decompose into specialist tools and return typed response blocks.

import asyncio
import pandas as pd

from memframe import MemFrame


async def main():
    mf = MemFrame(
        connection_type="local",
        connection_params={"db_path": "memFrame.duckdb"},
    )
    await mf.aconnect()

    await mf.aenable_agent(
        provider="openai",
        model="gpt-5.5",
        api_key="sk-...",
    )

    ds = await mf.aupload_df(
        pd.DataFrame(
            {
                "name": ["Alice", "Bob", "Charlie"],
                "score": [95.5, 82.0, None],
            }
        ),
        filename="customers",
    )

    result = await ds.achat("fill null scores with the mean")
    print(result["answer"])
    print(result["plots"])  # any charts the agent built (id, title, spec_preview)

    await mf.aclose()


asyncio.run(main())

The agent supports OpenAI, Anthropic, Google, and Ollama. Pick a provider when you enable the agent:

await mf.aenable_agent(api_key="sk-...", provider="anthropic", model="claude-...")

Documentation

Full reference lives in docs/:

• Getting Started — connect, upload, first query.
• Connector & Connection — DuckDB / Postgres / ClickHouse wiring.
• Upload Manager — CSV / Parquet / DataFrame ingestion.
• Dataset Operations — table and active-dataset management.
• Inspection · Selection · Cleaning
• Statistics · Arithmetic
• Plotting · Caching

Serve locally:

mkdocs serve