pydantic-ai-pixeltable

Pydantic AI Harness integration for Pixeltable. Two independent layers:

• PixeltableMemoryStore: persist Harness Memory in a Pixeltable table (same Memory(store) slot as FileStore)
• Pixeltable: read-only catalog tools (list_tables, describe_table, query_table, similarity_search) over tables you already have

Neither requires the other. Requires Pixeltable >= 0.7.8 and pydantic-ai-harness >= 0.29.0.

This is an interoperability bridge. Native Pixeltable agents still use a TableModel and computed columns.

How it fits together

Native backend (FileStore)                 This package (Pixeltable)

Memory(FileStore)                          Memory(PixeltableMemoryStore)
  └ .agent-memory/                           └ pxt table 'harness.memory'
      ├ main/MEMORY.md                           ├ kind='file' rows  path, content, version
      ├ ... (one .md per memory path)            └ kind='op' rows    '__op__/<id>' receipts
      └ .memory-store.sqlite3                  same catalog as your data; inspect via store.table
          (versions + operation journal)
                                           Pixeltable(tables=[...])  (optional, independent)
                                             └ your existing tables/views, incl. pxt.Document and
                                               embedding indexes -> query_table, similarity_search

Unique to this backend: memory rows are ordinary catalog rows. store.search stays lexical (the Harness contract), but through store.table you can add an embedding index on content and run Pixeltable similarity queries over memory, or join it with the rest of the catalog.

Installation

pip install pydantic-ai-pixeltable

The examples use an Anthropic model; add its client with pip install "pydantic-ai-slim[anthropic]".

Quick Start

Memory in the catalog, plus read-only tools over the tables you already have:

from pydantic_ai import Agent
from pydantic_ai_harness import Memory, ToolOutputLimits
from pydantic_ai_pixeltable import Pixeltable, PixeltableMemoryStore

agent = Agent(
    "anthropic:claude-sonnet-4-6",
    capabilities=[
        Memory(PixeltableMemoryStore(table_name="harness.memory")),
        Pixeltable(tables=["my_app.doc_chunks"]),
        ToolOutputLimits(),
    ],
)

Prefer FileStore (or PostgresMemoryStore) when the notebook does not need to live in the catalog; the choice is per project.

tables is a required allowlist of table paths or directory prefixes. Pass tables=["*"] to allow the whole catalog, including the memory table and its __op__ receipt payloads.

Two Pixeltable(...) instances share id="pixeltable" and merge by narrowing: an entry survives only when every capability covers it, and a disjoint merge raises. A capability passed to a single run replaces the agent's rather than merging with it, so it can widen the allowlist. pydantic-ai-chdb also registers list_tables and describe_table; wrap one side in PrefixTools when you pair them.

Catalog tools

Pixeltable does not create tables or insert rows. Point it at a table or view that already has data, plus an embedding index for similarity_search.

• query_table filters with equality only ({"status": "open"}); timestamp, date, and UUID values are ISO strings. Media, array, and binary columns reject non-null filters, and computed columns that are not stored reject all filters.
• similarity_search calls column.similarity(string=query) and needs an embedding index on that column.
• Output is bounded by max_rows and max_chars: an oversized value is cut (ending in ...) rather than dropping its row, and the minimal {"table", "rows", "truncated"} envelope is always returned, even when max_chars is set below its size. Default columns skip media, array, binary, and computed columns that are not stored (they would recompute per query); a named media column returns a file URL, not a blob.

Declare the tables and index on a TableModel in app.py; pxt schema update creates them:

import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.functions.huggingface import sentence_transformer

TableModel = pxt.model_base()
embed_fn = sentence_transformer.using(model_id="intfloat/multilingual-e5-large-instruct")


class Docs(TableModel, name="docs"):
    document: pxt.Document


class Chunks(
    TableModel,
    name="doc_chunks",
    base=Docs,
    iterator=pxtf.document.document_splitter(Docs.document, separators="paragraph"),
):
    __indexes__ = [pxt.EmbeddingIndex(text, embedding=embed_fn, name="chunks_embed")]  # type: ignore[name-defined]

pxt init
pxt schema update app.py my_app

A notebook or REPL can still call create_table, create_view, and add_embedding_index. Do not put those calls in app.py.

YAML spec

agent = Agent.from_spec(
    {
        "model": "anthropic:claude-sonnet-4-6",
        "capabilities": [{"Pixeltable": {"tables": ["my_app.doc_chunks"]}}],
    },
    custom_capability_types=[Pixeltable],
)

The short form {"Pixeltable": ["my_app.doc_chunks"]} passes the allowlist alone. Without custom_capability_types=[Pixeltable], Agent.from_spec does not know the class.

Memory store

• The table is created on first use; you do not run pxt schema update for it.
• Each path is one row (kind == "file"); operation receipts are rows under __op__/. Path roots __meta__ and __op__ are reserved.
• Compare-and-set: expected_version must equal the row's version. Versions are unique UUID strings, not monotonic. A stale version raises MemoryConflictError.
• Operation receipts are journaled __op__ rows: the intended mutation is recorded before it is applied, so a mid-write crash rolls forward or replays cleanly instead of double-applying.
• Paths are limited to 255 characters, since the primary-key index covers the first 256.
• Pixeltable keeps old row versions for every update and delete, and receipts are never pruned, so the table grows with history.
• Memory(PixeltableMemoryStore) is Python-only. Harness YAML backends are memory, file, and sqlite.
• This package emits no telemetry of its own; the memory.* spans come from the Harness Memory capability.

Escape hatch: .table

store = PixeltableMemoryStore(table_name="harness.memory")
t = store.table
t.where(t.kind == "file").select(t.path, t.content, t.version).collect()

Use .table to query. A raw t.update of content is not a Memory write: the version does not change, and the next compare-and-set can overwrite it.

Measured results

Pixeltable 0.7.8 on embedded PostgreSQL, ~20k rows per test, laptop hardware (pytest tests/test_stress.py -v -m expensive):

list_paths and search sort every path under the prefix in Python, because database ordering depends on collation, so their cost grows with the files in that namespace, not with the whole table.

Development

pip install -e ".[dev]"
pytest tests/ -v
pytest tests/test_stress.py -v -m expensive
ruff check . && ruff format --check .

pytest tests/ -v skips @pytest.mark.expensive (~20k-row Memory and catalog volume).

License

Apache 2.0