The backend agents build with
Multimodal database, orchestration, and serving in one file, by the co-creators of Apache Parquet and Impala.
Pixeltable is an open-source multimodal database you use from Python. Build AI apps on video, images, audio and documents without writing scripts to keep files, model outputs and search indexes in sync.
pip install -U 'pixeltable[serve]'
Quickstart | Documentation | CLI | Cloud | Discord | For agents
You declare tables, computed columns and HTTP routes in app.py. Insert a row and Pixeltable runs the computations, stores the results and updates the indexes.
You can also use Pixeltable directly from Python or a notebook without an HTTP server. Start with the 10-minute tour.
What it looks like
Upload a recording with speech. This API extracts its audio, transcribes it, summarizes what was said and indexes the summary for search. Save this as app.py:
import pixeltable as pxt
from pixeltable.functions import openai
from pixeltable.functions.video import extract_audio
from pixeltable.serving import FastAPIRouter
TableModel = pxt.model_base()
embed = openai.embeddings.using(model='text-embedding-3-small')
class Videos(TableModel, name='videos'):
video: pxt.Video # the only column you insert; the rest run on insert
audio = extract_audio(video, format='mp3')
transcript = openai.transcriptions(audio, model='whisper-1')
summary = openai.chat_completions(
model='gpt-4o-mini',
messages=[
{'role': 'system', 'content': 'Summarize this video in two sentences.'},
{'role': 'user', 'content': transcript.text},
],
).choices[0].message.content.astype(pxt.String)
__indexes__ = [pxt.EmbeddingIndex(summary, embedding=embed)]
@pxt.query
def search(query: str):
sim = Videos.summary.similarity(string=query)
return Videos.order_by(sim, asc=False).limit(3).select(Videos.video, Videos.summary)
api = FastAPIRouter(name='video-api')
api.add_insert_route(Videos, path='/videos', uploadfile_inputs=[Videos.video], outputs=[Videos.summary])
api.add_query_route(path='/search', query=search)
You need Python 3.11 or later and an OpenAI API key. From the same folder, run:
pip install -U 'pixeltable[serve]' openai
export OPENAI_API_KEY=sk-... # set this before the first pxt command
pxt init # makes this folder a project
pxt schema update app.py my_app # creates the table and its index
pxt service update app.py my_app # starts the service and prints its URL
Replace PORT with the printed port and talk.mp4 with your recording:
curl -F video=@talk.mp4 http://127.0.0.1:PORT/videos
# {"summary": "..."}
curl -X POST http://127.0.0.1:PORT/search -H 'Content-Type: application/json' -d '{"query": "pricing"}'
# {"rows": [{"video": "...", "summary": "..."}]}
Uploading more recordings processes only the new rows. Pixeltable runs model calls concurrently, throttles requests using provider-reported or configured limits, and retries rate-limit errors.
Change the summary without redoing transcription
Say the summaries are too long. Change the system prompt in app.py to Summarize this video in one sentence., then run:
pxt schema update app.py my_app
pxt service update app.py my_app
pxt recompute my_app/videos summary
New uploads use the new prompt after the service restarts. Existing rows keep their summaries until you recompute summary, which also updates its search index. The audio and transcripts stay.
How it works covers dependencies, row errors, selective retries and table versions.
Why we built it
Multimodal AI projects kept needing the same stack: a bucket, a metadata database, an orchestrator, vector search and an API. Each piece is fine on its own. The trouble is in between: a sync script that hopes the bucket and the database still match, and a vector database still serving embeddings of last week's summaries.
We're database people, so our answer was to make the database track dependencies and run the work.
| You need to | Usually | With Pixeltable |
|---|---|---|
| Store files and their results | A bucket plus a metadata table | Typed file references and computed results in the same row |
| Split documents or sample frames | A preprocessing job | Views of chunks or frames, updated with the source table |
| Call models | Jobs and scripts that track what ran | Computed columns with stored results |
| Explore and query the results | Scripts that download the files and redo the frames in a notebook | Query any column from Python or a notebook. Remote files go into a local cache on first read, and frames are extracted from the video as you read them |
| Search the results | A vector database you keep in sync | An embedding index maintained by the table |
| Serve the results | API handlers over those systems | FastAPIRouter routes declared next to the tables |
| Let an agent call tools | An agent loop that keeps its state in memory | A computed column runs the tools the model picked (invoke_tools), and the calls and their results are columns on the message's row |
| Remember across sessions | A chat-history cache and a second vector store | A memory table with an embedding index, recalled by a @pxt.query in a computed column |
| Trace a running app | A tracing SDK per vendor, with logs somewhere else | OpenTelemetry spans for each table, row, column and UDF (pxt service update --otel, pixeltable[otel]) |
| Evaluate model output | Model outputs copied out of logs into an eval notebook | Store the model's full response as a JSON column and query its fields (response.usage). An eval is one more computed column |
| Rerun what failed | A dead-letter queue and a script to replay it | Cells that fail under on_error='ignore' keep their error. pxt errors lists them (the table needs a primary key) and prints the pxt recompute --errors-only that reruns just those. pxt computed shows each column's definition |
| Send results elsewhere | An ETL job into another database, and a sync to the bucket | export_sql() or export_parquet() copies a table or query out. A computed column's destination= writes its files to your own bucket |
| Analyze and audit | A warehouse export, and history rebuilt from logs | Query the tables where they are. Each change is a version (history(), pxt revert) |
If you already use Pinecone, Postgres, LangGraph or Airflow, Migrate to Pixeltable shows how the corresponding work looks here.
What you can build
- RAG over documents, with chunks and embeddings updated as documents arrive or leave.
- Video and image pipelines, with frames, transcripts and detections.
- Agents whose tool calls and results are columns you can query later.
Pixeltable fits best when files or prompts keep changing. If you don't want a typed schema, it's the wrong tool. It won't make your models better, and evaluation is still yours to manage. Why Pixeltable goes into the trade-offs.
Start from an example
For a fresh project, pxt service example --out app.py writes an app that needs no model key. Follow the quickstart to run it in a separate project folder.
Browse complete apps, or follow HTTP serving for uploads and TypeScript calls.
With a coding agent
npx skills add pixeltable/pixeltable-skill
# or, from the root of your repo, with only Python:
pxt skills install
The skill teaches your agent to write app.py, so you can review the backend change in one diff. AI coding agents covers plugins for Claude, Cursor and ChatGPT, and MCP setup.
Where it runs
- Locally, with the commands above. Uploaded and generated media go to local disk or your own S3, GCS or Azure bucket.
- On your infrastructure, from Python, inside your own FastAPI app, or with
pxt service. Self-hosting covers each option. - On Pixeltable Cloud, with the same
app.py, a hosted database and managed media storage. Without an account,pxt newcreates a free trial and prints its expiry and claim link.
Contributing
Bug reports and PRs are welcome: CONTRIBUTING.md. Questions go to Discord or GitHub Discussions. Pixeltable is Apache 2.0.
Metadata
Release files for pixeltable 0.7.24
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pixeltable-0.7.24-py3-none-any.whl | Python 3 | none | any | Details for pixeltable-0.7.24-py3-none-any.whl |
Release files / pixeltable-0.7.24-py3-none-any.whl
| Download URL | pixeltable-0.7.24-py3-none-any.whl |
|---|---|
| Size | 1.5 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a60bc6b694d89933b7504bb6d31e1bd9bd6373b3fee644fbeb77481874a7c060
|
|
BLAKE2b-256 checksum How to use checksums |
bd89f6e7eabc93be3ee281b268dd0203a4d322cc92f78d2765577b4a7867a85d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.3 {"installer":{"name":"uv","version":"0.12.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for pixeltable 0.7.24
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.