Skip to main content

Enterprise corpus — codebase, Slack, meeting notes, and documentation — flowing continuously through the CocoIndex incremental sync engine into a production AI agent with always-fresh context. Only the Δ (delta) is reprocessed on every change. Keywords: RAG pipeline, agent memory, enterprise retrieval, AI agent context, live indexing, retrieval-augmented generation, production LLM apps, streaming ETL, incremental ingestion.

Your agents deserve fresh context.

Star us ❤️ → Star CocoIndex on GitHub — open-source Python framework for RAG, vector search, and live agent context  ·  cocoindex.io — the CocoIndex homepage: incremental data pipelines for AI agents  ·  CocoIndex documentation — quickstart, connectors, ops, transformations, target stores, RAG and knowledge graph recipes  ·  Join the CocoIndex Discord community — help, showcase, release notes, and live chat with maintainers

CocoIndex turns codebases, meeting notes, inboxes, Slack, PDFs, and videos into live, continuously fresh context for your AI agents and LLM apps to reason over effectively — with minimal incremental processing. Get your production AI agent ready in 10 minutes with reliable, continuously fresh data — no stale batches, no context gap

Incremental · only the delta  ·  Any scale · parallel by default  ·  Declarative · Python, 5 min

stars downloads pypi python rust license discord

CI release links

cocoindex-io/cocoindex | Trendshift




Built with CocoIndex ❤️

CocoIndex-code — flagship MCP server for AI coding agents. AST-aware incremental semantic code index that keeps live call graphs, symbols, vectors, and chunks fresh on every commit. 70% fewer tokens per turn, 80-90% cache hits on re-index, sub-second freshness. Supports Python, TypeScript, Rust, and Go. Features: Δ-only incremental processing, semantic search by meaning (not grep), call graphs and blast-radius analysis, global repo view for duplicates and architecture. Build coding agents (generate, refactor) and code-review agents (catch, approve). One install — Claude Code, Cursor, and other MCP-aware agents see your whole repository instantly. Keywords: MCP server, coding agent, code intelligence, AST chunking, semantic code search, call graph, vector embedding, repository context, Claude Code, Cursor, incremental indexing, blast radius.

See all 20+ examples · updated every week →


Get started

pip install -U cocoindex

Declare what should be in your target — CocoIndex keeps it in sync forever, recomputing only the Δ.

import cocoindex as coco
from cocoindex.connectors import localfs, postgres
from cocoindex.ops.text import RecursiveSplitter

@coco.fn(memo=True)                          # ← cached by hash(input) + hash(code)
async def index_file(file, table):
    for chunk in RecursiveSplitter().split(await file.read_text()):
        table.declare_row(text=chunk.text, embedding=embed(chunk.text))

@coco.fn
async def main(src):
    table = await postgres.mount_table_target(PG, table_name="docs")
    table.declare_vector_index(column="embedding")
    await coco.mount_each(index_file, localfs.walk_dir(src).items(), table)

coco.App(coco.AppConfig(name="docs"), main, src="./docs").update_blocking()

Run once to backfill. Re-run anytime — only the changed files re-embed.

Building with an AI coding agent?
Drop in our CocoIndex skill so your agent writes correct v1 code — concepts, APIs, patterns, all in one file.
See Use with AI coding agents for install steps.

Full quickstart — open-book icon linking to the CocoIndex documentation quickstart: pip install, declare sources and targets, run the incremental engine    Learn the concept — lightbulb icon linking to the CocoIndex core-concepts guide: sources, targets, flows, incremental engine, and data lineage

Animated GitHub Star button for the cocoindex-io/cocoindex repository: a cursor clicks the star, it fills yellow, confetti bursts, the star count ticks up, and an 'Appreciate a star if you like it!' caption with a beating heart shows below the button



React — for data engineering

React — for data engineering. The CocoIndex mental model: Target = F(Source). A persistent-state-driven dataflow where you declare the desired target state and the engine keeps it in sync with the latest source data and code, forever, at low latency and low cost. Source files (.py, .md, .pdf, .ts) flow through your Python transformation F into a live target dots-matrix index; only the Δ is reprocessed on every change, and every target dot traces back to its exact source byte. Four core properties: Python not a DAG (sky), declare target state (yellow bullseye), lineage end-to-end (coral connected dots), and incremental at any scale (mint Δ+1). Your code is as simple as the one-off version — the engine does the rest. Keywords: React for data engineering, declarative ETL, persistent state, data lineage, dataflow, Δ only, incremental indexing, CocoIndex.

What happens when either side changes — CocoIndex tracks per-row provenance so the Δ propagates at minimum cost. Two scenarios shown in one illustration: (top) Source change — one file (b.md) is edited and only one target dot re-syncs (coral pulse). (bottom) Code change — the transformation function F is rewritten from v1 to v2 and only the dots whose outputs depend on the changed code re-run (amber/yellow pulses). Source on the left, F in the center (Python code block), target dots-matrix on the right. Keywords: incremental indexing, change data capture, delta processing, fine-grained invalidation, code-aware caching, hash-of-code invalidation, memoization, reproducible pipelines, incremental recomputation.

See the React ↔ CocoIndex mental model →



Incremental engine for long-horizon agents

Data transformation for any engineer, designed for AI workloads —
with a smart incremental engine for always-fresh, explainable data.

Learn the concept — purple button with a lightbulb icon linking to the CocoIndex core-concepts guide: sources, targets, flows, incremental engine, and data lineage

CocoIndex's Python-native transformation flows connect 8 source categories (Codebases, Meeting Notes, Web · APIs, File System · Blob Stores, Databases, Message Queues, Images · Video, Voice · Transcripts) through the incremental engine out to 6 target stores (Relational DB, Data Warehouse, Vector DB, Graph DB, Message Queue, Feature Store). A flow.py code block (@coco.fn · def f(src): · chunks = split(src) · target.row(embed(chunks))) shows the shared pipeline; only the Δ is reprocessed — unchanged src hits the cache, changed src re-runs split() and Δ → re-embed. The persistent data-pipeline control plane runs eight always-on subsystems: live caching, pipeline catalog, version tracking, continuously learning, lineage, task scheduling, metrics collection, and failure management. Keywords: data pipeline, ETL, source connectors, vector database, graph database, incremental engine, streaming ingestion, caching, lineage, versioning, scheduling, metrics, retries.



Why incremental?

Your agents are only as good as the data they see.
Batch pipelines drift stale. CocoIndex stays live — and only runs the Δ.

Why incremental? — one illustration combining the four core benefits of CocoIndex's incremental engine. Sub-second fresh (mint): a stopwatch ticking under a second, source changes propagate to the target in under a second so agents see the world as it is, not as it was yesterday. 10× cheaper at scale (yellow): a 10,000-row corpus block where only a thin Δ 0.1% column re-runs and 99.9% stays cached — you skip the other 99.9% of your corpus and pay a fraction of the compute, embedding, and LLM bill. Explainable by default (coral): a lineage thread links a source byte (handbook.md L42) to a target vector — every vector, row, or graph node in the target traces back to its exact source byte for debuggable, auditable, regulator-friendly AI pipelines. Production-grade (purple): a shield stamped with the Rust crab surrounded by retry loops, back-off dots, a DLQ tray, and a no-data-loss check — Rust core with retries, exponential back-off, dead-letter queues, and no-data-loss guarantees, production-ready for long-horizon AI agents. Keywords: incremental indexing, Δ-only reprocessing, sub-second freshness, low-latency RAG, cost-efficient embeddings, data lineage, retrieval-augmented generation, Rust core, retries, back-off, dead letters, no data loss, long-horizon agents.



What can you build?

See all 20+ examples · updated every week →

Working starters from the examples tree — clone, plug your source, ship.

Real-time code index — walk a git repo, AST-chunk source files, embed with sentence-transformers, upsert to pgvector / LanceDB, incremental on every commit. Keywords: code search, code embedding, semantic code retrieval, Python.

PDF → RAG index — ingest PDFs from local, S3, or GDrive, extract + chunk text, embed chunks, upsert to pgvector / LanceDB. Classic retrieval-augmented-generation stack, incremental. Keywords: RAG, document Q&A, PDF search, vector database.

HN trending topics — pull Hacker News threads via Algolia, recursively parse comments, LLM-extract topics with Gemini 2.5 Flash, rank by weighted hit count (thread=5, comment=1), store in Postgres. Incremental. Keywords: Hacker News, trending topics, LLM extraction, Gemini, Postgres, news intelligence, topic ranking.

Conversation → knowledge graph — LLM extracts people, topics, decisions, action items from transcripts and upserts into Neo4j / Kuzu. Live graph, incremental. Keywords: knowledge graph, entity extraction, meeting intelligence, agent memory.

Multi-repo summarization — walk N git repos, extract structure, LLM-summarize per-repo + a rolled-up org summary, refresh on every push. Keywords: internal platform, developer experience, monorepo, SDK docs.

Structured extraction — BAML / DSPy typed schema extraction from forms, PDFs, intakes, invoices into Postgres / warehouse. Incremental. Keywords: ETL, LLM extraction, schema-first, patient intake, invoice processing, KYC, contracts.

Podcast → knowledge graph — transcribe YouTube / Spotify audio with speaker diarization, LLM-extract speakers and statements, resolve entities across episodes, store in SurrealDB / Neo4j. Keywords: podcast, diarization, YouTube, Whisper, SurrealDB, knowledge graph, entity resolution.

CSV → Kafka live — watch a folder of CSV files, publish each row as a JSON message to a Kafka topic via CocoIndex's Kafka target connector. Incremental, sub-second, no producer loop. Keywords: Kafka, CDC, streaming, StreamNative, Confluent, CSV ingestion, event streaming.


Share what you build — a banner with a trail of tiny hearts rising from the bottom behind the text, inviting the CocoIndex community to share projects built with the framework

Building something with CocoIndex? We want to see it.
Tag @cocoindex_io on X or drop a link in #showcase on Discord. We'll boost it. 🥥



Community

Join the CocoIndex Discord community — live chat with maintainers and users, showcase your projects, get help building RAG pipelines and knowledge graphs Subscribe to the CocoIndex YouTube channel — video tutorials, live demos, architecture deep dives, and AI agent recipes Read the CocoIndex blog — engineering deep dives, release notes, RAG and knowledge graph tutorials, and case studies Follow @cocoindex_io on X (formerly Twitter) for release notes, demos, launches, and AI data pipeline updates



We love Contributors — section title banner with a pulsing coral heart badge and cream twinkle sparkles. Every typo fix, new connector, and doc tweak makes CocoIndex better. Keywords: open-source contribution, pull request, typo fix, new connector, good first issue, Hacktoberfest, community, coconut heart.

We are so excited to meet you.
Every typo fix, new connector, doc tweak, or full-on rewrite makes CocoIndex better.
Come hang out — big PRs and small ones, both welcome.

📝 Read the contributing guide  ·  🐛 good first issues  ·  💬 Say hi on Discord



CocoIndex Enterprise

CocoIndex Enterprise — built for enterprise scale. Four headline stats for PB-scale incremental indexing: PB corpus scale incrementally indexed (coral), 10× fewer LLM embedding calls vs. full recompute (yellow), 100% lineage coverage with every byte traceable (mint), Δ only the delta always (sky). Below, a wide 50×8 corpus matrix of 400 dim tiles represents a petabyte-scale store where a single coral Δ slice of 8 tiles re-runs while the other 99.9% stays cached. Keywords: enterprise RAG, petabyte-scale indexing, incremental compute, delta-only, lineage, parallel chunking, zero-copy, failure isolation.

Large corpus — built for enterprise scale.

Incremental compute is the only way to keep large corpora fresh without re-embedding them every cycle.
CocoIndex scales from a single repo to petabyte-scale stores — parallel by default, delta-only by design.


Process once. Reconcile forever.

When a source changes, CocoIndex identifies the affected records, propagates the change
across joins and lookups, updates the target, and retires stale rows —
without touching anything that didn't change.


Built on a Rust engine.

The core is Rust — production-grade from day zero.
Parallel chunking, zero-copy transforms where possible, and failure isolation
so one bad record doesn't stall the flow.



Explore CocoIndex Enterprise — bright blue pill button linking to cocoindex.io/enterprise, the PB-scale incremental data pipeline for AI agents



Apache 2.0 · © CocoIndex contributors 🥥

Metadata

Release files for cocoindex 1.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cocoindex 1.2.0
File Size Uploaded Provenance Actions
cocoindex-1.2.0.tar.gz 967.3 kB Without provenance Details for cocoindex-1.2.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for cocoindex 1.2.0
File Actions
cocoindex-1.2.0-cp314-cp314t-win_amd64.whl CPython 3.14 CPython 3.14 free-threading Windows x86-64 Details for cocoindex-1.2.0-cp314-cp314t-win_amd64.whl
cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ x86-64 Details for cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl
cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ ARM64 Details for cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl
cocoindex-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl CPython 3.14 CPython 3.14 free-threading macOS 11.0+ ARM64 Details for cocoindex-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl
cocoindex-1.2.0-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for cocoindex-1.2.0-cp311-abi3-win_amd64.whl
cocoindex-1.2.0-cp311-abi3-manylinux_2_28_x86_64.whl CPython 3.11 abi3 Linux glibc 2.28+ x86-64 Details for cocoindex-1.2.0-cp311-abi3-manylinux_2_28_x86_64.whl
cocoindex-1.2.0-cp311-abi3-manylinux_2_28_aarch64.whl CPython 3.11 abi3 Linux glibc 2.28+ ARM64 Details for cocoindex-1.2.0-cp311-abi3-manylinux_2_28_aarch64.whl
cocoindex-1.2.0-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for cocoindex-1.2.0-cp311-abi3-macosx_11_0_arm64.whl
cocoindex-1.2.0-cp311-abi3-macosx_10_12_x86_64.whl CPython 3.11 abi3 macOS 10.12+ x86-64 Details for cocoindex-1.2.0-cp311-abi3-macosx_10_12_x86_64.whl

Total release size: 92.5 MB

Release files / cocoindex-1.2.0.tar.gz

Download URL cocoindex-1.2.0.tar.gz
Size 967.3 kB
Tags Source
SHA-256 checksum
How to use checksums
cabb7e245bfdcaeef1c8009e0b8c0b095b70267d4cc9636a60d4223adedd33df
BLAKE2b-256 checksum
How to use checksums
29607c949e67237f19725dfab3a0d1874c7b5fc089bfaf120febd71803eb41ed
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp314-cp314t-win_amd64.whl

Download URL cocoindex-1.2.0-cp314-cp314t-win_amd64.whl
Size 10.5 MB
Tags CPython 3.14 CPython 3.14 free-threading Windows x86-64
SHA-256 checksum
How to use checksums
ffabd27701628b0259b823b7d0d9b6317a8bf4ed6fa00b0f5d9ef99ecbfd1933
BLAKE2b-256 checksum
How to use checksums
f17bc6ed4559685fdeb9c97c4631fd1a860fb4f24b5962b4be34399a9857e7da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl

Download URL cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl
Size 10.2 MB
Tags CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
e20c7eee66a568c991dee2521b4958ae8faf02da704334555c7eaa2e1eab9e12
BLAKE2b-256 checksum
How to use checksums
81f83dd3add4e02fed40b713ec1647bf193f0c91f2f9bc3528287d533a8b92b7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl

Download URL cocoindex-1.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl
Size 9.9 MB
Tags CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ ARM64
SHA-256 checksum
How to use checksums
07207feeb120383deb61a324a46b1e5c961e3f77774c764f1a6242a35fd9a9c7
BLAKE2b-256 checksum
How to use checksums
856164e70aae8e7e152027d59a0c9226362c9840fc17a76067953b2e553eef4e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl

Download URL cocoindex-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl
Size 10.0 MB
Tags CPython 3.14 CPython 3.14 free-threading macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
7d4e97eff96d025a26d9f83365d273ab124b7a50e854ff577a2174ab257fccf5
BLAKE2b-256 checksum
How to use checksums
6e9bcd2d7b22475aebeb619288869d8c59e111146568c4b15656d09b11e46f5f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp311-abi3-win_amd64.whl

Download URL cocoindex-1.2.0-cp311-abi3-win_amd64.whl
Size 10.5 MB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
a84d107496bf787fecc89c657b15bf8e68f710dd50c5866efddb5e6aaaf68201
BLAKE2b-256 checksum
How to use checksums
66ce8894b36407787f0395bf0cc910bc741e236d54409d8c7935cfb35a0ccece
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp311-abi3-manylinux_2_28_x86_64.whl

Download URL cocoindex-1.2.0-cp311-abi3-manylinux_2_28_x86_64.whl
Size 10.2 MB
Tags CPython 3.11 Linux glibc 2.28+ x86-64 abi3
SHA-256 checksum
How to use checksums
91acd866afc336278cd030664acdabf95fdb56f63f19fd4a9a2bab84c73de06e
BLAKE2b-256 checksum
How to use checksums
516ac0e033c4a6b4ee0cd8521bee45a3c26b6776dfd1497e59b1bdd625f1b7f0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp311-abi3-manylinux_2_28_aarch64.whl

Download URL cocoindex-1.2.0-cp311-abi3-manylinux_2_28_aarch64.whl
Size 10.0 MB
Tags CPython 3.11 Linux glibc 2.28+ ARM64 abi3
SHA-256 checksum
How to use checksums
70f016911d1a9da6e1fc1b97bfb76655512d333e39b6169ce874f386e5953d55
BLAKE2b-256 checksum
How to use checksums
c5496efe6292b8f6f0ce60cb101aef810700004058b0287a8d21552d58ff0297
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp311-abi3-macosx_11_0_arm64.whl

Download URL cocoindex-1.2.0-cp311-abi3-macosx_11_0_arm64.whl
Size 10.1 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
831a23ac2deddd0e49362d3c90f677b7271f5717c92948dbe4314370dca71f6f
BLAKE2b-256 checksum
How to use checksums
21510eb1f7b88199362024c1acb2d231a336c53ff45fd851b1c7f56ad1a47ad4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Release files / cocoindex-1.2.0-cp311-abi3-macosx_10_12_x86_64.whl

Download URL cocoindex-1.2.0-cp311-abi3-macosx_10_12_x86_64.whl
Size 10.0 MB
Tags CPython 3.11 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
830c2f1afbdad0078a43be023f6a4ed24c0c994668eeb9322a544be885a5084c
BLAKE2b-256 checksum
How to use checksums
176d1c306f140e165917e47dc845cd72c17f1139e265ae42ef6dab54a5658bcf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via maturin/1.16.0

Security for cocoindex 1.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.2.0 This release

10 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page