AgentPack

▶️ Watch the 90-second launch video

AgentPack improves the context pipeline for document-grounded agents.

Instead of forcing AI agents to parse messy, disparate file formats (PDFs, CSVs, Markdown, text) at runtime, AgentPack is an offline document-to-agent-context compiler. It takes unstructured knowledge bases, turns them into clean semantic chunks with citations, retrieves the right evidence, and sends only high-signal context to the model.

Why AgentPack: across 9 retrieval strategies on 2 corpora, it delivers the most consistent retrieval quality of any method tested — 0.83 Hit@3 on both homogeneous and heterogeneous document sets — while keeping context ~100× smaller than raw document stuffing. See the full benchmark results.

The Benchmark

Given the same LLM, AgentPack provides better context than raw document stuffing or naive RAG.

I benchmarked AgentPack against standard RAG baselines on 42 complex financial queries from Patronus AI FinanceBench. The results prove that AgentPack reduces context bloat, improves evidence retrieval, preserves citations, and helps the exact same LLM produce more grounded answers.

Benchmark Highlights:

• 161x Reduction in Token Cost: Cut context token usage from 424k to 2.6k, saving ~$0.10 per query.
• Highest Correctness of Any Strategy: AgentPack (Vector) scored 3.95/5 judge-graded correctness on FinanceBench — the top of all 9 retrieval strategies tested.
• ~1.7x Context Relevance: Retrieved context graded ~1.7x more relevant than naive chunking (3.14 vs 1.83) by preserving semantically complete financial tables.

AgentPack is best treated as an offline document-to-agent-context compiler. It reduces context bloat, but a strong reasoning model is still required to solve complex queries.

Signal	What good looks like
Token reduction	~161x reduction (99% smaller) compared to raw document stuffing
Context per query	Averages ~2.6k high-signal tokens per retrieval (vs 400k+ for raw files)
Context Relevance	~1.7x more relevant than naive chunking (3.14 vs 1.83); preserves tabular and semantic boundaries
Cost Savings	Drops LLM input cost per query from ~$0.11 to <$0.0007
Answer Correctness	Highest judge-graded correctness (3.95/5) of any retrieval strategy tested on FinanceBench
The Bottleneck	AgentPack provides the context, but you still need a frontier model to perform the final reasoning

Use deterministic, LLM-as-a-judge evals instead of trusting raw compression numbers.

Read the full scientific methodology and results in BENCHMARK.md.

Installation

You can install AgentPack via pip or npm. To use the new interactive Corpus Explorer UI, you must install the [ui] extra dependencies.

Option 1: Using pip (Python)

# Core only
pip install agent-context-packager

# With Corpus Explorer UI
pip install "agent-context-packager[ui]"

Option 2: Using npm (Node.js/CLI binary)

npm install -g agent-context-packager

Option 3: From Source

git clone https://github.com/Vedant1202/agentpack.git
cd agentpack
python3 -m venv venv
source venv/bin/activate
pip install -e .

Quick Start

1. Scan for Secrets (Recommended)

Before compiling a pack, ensure you aren't accidentally leaking API keys or secrets into the LLM context window. AgentPack automatically installs Yelp's detect-secrets.

detect-secrets scan > .secrets.baseline

2. Compile a Pack

Point AgentPack at any folder containing your documents (.txt, .md, .csv, .pdf, .docx, .pptx, .xlsx, .html).

agentpack pack ./my_docs --out ./agentpack-output

Key Compilation Options:

• --include "*.md,*.txt": Only pack specific files or extensions.
• --ignore "tests/,drafts/": Exclude specific directories or files.
• --remove-empty-lines: Compress text files to save LLM tokens.
• --no-gitignore: Ignore .gitignore rules and pack everything.
• --fast: Fast mode (PyMuPDF for PDFs; skips Docling). Best for quick iteration on small corpora.
• --no-map: Skip building the hierarchical knowledge map (map.yml). The map is generated by default.

Settings can also be stored in an agentpack.toml file in your input directory:

[pack]
chunk_max_tokens = 800
exclude = ["drafts/", "*.log"]

2b. Pre-build Indexes (optional)

Run this after packing to avoid paying the index-build cost on the first query:

agentpack index ./agentpack-output

2c. The Knowledge Map (map.yml)

Every pack also emits a map.yml — a compact, hierarchical map of what information lives where (corpus → document → section → chunk), built for agentic/RAG navigation. Each section carries page ranges, a has_tables flag, and deterministic, offline descriptors (YAKE keyphrases + a TextRank gist — no LLM, no network). It is purely additive and never touches the retrieval indexes.

agentpack map ./agentpack-output     # (re)build map.yml for an existing pack

🗺️ Read the full Knowledge Map guide

2d. The Concept Graph (graph.yml)

map.yml describes what is inside each document. graph.yml describes how the documents relate to each other — which topics recur across the corpus, which documents reference each other, and which are connected to nothing at all.

Documents, sections, and recurring concepts become nodes; contains, mentions, references, and similar_to become edges. Communities are detected with seeded Louvain clustering. Like the map, it is deterministic, offline, and never touches the retrieval indexes.

flowchart LR
    D1[onboarding.md] -->|contains| S1[Deployment Basics]
    D2[incident-response.md] -->|contains| S2[The Deployment Pipeline]
    S1 -->|mentions| C1([Deployment Pipeline])
    S2 -->|mentions| C1
    D1 -.->|references| D2

Every pack also writes reports/graph_report.md, a plain-language read on the corpus:

## Top Concepts
- **alerting system** (3 mention(s))
- **Deployment Pipeline** (2 mention(s))

## Isolated Documents
- No isolated documents.

That report answers questions retrieval alone cannot: is this corpus actually about what I assumed? and is any document disconnected from the rest — either off-topic, or a sign the document that would connect it is missing.

agentpack graph ./agentpack-output                     # (re)build graph.yml
agentpack graph ./agentpack-output --with-similarity   # add embedding-based similarity edges

Concept promotion is tunable per corpus in agentpack.toml:

[graph]
df_cap   = 0.30   # ignore phrases appearing in >30% of sections (boilerplate)
min_docs = 2      # a concept must span at least 2 documents

Read the full Concept Graph guide

3. Retrieve

AgentPack comes with a built-in hybrid search engine (SQLite FTS5 + HNSW vector search, fused with RRF) to test your chunks instantly.

agentpack retrieve ./agentpack-output "eligibility criteria" --top-k 5

# Narrow results with metadata filters
agentpack retrieve ./agentpack-output "revenue" --source "annual_report" --page 12

3. Deterministic Eval

Benchmark AgentPack against naive chunking using our offline evaluation harness.

agentpack eval ./benchmarks/my_dataset

4. Visualize with the Corpus Explorer

If you installed AgentPack with the [ui] extra, you can launch a local 2D force-graph explorer of your compiled chunks. This allows you to visually debug chunk sizes, semantic similarities, and ranked retrieval results.

agentpack ui ./agentpack-output --port 8000

The explorer has two views, switched from the header:

• Universe — every chunk as a node, clustered by document. Run a query and watch which chunks the hybrid retriever actually returns.
• Concepts — the concept graph, colored by community. Click a concept to see which sections mention it and whether it bridges clusters; click a document to see whether it is isolated. Individual edge types can be hidden to cut through dense graphs.

🖼️ Read the full UI breakdown

Comprehensive CLI Documentation

AgentPack provides a rich CLI for auditing, validating, and testing your context packs (including Generative QA evaluations).

📖 Read the full CLI Reference

Supported Parsers

• TXT: Paragraph-aware splitting.
• Markdown: Semantic heading-aware section path tracking.
• CSV: Uses Pandas & Tabulate to convert tabular data into Markdown tables.
• PDF: Docling structured-tree parse (default) — preserves page numbers, sections, and tables. PyMuPDF spatial extraction with --fast.
• DOCX / PPTX / XLSX / HTML: Docling semantic parse — same structured-tree path as PDF.

Architecture Overview

flowchart LR
    Docs[Raw Docs] --> Parsers[Parsers]
    Parsers --> Chunker[Chunker]
    Chunker --> Pack[Context Pack]
    Pack --> Map[Knowledge Map map.yml]
    Map --> Graph[Concept Graph graph.yml]
    Pack --> Graph
    Pack --> Agent[LLM Agent]
    Map --> Agent
    Graph --> Agent

Three layers, each answering a different question:

Artifact	Answers	Scope
manifest.yml	Where did this text come from?	Per chunk
map.yml	What is inside this document, and where?	Per document
graph.yml	How do these documents relate to each other?	Whole corpus

For a deep dive into how AgentPack parses, chunks, and indexes data, see Architecture & Internals.

Current Limitations & Roadmap

• Image Understanding / Vision: OCR and vision models on embedded images are not yet supported. Images are ignored during parsing.
• Complex Nested Tables: Highly merged-cell tables in PDFs may not perfectly reconstruct.
• Web Crawling: Local files only; URL scraping is planned.
• Cloud Vector DB Integration: Retrieval runs locally (SQLite FTS5 + HNSW). Connectors for Pinecone, Weaviate, or Qdrant are planned.
• Cross-Encoder Reranking: A secondary rerank pass is on the roadmap (deferred to v0.4).
• Map Enrichment Extras: The knowledge map's descriptors are deterministic/offline today; opt-in LLM-abstractive summaries and typed-entity (NER) extraction are planned as optional add-ons.
• Concept Quality: Concept graph concepts come from statistical keyphrase extraction (YAKE), so corpora with heavy shared boilerplate can surface generic vocabulary alongside real topics. Structural duplicates and self-references are filtered; semantically-empty phrases are not. Optional LLM concept enrichment is planned.
• Graph-Aware Retrieval: The concept graph is a navigation and audit layer today — it never influences what agentpack retrieve returns. Fusing graph signals into hybrid retrieval as a third RRF contributor is planned, gated on demonstrated lift in agentpack eval.

---

Built with ❤️ for Agents.