Skip to main content

AirVault

Air-Gapped Document Intelligence SDK

A Python SDK that turns unstructured enterprise data into a queryable, access-controlled vector knowledge base — fully offline, zero external API calls, embeddable in any application.

from airvault import AirVault

engine = AirVault()
await engine.ingest("contracts/nda.pdf", collection="legal")
results = await engine.query("What is the non-compete clause?", clearance="confidential")
print(results.citations[0].excerpt)
# → "The Employee shall not engage in competing business for 12 months..."
print(f"Redacted: {results.chunks_redacted} chunks above your clearance")

Why AirVault?

  • Embeddable SDK — pip install airvault → 5 lines to ingest + query. No server required.
  • Fully offline — local embedding models, zero external API calls, air-gap deployable
  • Compliance-aware retrieval — document sensitivity classification + user clearance enforced at the vector DB query level (not post-retrieval)
  • Multi-format — PDFs, audio (Whisper), scanned images (OCR), text, Markdown
  • Audit trail — every query, every chunk returned, every chunk redacted — logged
  • Multiple integration modes — Python SDK, REST API, MCP server, File Drop

Install

pip install airvault                  # Core SDK
pip install airvault[server]          # + REST API server
pip install airvault[mcp]             # + MCP server for AI agents
pip install airvault[all]             # Everything

Try It Yourself (2 minutes)

# 1. Clone and install
git clone https://github.com/kishlay2002/airvault.git && cd airvault
pip install -e "sdk/[dev]"

# 2. Start infrastructure (Qdrant + PostgreSQL, schema auto-applied)
docker compose -f docker-compose.demo.yml up -d

# 3. Run unit tests (no infra needed)
cd sdk && pytest tests/ -v   # 62 tests, ~8 seconds

# 4. Run the full end-to-end demo
python examples/run_demo.py

The demo ingests documents at different sensitivity levels and queries as users with different clearances — showing that a PUBLIC user never sees CONFIDENTIAL chunks.

Quick Start

As a Python SDK (in-process, no server)

from airvault import AirVault, SensitivityTier

async with AirVault() as engine:
    # Ingest
    await engine.ingest("report.pdf", collection="hr")
    await engine.ingest_text("Confidential merger details...", collection="deals",
                             sensitivity=SensitivityTier.CONFIDENTIAL)

    # Query with access control
    results = await engine.query("merger timeline", collection="deals",
                                 clearance=SensitivityTier.INTERNAL)
    # → chunks_redacted=1 (CONFIDENTIAL chunk filtered for INTERNAL user)

# Sync wrapper for non-async code
from airvault import AirVaultSync
engine = AirVaultSync()
engine.ingest("report.pdf")
results = engine.query("quarterly revenue", clearance="internal")

Configuration via Environment

export AIRVAULT_QDRANT_HOST=10.0.0.5
export AIRVAULT_EMBEDDING_MODEL=BAAI/bge-base-en-v1.5
export AIRVAULT_EMBEDDING_DIMENSION=768
export AIRVAULT_LOG_LEVEL=DEBUG

python my_app.py   # Config auto-loaded from env

As a REST API

airvault serve --port 8000

curl -X POST http://localhost:8000/api/v1/query \
  -d '{"query": "data retention policy", "clearance": "internal", "top_k": 5}'

As an MCP Server (for AI agents)

airvault serve --mode mcp   # stdio transport
// claude_desktop_config.json
{"mcpServers": {"airvault": {"command": "airvault", "args": ["serve", "--mode", "mcp"]}}}

Architecture

┌──────────────────────────────────────────────────────┐
│                Your Application                       │
│  from airvault import AirVault                      │
│  engine = AirVault(config)                           │
│  results = await engine.query(...)                    │
└──────────────────┬───────────────────────────────────┘
                   │
         ┌─────────▼──────────┐
         │   AirVault SDK     │   ← The engine
         │  Extractors         │
         │  Chunker            │
         │  Embedder           │
         │  Classifier         │
         │  Access Filter      │   ← Core differentiator
         │  Audit Logger       │
         └────────┬────────────┘
                  │
    ┌─────────────▼──────────────┐
    │  Qdrant    │  PostgreSQL    │   ← Your infra (local)
    │  (vectors) │  (metadata)    │
    └────────────────────────────┘

  Optional wrappers (same SDK underneath):
  ┌──────────┐ ┌──────────┐ ┌─────────┐ ┌──────────┐
  │ REST API │ │MCP Server│ │Admin CLI│ │File Drop │
  │ (FastAPI)│ │ (stdio)  │ │ (Click) │ │(Go)      │
  └──────────┘ └──────────┘ └─────────┘ └──────────┘

Compliance-Aware Retrieval (Key Differentiator)

Standard RAG retrieves chunks by similarity regardless of who is asking. AirVault filters at the Qdrant query level — restricted chunks are never loaded into memory for unauthorized users.

# PUBLIC user queries "nuclear"
results = await engine.query("nuclear", clearance="public")
# → Only PUBLIC chunks returned
# → RESTRICTED "launch code" chunks are NEVER loaded
# → results.chunks_redacted = 1

# RESTRICTED user queries "nuclear"
results = await engine.query("nuclear", clearance="restricted")
# → All chunks returned (PUBLIC + RESTRICTED)
# → results.chunks_redacted = 0

See examples/access_control_demo.py for a full demo.

SDK API

async with AirVault(config) as engine:
    # Ingestion
    await engine.ingest(file_path, collection, sensitivity, metadata)
    await engine.ingest_batch(file_paths, collection)
    await engine.ingest_text(text, source_name, collection)

    # Querying
    result = await engine.query(text, collection, clearance, top_k)
    # → result.answer, result.citations, result.chunks_redacted

    # Collections
    await engine.list_collections()
    await engine.create_collection(name, description)
    await engine.collection_stats(name)

    # Documents
    await engine.list_documents(collection, sensitivity)
    await engine.get_document(doc_id)
    await engine.delete_document(doc_id)

    # Audit
    await engine.audit_log(username, since, limit)

    # Health
    await engine.health()  # {"status": "ok", "checks": {"postgres": "ok", ...}}

# Errors
from airvault import DuplicateDocumentError, UnsupportedFileTypeError, AirVaultError
try:
    await engine.ingest("report.pdf")
except DuplicateDocumentError:
    pass  # Already ingested (dedup by SHA-256)
except AirVaultError:  # Catch-all for any SDK error
    pass

Project Structure

airvault/
├── sdk/                          # ← THE CORE: pip install airvault
│   └── airvault/
│       ├── engine.py             # AirVault class (main entry point)
│       ├── config.py             # AirVaultConfig (pydantic-settings, env vars)
│       ├── types.py              # SensitivityTier, QueryResult, Citation
│       ├── errors.py             # AirVaultError hierarchy
│       ├── sync.py               # AirVaultSync (sync wrapper)
│       ├── py.typed              # PEP 561 typing marker
│       ├── ingestion/            # Extract → chunk → classify → embed → store
│       ├── embedding/            # Local sentence-transformers (batched)
│       ├── retrieval/            # Query engine + access filter
│       ├── storage/              # Qdrant + PostgreSQL
│       ├── _rest_server.py       # Thin FastAPI wrapper
│       └── _mcp_server.py        # Thin MCP wrapper
│   └── tests/                    # 62 unit tests
├── services/
│   ├── ingestion-worker/         # arq worker (calls SDK)
│   └── file-sentinel/            # Go file watcher sidecar
├── docker-compose.demo.yml       # One-command infra setup
├── examples/                     # Usage examples + run_demo.py
├── migrations/                   # PostgreSQL schema
└── docs/                         # RUNBOOK, THREAT_MODEL

Air-Gapped Deployment

The SDK is fully offline. The only network calls are to Qdrant + PostgreSQL on your own network.

# Connected machine: build + package
make build && bash deploy/scripts/airgap-package.sh

# Air-gapped machine: load + deploy
bash deploy/scripts/airgap-deploy.sh

All model weights baked into Docker images. imagePullPolicy: Never. Zero downloads at runtime.

Embedding Models

Default: BAAI/bge-small-en-v1.5 (33MB, 384d) — optimized for CPU-only, air-gapped environments.

Swap to a better model with one config change:

from airvault import AirVault, AirVaultConfig

# Better quality (recommended if you have the resources)
config = AirVaultConfig(
    embedding_model="BAAI/bge-base-en-v1.5",
    embedding_dimension=768,
)

# Best quality (needs more RAM/CPU)
config = AirVaultConfig(
    embedding_model="BAAI/bge-large-en-v1.5",
    embedding_dimension=1024,
)

engine = AirVault(config)
Model Size Dimensions Quality CPU Speed Best For
bge-small-en-v1.5 (default) 33MB 384 Good ~5ms Air-gapped, constrained hardware
bge-base-en-v1.5 110MB 768 Better ~15ms General production use
bge-large-en-v1.5 335MB 1024 Best ~50ms Maximum accuracy, GPU available
all-MiniLM-L6-v2 80MB 384 Good ~8ms Popular alternative

Any sentence-transformers model works — just set the name and dimension.

Security

  • Access control at retrieval boundary — not post-retrieval, not at the API
  • Redaction counting without data access — uses Qdrant count() with inverted filter; restricted content is never loaded into memory, even for metrics
  • Idempotent ingestion — SHA-256 dedup prevents double-ingestion (configurable)
  • API key auth — bcrypt-hashed, never stored plaintext
  • Sensitivity auto-classification — rule-based, auditable, explainable
  • Immutable audit log — no DELETE on audit table
  • Custom error hierarchy — AirVaultError base class for safe error handling
  • See THREAT_MODEL.md

Testing

cd sdk && pip install -e ".[dev]"
pytest tests/ -v                 # 62 tests, ~8 seconds

Tests cover: types, access filters, redaction filters, chunker, classifier, config (env vars), error hierarchy.

License

MIT

Metadata

Release files for airvault 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for airvault 0.1.1
File Size Uploaded Provenance Actions
airvault-0.1.1.tar.gz 34.6 kB Without provenance Details for airvault-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for airvault 0.1.1
File Interpreter ABI Platform Actions
airvault-0.1.1-py3-none-any.whl Python 3 none any Details for airvault-0.1.1-py3-none-any.whl

Total release size: 69.7 kB

Release files / airvault-0.1.1.tar.gz

Download URL airvault-0.1.1.tar.gz
Size 34.6 kB
Tags Source
SHA-256 checksum
How to use checksums
5d60ec63a9f3a0fb872d84a147c047803ab0e24554becb92420371a98d92f475
BLAKE2b-256 checksum
How to use checksums
bbd990f71e13f10f127d65803d980db0d95fc2631363c7305078e2897627ee3e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.9

Release files / airvault-0.1.1-py3-none-any.whl

Download URL airvault-0.1.1-py3-none-any.whl
Size 35.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
42ad7a48442287d5e5c3bb0ee1082177931e83918b9903224525a3b07157c92f
BLAKE2b-256 checksum
How to use checksums
2d28174482e347a03e1fbea207dbca9edd1733863b13a2e47bbccac00402a0f0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.9

Security for airvault 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page