DeepCompress
📄 Compress OCR-heavy documents into smaller, page-cited context for LLM and RAG workflows.
DeepCompress helps you reduce token usage while keeping important document facts traceable. It is useful for contracts, invoices, financial packets, claims, and other long documents where exact values matter.
Why Use It?
- ⚡ Reduce context size before sending document text to an LLM.
- 🔎 Keep page-cited output for retrieval and answer evidence.
- 🧾 Protect exact values like IDs, dates, amounts, emails, phone numbers, clauses, and totals.
- 📊 Measure token savings with provider-aware token counters.
- 🔐 Scrub common PII patterns before LLM analysis when needed.
- 🧪 Run offline demos and benchmarks without API keys or external services.
Installation
pip install deepcompress
Optional extras:
pip install "deepcompress[gpu]" # OCR model support
pip install "deepcompress[llm]" # OpenAI / Anthropic integrations
pip install "deepcompress[vector-db]" # Pinecone / Weaviate integrations
pip install "deepcompress[all]" # Everything for local development
Quick Start
Compress a document and inspect the measured savings:
import asyncio
from deepcompress import DeepCompressConfig, DocumentCompressor
async def main():
config = DeepCompressConfig(
dtoon_mode="raw",
protect_facts=True,
vector_db_provider="none",
cache_enabled=False,
)
compressor = DocumentCompressor(config)
result = await compressor.compress("contract.pdf")
print(result.optimized_text)
print(f"Original tokens: {result.original_tokens_measured:,}")
print(f"Compressed tokens: {result.compressed_tokens_measured:,}")
print(f"Compression ratio: {result.compression_ratio_measured:.2f}x")
print(f"Protected facts: {result.protected_facts}")
asyncio.run(main())
Ask A Question
Use the high-level API when you want compression plus an LLM answer:
import asyncio
from deepcompress import compress_and_analyze
async def main():
result = await compress_and_analyze(
file="loan_application.pdf",
query="What is the applicant's total monthly income?",
llm="openai",
scrub_pii=True,
)
print(result.answer)
print(f"Tokens saved: {result.tokens_saved:,}")
asyncio.run(main())
For LLM-backed usage, set your provider key in the environment:
# macOS / Linux
export LLM_API_KEY=your-api-key
# Windows PowerShell
$env:LLM_API_KEY="your-api-key"
Common Configuration
from deepcompress import DeepCompressConfig
config = DeepCompressConfig(
dtoon_mode="raw", # raw, structured, or rag
ocr_device="cpu", # cpu, cuda:0, cuda:1
token_counter_provider="openai",
token_counter_model="gpt-4o",
protect_facts=True,
cache_enabled=False,
vector_db_provider="none",
)
Useful modes:
| Mode | Best For |
|---|---|
raw |
Fast compression without an LLM provider |
structured |
Cleaner structured summaries using an API-backed LLM |
rag |
Retrieval-friendly chunks for RAG pipelines |
Offline Demo
Run the enterprise RAG demo without API keys, Redis, Pinecone, CUDA, or network access:
python examples/enterprise_rag_demo/demo.py
The demo compresses a synthetic loan document, builds a local in-memory index, answers a question with evidence, and prints token and cost savings.
Benchmarks
Run the deterministic benchmark:
python benchmarks/run_benchmark.py
The benchmark compares full-text RAG with DeepCompress RAG and reports token usage, cost, answer accuracy, retrieval hit rate, citation correctness, exact-value preservation, and latency.
What You Get Back
DocumentCompressor.compress() returns a compressed document result with:
optimized_text- the compressed D-TOON document textoriginal_tokens_measured- measured input token countcompressed_tokens_measured- measured compressed token countcompression_ratio_measured- measured compression ratiotokens_saved- token reductioncost_saved_usd- estimated GPT-4o-style input savingsprotected_facts- exact values preserved during compressionprocessing_time_ms- total processing time
Notes And Limits
- Compression quality depends on document type, OCR quality, and selected mode.
structuredandragmodes require an API-backed LLM provider.- PII scrubbing is pattern-based and should be reviewed for your data domain.
- Built-in benchmarks use synthetic fixtures; validate results on your own documents before publishing savings claims.
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for deepcompress 1.4.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| deepcompress-1.4.9.tar.gz | 62.2 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| deepcompress-1.4.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 117.0 kB
Release files / deepcompress-1.4.9.tar.gz
| Download URL | deepcompress-1.4.9.tar.gz |
|---|---|
| Size | 62.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a1e8234b5d53234c69f0d5b2eb13c4465dba68ce3242220518946ef64c7a463e
|
|
BLAKE2b-256 checksum How to use checksums |
bee119aa026d10bf93a9eb7adfceaeac05a6c59e1c6bd12719d5ee6e04655958
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.5
|
Release files / deepcompress-1.4.9-py3-none-any.whl
| Download URL | deepcompress-1.4.9-py3-none-any.whl |
|---|---|
| Size | 54.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0901e29a8a611e56a880a7316f65bd7428affebc066426e9199fa69869450952
|
|
BLAKE2b-256 checksum How to use checksums |
98f7be09ed45cb638989b863d39d7950c25f02c723b949ed783d0b9bd48dd822
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.5
|
Security for deepcompress 1.4.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
