docvion 0.2.0

A canonical, engine-agnostic schema, adapter suite, and semantic chunker for parsed documents

Docvion 📄⚡

One schema for every document parser.

---

1. Problem Statement

Every document-parsing engine (Docling, Tesseract, PaddleOCR, Azure Document Intelligence, Google Document AI, AWS Textract) returns output in its own incompatible shape. A developer building a RAG pipeline or memory ingestion engine against one parser's native output format is locked in. Switching parsers, comparing engine accuracy, or combining outputs from multiple engines requires rewriting downstream code every single time.

There has been no neutral, engine-agnostic representation of a parsed document — until now.

2. Solution: Before & After

docvion is a schema-first Python library created by Prolixis that defines one canonical representation of a parsed document — DocvionDocument — plus a suite of zero-overhead adapters that convert any supported parser's output into that canonical form.

Before Docvion (Parser Lock-In)

# Locked to Docling's specific dictionary layout:
def build_llm_prompt(docling_json):
    texts = docling_json["texts"]
    prompt = ""
    for item in texts:
        if item["label"] == "title":
            prompt += f"# {item['text']}\n"
        elif item["label"] == "paragraph":
            prompt += f"{item['text']}\n"
    return prompt

After Docvion (Engine-Agnostic Downstream Pipeline)

from docvion import DocvionDocument, convert

# Downstream code relies strictly on DocvionDocument. Swap engines freely!
doc: DocvionDocument = convert(engine="docling", raw_output=raw_data)
# Or swap to Tesseract / PaddleOCR without changing 1 line of downstream code:
# doc: DocvionDocument = convert(engine="tesseract", raw_output=tesseract_data)

prompt_context = doc.export_to_markdown()
json_payload = doc.export_to_json()

---

3. Landscape Comparison: Docvion vs. Alternatives

Docvion is not an OCR engine or an ETL platform—it is a neutral schema specification and adapter layer. Here is how Docvion compares to existing tools:

Dimension	Docvion	Unstructured.io	Raw Engine (e.g. Docling)	LlamaParse
Primary Architecture	Schema-First Neutral Layer	Document ETL & Partition Suite	Specific Parser/OCR Model	Cloud Parsing API
Schema Portability	High: Single canonical DocvionDocument across all engines	Medium: Custom Element list schema	None: Locked to engine's native JSON output	None: Locked to LlamaParse proprietary JSON
Engine Lock-In	Zero: Swap Docling, Tesseract, PaddleOCR freely	Low/Medium: Tied to Unstructured partitioners	100% Locked: Downstream code tightly coupled	100% Locked: Coupled to LlamaCloud service
Execution Environment	100% Local / Air-Gapped	Hybrid (Local & Cloud options)	100% Local / Local Models	Cloud API Required
Core Base Weight	Ultra-Light (pydantic & psutil only)	Heavy (Pulls torch, transformers, poppler)	Heavy (ML model dependencies)	Light (API Client wrapper)
Cost & License	100% Free & Open Source (MIT)	Freemium (Usage-based Cloud API)	Free & Open Source	Paid API (per-page credit cost)
Downstream Pipeline Reusability	Build Once: Write downstream code once against DocvionDocument	Medium: Custom element processing	Low: Rewrite downstream code per parser	Low: Tied to LlamaIndex ecosystem

---

4. Quickstart

from docvion import DocvionDocument, convert

# Option A: Bring your own engine output
doc = convert(engine="docling", raw_output=my_docling_result)

# Export clean, structured Markdown for LLM prompt context
markdown_text = doc.export_to_markdown()

# Export full-fidelity JSON (bboxes, confidence, table cells)
json_payload = doc.export_to_json()

Convenience one-liner that executes the engine and converts in a single call:

from docvion import parse

# Parse invoice and get canonical schema immediately
doc = parse("invoice.pdf", engine="docling")
print(doc.export_to_markdown())

---

5. Structure-Aware Chunking

Most chunkers split by raw token count, which routinely breaks tables mid-row and cuts sentences in half. Docvion's SemanticChunker uses the document's own structure — block types and reading order — to make chunking decisions:

• Tables are never split across chunks (atomic table blocks)
• Sentences are never split across chunks
• Headings attach to subsequent content, propagating heading_context to every chunk for context reconstruction during retrieval

from docvion import parse, SemanticChunker

doc = parse("report.pdf", engine="docling")
doc.chunks = SemanticChunker(target_tokens=512).chunk(doc)

for chunk in doc.chunks:
    print(chunk.heading_context, "->", chunk.token_count, "tokens")

Note on token counting: Docvion uses a fast character-based approximation (~4 chars/token) for chunk sizing, not an exact tokenizer for any specific LLM. This is intentional — it keeps the core package dependency-light. For exact token counts matching a specific model, count tokens post-export using that model's own tokenizer.

---

6. Supported Engines & Confidence Calibration

Engine	Native Features	Docvion Adapter	Per-Block Confidence Calibration
Docling	Layout, Tables, Bounding Boxes, Metadata	✅	Full: Native block-level confidence scores passed directly (0.0–1.0).
Tesseract	Text & Bounding Boxes (image_to_data)	✅	Averaged: Calculates mean confidence across word boxes in grouped paragraph.
PaddleOCR	Layout, PP-Structure HTML Tables	✅	Full: Region detection score & text recognition confidence.
Azure Doc AI	Planned v0.2	🔄	Coming soon
AWS Textract	Planned v0.2	🔄	Coming soon

[!NOTE] Transparent Limitation Note on Confidence Calibration: Engines handle confidence differently. Docling and PaddleOCR supply block/region confidence natively. Tesseract provides word-level confidence (conf between 0–100); Docvion averages valid word confidences across grouped lines into Block.confidence. Docvion never fabricates confidence scores—if an engine provides none, confidence remains None.

---

7. Core Schema (DocvionDocument)

All models are built on Pydantic v2, giving instant JSON serialization, strict validation, and full IDE autocomplete.

Key design principles:

• Resolution-Independent Bounding Boxes: Bounding boxes (BoundingBox) are normalized to 0.0–1.0 relative coordinates (x0, y0, x1, y1) and indexed by 1-based page number.
• Flat Table Representation: Tables (Table) are stored as flat cell lists (TableCell) with zero-based row, col, row_span, and col_span indices. Handles merged cells and irregular grids cleanly.
• Multi-Page Tables: Cells preserve global row/column indexing across page boundaries, preventing table fragmentation in multi-page documents.

class BlockType(str, Enum):
    TITLE = "title"
    HEADING = "heading"
    PARAGRAPH = "paragraph"
    TABLE = "table"
    IMAGE = "image"
    FIGURE = "figure"
    LIST_ITEM = "list_item"
    CAPTION = "caption"
    HEADER = "header"
    FOOTER = "footer"
    FOOTNOTE = "footnote"
    OTHER = "other"

---

8. Installation

The core docvion package is lightweight and depends only on pydantic>=2.0 and psutil. Heavy ML engine dependencies are kept optional so base installation takes seconds.

# Core package (zero heavy dependencies)
pip install docvion

# Optional engine extras (install only what you need):
pip install "docvion[docling]"     # Pulls docling
pip install "docvion[tesseract]"   # Pulls pytesseract & Pillow
pip install "docvion[paddleocr]"   # Pulls paddleocr
pip install "docvion[all]"         # Pulls all supported engines

---

9. Command Line Interface (CLI)

Docvion comes with a CLI for converting documents straight from your shell:

# Convert PDF using Docling into Markdown
docvion convert invoice.pdf --engine docling --format markdown

# Benchmark parsing latency, memory, and schema completeness across engines
docvion benchmark invoice.pdf --engines docling,tesseract,paddleocr --format markdown

# Check installed engine dependencies
docvion doctor

---

10. Runnable Examples

Check out the included scripts in examples/:

• examples/basic_usage.py: Quickstart demonstration converting raw outputs and exporting to Markdown/JSON/Dict.
• examples/engine_swap_demo.py: Swap between Docling, Tesseract, and PaddleOCR without modifying downstream LLM prompt code.
• examples/chunking_demo.py: Structure-aware chunking with table preservation and heading context.
• examples/benchmark_usage.py: Benchmark execution and report generation.

Run an example:

python examples/basic_usage.py
python examples/engine_swap_demo.py
python examples/chunking_demo.py
python examples/benchmark_usage.py

---

11. Third-Party Notices & Licenses

See THIRD_PARTY_NOTICES.md for full licensing and attribution information for Docling, Tesseract OCR, PaddleOCR, and Pydantic.

---

12. Positioning: Docvion & Contivon

Docvion is infrastructure for structured memory.

• Docvion: Turns any document into a clean, engine-agnostic structure.
• Contivon: Turns that structure into governed, retrievable memory.

---

13. Development & Testing

Run unit & stress tests:

python -m unittest discover tests

---

14. License & Company

Maintained by Prolixis (OPC) Pvt Ltd. Released under the MIT License.