dot-parser

Turn documents into clean Markdown, then into retrieval-ready chunks.

from dot_parser import parse, chunk

markdown = parse("report.pdf")
chunks = chunk(markdown)

for c in chunks:
    print(c.section_path, len(c.content))
    # ['# Report', '## Methods', '### Analysis'] 842

Why dot-parser

A RAG pipeline needs two things from a document: faithful text, and chunks that keep their place in the document's structure. Most tools give you one or the other — parsers stop at raw text, splitters assume you already have Markdown and cut it blind to headings.

dot-parser does both in one step. It converts PDF, DOCX, PPTX, HTML, XLSX, CSV, Markdown and plain text into clean Markdown, then splits that Markdown into chunks carrying their heading hierarchy in section_path — so a chunk still knows it came from Report › Methods › Analysis.

The default install stays light: heavy backends are optional extras, and you pick per document whether to run locally or through a cloud OCR. See docs/DESIGN.md for how it compares to Docling, MarkItDown and LangChain splitters.

Features

• One parse() call for PDF, DOCX, PPTX, HTML, XLSX, CSV, Markdown and text
• Swappable PDF backends: local and fast, local and layout-aware, or cloud OCR
• Heading-aware chunking with section_path metadata
• Image extraction with per-image descriptions, from a VLM or from OCR annotations
• Batch PDF parsing through the Mistral Batch API (50% cheaper)
• Cost estimation before you send anything
• Light by default — heavy backends live behind extras

Installation

pip install dot-parser

# With optional PDF backends (each ~50 MB - 1 GB):
pip install 'dot-parser[docling]'
pip install 'dot-parser[mistral]'
pip install 'dot-parser[llama]'
pip install 'dot-parser[all]'

Requires Python 3.12+.

Quick start

from dot_parser import parse, chunk

markdown: str = parse("report.pdf")
chunks: list[Chunk] = chunk(markdown)

for c in chunks:
    print(c.section_path, c.heading, len(c.content))
    # ["# Report", "## Methods", "### Analysis"], "### Analysis", 842

For a different PDF backend:

from dot_parser import parse, parse_pdfs, Mistral, Docling

# single PDF, cloud OCR
md = parse("scanned.pdf", backend=Mistral())

# many PDFs, batched (50% off via Mistral Batch API)
mds = parse_pdfs(["a.pdf", "b.pdf", "c.pdf"], backend=Mistral())

# local layout-aware ML pipeline
md = parse("paper.pdf", backend=Docling())

API

parse(source, format=None, backend=None) -> str

Converts a document to Markdown.

• source -- file path (str or Path) or raw bytes
• format -- explicit format hint (e.g. "pdf", "docx"). Required when source is bytes, otherwise inferred from the file extension.
• backend -- optional PDF backend (Pymu, Docling, Mistral, Llama). Defaults to Pymu(). Backends only apply to PDF input.

Supported formats: .pdf, .md, .txt, .docx, .pptx, .html, .xhtml, .htm, .xlsx, .csv

Raises ParseError on unsupported formats or conversion failures.

parse_pdfs(sources, backend=None) -> list[str | None]

Converts a list of PDFs to Markdown, in input order. Backends that implement a parse_pdfs method (e.g. Mistral via the Batch API, 50% off) handle the whole list in a single optimized call. Otherwise falls back to looping parse() per source. Failed items are returned as None (no exception raised).

parse_with_images(source, format=None, *, backend=None, vlm=None, annotate_images=True, include_tables=True) -> ParseResult

Converts a .pdf, .docx or .pptx and extracts its images alongside the Markdown, keeping ![name](name) anchors at each image's position. Each image is described either by a VLM or by OCR annotations from the same call. See docs/IMAGE_EXTRACTION.md for the two available paths per format.

Backends

• Pymu(ocr_fallback=True) -- pymupdf4llm, default. Fast and CPU-only. The built-in OCR fallback kicks in when an OCR engine (tesseract, rapidocr, paddleocr) is available. Set ocr_fallback=False to force the no-OCR code path (matches a runtime image without OCR shipped).
• Docling(force_full_page_ocr=True) -- IBM Docling, layout-aware ML pipeline + Tesseract OCR. Robust on image-only and broken-encoding PDFs. Install with pip install dot-parser[docling].
• Mistral(api_key=None, model="mistral-ocr-4-0") -- Mistral OCR cloud API. Reads MISTRAL_API_KEY from env. Implements parse_pdfs via the Batch API. Install with pip install dot-parser[mistral].
• Llama(api_key=None, tier="cost_effective") -- LlamaCloud parsing API. Reads LLAMA_CLOUD_API_KEY from env. Tiers: fast, cost_effective, agentic, agentic_plus. Install with pip install dot-parser[llama].

Mistral OCR enrichments

Three optional extras on the Mistral(...) constructor, applying to the *_with_images methods. All default to off, so existing callers see no change:

from dot_parser import parse_with_images, Mistral

result = parse_with_images(
    "spec.pdf",
    backend=Mistral(
        extract_headers_footers=True,  # page furniture -> result.pages[i].header/.footer
        confidence_scores="page",  # or "word"   -> result.pages[i].*_confidence
        extract_captions=True,  # figure captions -> image.original_caption
    ),
)

for page in result.pages:
    print(page.page, page.average_confidence, page.minimum_confidence)

• extract_headers_footers moves running headers/footers out of the markdown and into ParseResult.pages, so repeated "page 4 of 27" noise stays out of retrieval chunks. It removes that text from the markdown -- the only one of the three that changes result.markdown.
• confidence_scores surfaces OCR's self-reported quality. Calibrate any threshold on your own corpus -- absolute cutoffs do not transfer. Measured over the 39 pages of the benchmark corpus (native_simple, image_only, broken_encoding), average_confidence sat at 0.985-0.989 and minimum_confidence at 0.23-0.46 on every document regardless of quality. minimum_confidence is the single worst region on the page, so it is low even on clean pages and is not a page-quality alarm by itself.
• extract_captions fills ExtractedImage.original_caption from the figure caption printed beside each image. Needs OCR 4+ and mistralai>=2.8; on older SDKs it logs a warning and leaves captions None rather than failing.

If the OCR model rejects any of these parameters, the call is retried once without them -- you get the markdown and images, minus the enrichments.

Caption-grounded image interpretation

Mistral OCR's inline annotations (annotate_images=True) are produced from the cropped image alone -- the annotating model sees no caption and no page text, which on domain-specific figures yields confidently wrong descriptions. interpret_images() replaces that pass with client-side VLM calls grounded in each figure's own caption and surrounding markdown:

from dot_parser import parse_with_images, interpret_images, Mistral, MistralVLM

result = parse_with_images(
    "paper.pdf",
    backend=Mistral(extract_captions=True),
    annotate_images=False,  # skip the ungrounded inline pass
)
result = interpret_images(result, MistralVLM())

Same number of VLM calls as annotate_images=True, but each one carries the caption. On a figure-heavy biomechanics paper this turned "magnetic field components" / "fluid flow through a curved pipe" into accurate descriptions of the actual rib diagrams.

chunk(markdown, max_tokens=None, merge=True) -> list[Chunk]

Splits Markdown into chunks with heading-hierarchy metadata.

• markdown -- Markdown string (typically output of parse())
• max_tokens -- maximum tokens per chunk. Auto-computed from the 95th percentile of section sizes (clamped 64-512) when None.
• merge -- when True (default), merges small adjacent sections sharing the same parent heading to reduce fragmentation.

The chunking algorithm is described in docs/DESIGN.md.

Chunk

@dataclass(frozen=True)
class Chunk:
    content: str  # full text of the chunk (heading + body)
    heading: str | None  # the heading line, if any
    section_path: list[str]  # hierarchy of headings leading to this chunk

ParseError

Raised by parse() when a document cannot be converted.

Cost estimation

cost_per_1k_pages(backend) and estimate_cost(backend, pages, batch=False) report what a cloud backend will cost, without instantiating it or needing an API key.

Stability

dot-parser follows semantic versioning: everything exported from the top-level package is covered, anything underscore-prefixed is internal and may change in any release. Public names are never removed without a deprecation period.

dependencies = ["dot-parser>=1.0,<2"]

See docs/VERSIONING.md for the full policy.

Roadmap

• Table extraction as structured data, not just Markdown
• Streaming parse for very large documents
• Additional cloud OCR backends

Documentation

Contributing

Contributions are welcome. See CONTRIBUTING.md for the DCO sign-off requirement, the licensing terms that apply to contributions, and how to submit a change.

Licence

Copyright (C) 2026 Kannon For Deep Tech (deepika)

This software is distributed under the GNU Affero General Public License, version 3 or later — see LICENSE.md.

A commercial licence is available for use in proprietary environments. Contact: louis.letarnec@deepika.ai