Skip to main content

heocr

The best open pipeline for getting Hebrew text out of any document.

heocr looks at a document and works out what it is: its format, and for a PDF the state of each page's text layer. It then sends each part to the right tool:

  • the text layer, where it can be trusted;
  • our own readers for office files and web pages;
  • OCR, where the text exists only as an image.

Every decision is recorded in the output. Text that could not be read is reported in unread, never dropped silently.

Status: 0.1.2: the gated 0.1.0 plus the fixes made since, which are not yet re-gated. The gate's verdicts, per format and page class, are below.

Quickstart

pip install .            # from a clone; the core needs only defusedxml, pypdfium2 and Pillow
import heocr

doc = heocr.extract("tender.pdf")   # a path or the file's bytes; any format below
doc.text        # plain text in reading order; notes, headers, footers and navigation follow the body
doc.markdown    # headings, numbered sections and tables kept
doc.to_json()   # the heocr_document.v1 record: every block with its route, tool and flags
doc.unread      # what was not read, and why
heocr tender.pdf                      # text to standard output
heocr tender.docx -o tender.md        # the format follows the suffix: .md, .json, else text
heocr scan.pdf -f json --ocr-lang heb+eng
heocr report.pdf --no-ocr             # never run OCR; pages that need it are reported unread
heocr report.pdf --force-ocr          # read every page by OCR, ignoring its text layer

The CLI prints each unread entry on stderr (-q silences them). It exits with 1 only when the file cannot be opened or an option is invalid. A document read only in part, or not at all, exits with 0, and its unread entries say why.

Formats and routes

The format is decided from the content, never from the file name. The file name is never recorded.

Input Route Notes
PDF, page by page The text layer, read with our own right-to-left line building, ruled and unruled tables, page furniture (running headers and footers) and repairs. Each page gets a class from its signals; see the class table below.
Region OCR for images on text-layer pages. Ruled tables inside such images are read cell by cell.
Whole-page OCR for scans. Ruled tables are read cell by cell; amounts in cells get a second reading by eng.
Spreads (two pages side by side) are read as two, the right half first in RTL. A web page printed from a browser has its site menu given the navigation role.
DOCX Our own WordprocessingML reader. List numbering rendered, foot/endnotes tied to their place, hidden and white text dropped and flagged, embedded images OCR'd.
DOC, RTF, ODT LibreOffice converts to DOCX (optional, see below), then the DOCX reader.
XLSX Our own reader: each sheet is a table of the rows and columns that hold a value. Cached values only, never recalculated; numbers and dates as their format shows them; hidden sheets, rows and columns read and flagged.
ODS, XLS LibreOffice converts to XLSX, then the XLSX reader.
PPTX Our own reader: slides in order, shapes in reading order by position, tables, speaker notes (role notes), pictures by OCR.
ODP, PPT LibreOffice converts to PPTX, then the PPTX reader.
HTML The standard library's parser. Encoding from the BOM, <meta> or windows-1255. Menus, page headers and footers get the role navigation. A page filled in by script is reported unread. Legislation's section numbers and margin headings are kept with their sections.
Images (PNG, JPEG, TIFF with all frames, GIF, BMP, JPEG 2000, WebP) OCR, each frame a page. DPI from the file, else 300 (flagged).

PDF page classes:

Class What the page is Route
T1 A good text layer Text layer
T2 A text layer in visual order Text layer, reordered (or OCR when words are drawn reversed)
T3 A broken font mapping Repairs; OCR for the codes left
T4 Someone else's OCR layer Ignored; OCR
T5 An image with no text layer (a scan) OCR
T6 A text layer with significant images Text layer, plus region OCR

What the release gate supports

heocr's claims come from a release gate registered before the candidate existed. Per cell, heocr is compared with stock Tesseract, Docling, MarkItDown, pymupdf4llm and a production tender-extraction pipeline, on a fresh synthetic set with exact gold. A held-out split of real documents can veto a cell. Verdicts for the release candidate 0.1.0rc1 (aggregates only):

Cell Verdict Why
DOCX Supported
DOC Supported
PDF T1 (good text layers) Supported
PDF T2 (text layers in visual order) Supported
PDF T3 (broken font mappings) Supported
PDF spread (two pages side by side) Supported
Scanned tables Supported
PDF T4 (under someone else's OCR layer) Not supported Character errors above stock Tesseract's margin. Most of the excess came from one page whose bare paper was read as text: addressed in 0.1.1; not yet re-gated (RC2 decides).
PDF T5 (scans) Not supported Character errors just above stock Tesseract's margin, from two documents that produced no record; that crash is gone in 0.1.1. Numbers missed more often than pymupdf4llm, which reads scans with Tesseract's English model: addressed in part in 0.1.1. The cell itself is not yet re-gated (RC2 decides).
PDF T6 (text layers with image regions) Not supported Vetoed by the held-out real documents: amounts in mixed right-to-left and left-to-right text came out reversed more often than stock Tesseract's.

"Not supported" means no claim is made for that cell. heocr still reads those pages, and its record still flags what it could not read. 0.1.1's fixes are not yet gated: a fresh registered gate (RC2) decides the next claims.

What the record tells you

  • Every block carries its source (text_layer, office_xml, converted_docx/xlsx/pptx, html or ocr) and its tool. OCR blocks are unverified, with their engine, settings and confidence.
  • Roles: body text is body. header, footer, footnote, endnote, comment, textbox, navigation and notes are kept in the record and out of the body flow.
  • Repairs fix characters the source meant but its font or producer broke, such as reversed brackets, ð for נ, symbol-font bullets, private-use codes and reversed digit groups. Each one fires only on evidence, and is logged with its counts. --no-repairs turns them all off: the output is then the extracted text, byte for byte.
  • Flags say where the reading is a guess. For example:
    • reading_order_unresolved_columns: side-by-side text whose order is unknown;
    • unruled_table: a table inferred from layout alone, so its page is never selection_safe;
    • ocr_short_for_ink: ink that OCR left unread.

unread: what was not read, and why

Every entry has a scope (document, page or block), a reason, and where possible the page and a box. The main reasons:

Reason When
format_not_supported A format with no route yet (EPUB, email, archives)
converter_unavailable, conversion_failed, conversion_timeout LibreOffice missing, refusing the file, or stopped at its time limit
ocr_route_unavailable, ocr_failed, ocr_found_no_text, ocr_short_for_ink Tesseract missing or failing, or a page whose ink OCR did not read
image_region_*, image_* Image regions and embedded images: not read, too small, or low confidence. A region whose text is already in the layer, or that is blank, is no loss: it is counted in the page's coverage (image_regions_text_in_layer, image_regions_blank), not here
table_row_ink_unread A table row with ink in its cells that OCR read none of
table_cell_unconfirmed A scanned table cell whose two readings (heb and the number model) differ, neither confident; both are kept in the cell
rotated_text_not_read Characters drawn at an angle other than vertical. Vertical text (90 or 270 degrees: a margin note, a stamp's caption) is read as its own lines after the page's body, flagged rotated_text
text_outside_page Characters drawn outside the page's visible box
html_rendered_by_script A web page whose content is filled in by JavaScript; print it to PDF from a browser
page_limit, time_limit, sheet_truncated, html_truncated, merges_not_applied, image_too_large A limit was reached (below)
docx_unreadable, xlsx_unreadable, pptx_unreadable, html_unreadable, image_unreadable A file that does not parse
chart_not_read, smartart_not_read, xlsx_drawing_not_read, embedded_content_not_read Charts, drawings, frames and embedded objects

Limits

Each limit, when reached, gives an unread entry or a flag:

  • pages or image frames: 2,000 (max_pages);
  • time per document: none by default; time_limit (seconds, --time-limit) is a safety net, checked before each page, frame and embedded image, and the step under way finishes;
  • input size: 512 MB;
  • LibreOffice: 180 s per conversion (convert_timeout); the conversion's processes are stopped;
  • HTML: 64M characters, 2M elements, nesting depth 100 (flattened, with no text lost);
  • XLSX: 2M cells per sheet;
  • PPTX: group nesting depth 100;
  • cell spans: 1,000;
  • renders for OCR: 300 dpi, capped at 40M pixels.

What is not detected:

  • Pointed text in a kept OCR layer (0.1.0 and 0.1.1 only). Some scans carry someone else's OCR layer with niqqud or cantillation, each mark in its own box, as Internet Archive's scans do. 0.1.0 and 0.1.1 keep such a layer but build its marks into separate pieces, so pointed lines come out fragmented. Fixed in 0.1.2. With an older version, convert those pages to images and read the images, which go through OCR.
  • A partial OCR layer: a scanned page under someone else's OCR layer that heocr keeps is flagged ocr_short_for_ink when at least 10% of the scan's ink lies away from the layer's lines and rulings. A line or two missing from a full page's layer stays under that, and is not detected.

Read once:

  • A repeated image: an image drawn again in the same document (a letterhead, a logo, a stamp) is read by region OCR where it first appears. Each later page gets a copy of that reading in its own box, flagged repeated_image_text and pointing to the first page (checks.repeated_image), with no OCR: every page keeps its text, as running headers do. A first reading that was empty or unsure repeats as counts (image_regions_repeated), not as new unread entries.

Optional tools

The core runs without them. Without them, the parts that need them are reported in unread, never guessed.

  • Tesseract (Apache-2.0), for OCR, with the heb model (and eng for the second reading of numbers). heocr finds it on the PATH, in $HEOCR_TESSERACT, or through tesseract= / --tesseract. Other options:
    • ocr_lang / --ocr-lang for the languages (default heb);
    • --ocr-model for a HeOCR model, kept in $HEOCR_MODELS.
  • LibreOffice (MPL-2.0), for DOC, RTF, ODT, ODS, XLS, ODP and PPT. heocr finds it on the PATH, in $HEOCR_SOFFICE, or through soffice= / --soffice; on macOS also in /Applications. It runs with a throwaway profile and a time limit. convert=False / --no-convert turns conversion off.

Neither is bundled. heocr records the version and executable hash of each one it uses in the record's producer.components.

Licence and credits

Apache-2.0. The core depends only on permissive libraries: pypdfium2, Pillow and defusedxml. Copyleft tools are never in a default route. See NOTICE, and THIRD_PARTY_LICENSES.md for the audit of the dependencies and the libraries they bundle.

Project

heocr is part of HeOCR, an open Hebrew OCR project. Its plan is in planning/. The project-wide decisions behind it (0019, 0020) are in heocr-engine.

Metadata

Release files for heocr 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for heocr 0.1.2
File Size Uploaded Provenance Actions
heocr-0.1.2.tar.gz 271.4 kB With provenance Details for heocr-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for heocr 0.1.2
File Interpreter ABI Platform Actions
heocr-0.1.2-py3-none-any.whl Python 3 none any Details for heocr-0.1.2-py3-none-any.whl

Total release size: 458.1 kB

Release files / heocr-0.1.2.tar.gz

Download URL heocr-0.1.2.tar.gz
Size 271.4 kB
Tags Source
SHA-256 checksum
How to use checksums
ace511756468ba31e456de27061b2feb748fc7e0ddede8208f65e2ab957551cb
BLAKE2b-256 checksum
How to use checksums
c61f4873bafc24b07d9f7197ac570f4e47e816276287dc3be841a55e46ae3725
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Release files / heocr-0.1.2-py3-none-any.whl

Download URL heocr-0.1.2-py3-none-any.whl
Size 186.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a3cb85012a6c8b83a49a03ad087e00c106f94087e7ebe8c281f883762341c73f
BLAKE2b-256 checksum
How to use checksums
9fa7b4682d38be15a641fd2b827831e6bb0c7c6df392865a681bd2d3d7d47c28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Security for heocr 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for heocr-0.1.2.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page