trafipy

Python bindings for the rs-trafilatura crate — fast heuristic main-content extraction from HTML, in-process, no subprocess.

Unofficial bindings, not affiliated with the rs-trafilatura project.

Install

pip install trafipy

Linux x86-64 wheels are published (manylinux_2_34, abi3 for Python 3.12+), so glibc 2.34 or newer is required. On anything else pip falls back to the sdist, which needs a Rust toolchain to compile.

Use

import trafipy

result = trafipy.extract(html, url="https://example.com/page", favor_recall=True)
result.markdown     # GitHub Flavored Markdown, "" when nothing was extracted
result.text         # plain text of the same region
result.quality      # the crate's own predicted F1, in [0, 1]
result.confidence   # page-type classification confidence, or None
result.warnings     # list[str]

url is the document's own address, used by the crate as context for canonical-URL and hostname handling. It does not absolutize links in the returned markdown or text — those come back as the document wrote them. It is never fetched: there is no HTTP client anywhere in the dependency tree, and this library makes no network requests.

Options

extract(html, *, url=None, include_tables=True, include_links=True, include_images=True, include_comments=False, favor_recall=True, min_extracted_size=100, deduplicate=False)

Every option is passed straight through to the crate and every value comes back as the crate produced it. The binding makes no extraction decisions of its own.

Two decisions worth knowing

One call returns both textual forms. The crate's ExtractResult carries content_text and content_markdown together, so output_markdown is set unconditionally and you pay for one parse. The Python trafilatura API returns one format per call, so getting both there costs two parses.

Metadata is not exposed. The crate extracts titles, dates and authors, but this binding returns content only. If you already derive your own metadata, surfacing a second set invites mixing them, and two provenance stories for one field is worse than one.

Versioning

rs-trafilatura is pinned exactly (=0.2.2), and the pin is not a formality. The crate is pre-1.0, so a minor release can change extraction output — meaning the same page yields different content across an unpinned upgrade. Bump it deliberately and re-check against your own fixtures when you do.

Build from source

maturin develop                  # build and install into the current venv
maturin build --release --out dist --compatibility manylinux_2_28

Requires Rust 1.85+ (the dependency tree uses edition 2024).

License

Dual-licensed under MIT or Apache-2.0, at your option — matching the upstream crate.

The wheel statically links rs-trafilatura and its dependency tree; their license notices are collected in LICENSE-THIRD-PARTY.md.