Trafilatura Core

Also available as: Online playground | npm package CLI & lib | Source code on GitHub Docs Getting started | Python library help Social Star us on GitHub | Follow us on GitHub

Trafilatura Core extracts main content by removing boilerplate from HTML documents.

• Two language versions — TypeScript and Python: available as a TypeScript library on npm and a Python library on PyPI.

• This is Trafilatura Core's open-source native Python implementation. It translates the TypeScript port of Python Trafilatura's fast=True extraction path. go-trafilatura served as a DOM translation aid for the TypeScript port.

• The Core in its name means it is reduced to one task: extracting main content by removing boilerplate. An optional Source URL supplies metadata and image-resolution context; it is never fetched. Use Turndown for Markdown conversion and Markdownee for crawling live websites.

• Compared with Mozilla Readability, Trafilatura and Trafilatura Core use layered structural and content heuristics with fallback and recall escalation, rather than centering extraction on the candidate scoring inherited from Arc90’s original readability.js article extractor; Trafilatura Core also offers configuration options for boilerplate removal.

This package provides the native Python library, with library APIs only. It returns cleaned HTML, diagnostics, and available page metadata; see the Python reference for language differences.

Install and use

Requires Python 3.10 or newer:

pip install trafilaturacore

Save this as clean.py:

from trafilaturacore import clean

source = """<nav>Home</nav><article><h1>Reading saved pages</h1>
<p>Save the original HTML before cleaning a page. A local copy lets
you compare the extracted article with its navigation and footer.</p>
<p>Keep the source address beside the snapshot. It provides context
when relative image links need to be resolved after extraction.</p>
</article>"""
result = clean(source)
print(result.html)

python clean.py

The result prints the article heading and paragraphs without navigation. clean() accepts a string or UTF-8 bytes. Its CleanResult has html, messages, and optional metadata; aclean() accepts the same options for async callers. Cancelling its await does not stop an already running worker.

Configure cleanup

Select precision, balanced (default), or recall for extraction; keep cleans the whole document. Image, link, table, and user-comment handling are independent. The Python reference covers options, custom policies, async usage, resource limits, and errors.

Python uses lxml and nh3. Parsing, serialization, date/URL handling, custom configuration, CSS preservation, diagnostics, and limits can differ from the TypeScript library. Dependencies install separately; platforms without compatible dependency wheels need their build prerequisites.

Extraction is not a security boundary. Before rendering untrusted output, apply sanitization appropriate to your output context and a Content Security Policy. Remote links and images may remain.

Acknowledgements

• Trafilatura — original Python implementation by Adrien Barbaresi.
• go-trafilatura — Go port by Markus Mobius, used as a DOM translation aid.

See About for extraction lineage and scope.

Support

Report problems in the issue tracker.

License

Licensed under Apache-2.0. See the compact third-party notices.