turbohtml 0.3.0

A fast, fully typed HTML toolkit for Python, powered by a C-accelerated core.

turbohtml

A fast, fully typed HTML toolkit for Python with a C-accelerated core. turbohtml escapes and unescapes HTML to match the standard library byte for byte, tokenizes markup with a WHATWG-conformant streaming tokenizer, and parses whole documents into a navigable element tree you query with CSS selectors and serialize back to conformant HTML. Each operation runs several times faster than its pure-Python counterpart and supports the free-threaded build.

Install

$ pip install turbohtml

Wheels ship per interpreter for CPython 3.10–3.15 (including free-threading), so there is nothing to compile.

Usage

Escape text before interpolating it into HTML so it cannot break out of its context:

>>> import turbohtml
>>> turbohtml.escape('<a href="?x=1&y=2">Tom & Jerry</a>')
'&lt;a href=&quot;?x=1&amp;y=2&quot;&gt;Tom &amp; Jerry&lt;/a&gt;'

Inside a text node the quotes are safe, so pass quote=False to keep the output smaller:

>>> turbohtml.escape('He said "hi" & left', quote=False)
'He said "hi" & left'

Turn HTML character references back into text, following the full HTML5 rules (named, numeric, and longest-match references that omit the trailing semicolon):

>>> turbohtml.unescape("café & résumé 🎉")
'café & résumé 🎉'

escape and unescape reproduce html.escape and html.unescape exactly, so turbohtml is a drop-in replacement on hot paths.

Tokenize markup into a stream of tokens that follows the WHATWG tokenization algorithm:

>>> for token in turbohtml.tokenize('<p class="x">Tom &amp; Jerry</p>'):
...     print(token.type.name, token.tag or token.data, token.attrs)
START_TAG p [('class', 'x')]
TEXT Tom & Jerry None
END_TAG p []

For incremental input, Tokenizer.feed() returns the tokens completed by each chunk and close() flushes the rest:

>>> tokenizer = turbohtml.Tokenizer()
>>> [token.tag for token in tokenizer.feed("<div><sp")]
['div']
>>> [token.tag for token in tokenizer.feed("an>")]
['span']
>>> list(tokenizer.close())
[]

Parse a whole document into a tree and walk it with find, find_all, and the navigation accessors:

>>> doc = turbohtml.parse('<ul><li>one<li>two</ul>')
>>> [li.text for li in doc.find_all('li')]
['one', 'two']
>>> doc.find('ul').children[0].tag
'li'

Query with a CSS selector, and serialize a node back to HTML with the escaping you choose:

>>> from turbohtml import Formatter
>>> doc = turbohtml.parse('<article><h1>Tea</h1><p class=note>café &amp; cake</p></article>')
>>> doc.select_one('p.note').text
'café & cake'
>>> doc.select_one('p').serialize(formatter=Formatter.NAMED_ENTITIES)
'<p class="note">caf&eacute; &amp; cake</p>'

Pass bytes to sniff the encoding the WHATWG way (byte-order mark, then a <meta> declaration):

>>> doc = turbohtml.parse(b'<meta charset="iso-8859-2"><p>\xe1</p>')
>>> doc.encoding, doc.find('p').text
('iso-8859-2', 'á')

Parse a fragment as the contents of a context element, the way innerHTML does:

>>> cell = turbohtml.parse_fragment('<td>data', context='tr')
>>> cell.tag, cell.text
('tr', 'data')

Performance

turbohtml's C core makes every operation several times faster than its pure-Python counterpart, and it leads the C libraries on every read-path benchmark too. Measured with pyperf on an Apple M4:

• escape and unescape match the standard library byte for byte while running several times faster, up to 22× on no-op text and 13× on entity-dense input.
• tokenize is 9–16× faster than html.parser wherever markup appears.
• parse builds a full WHATWG tree 2–5× faster than the C parsers lxml and selectolax, and 30–80× faster than the pure-Python BeautifulSoup and html5lib.
• find_all and CSS select outrun lxml's C XPath and cssselect at every size (2–40×) and BeautifulSoup by 100×, and serialization is the fastest of the four.

See the performance page for the full sectioned tables and the methodology.

Documentation

Full documentation, including tutorials, how-to guides, the API reference, and the design rationale, lives at turbohtml.readthedocs.io.

License

turbohtml is released under the MIT license.