Pure-Python HTML parser with ElementTree support.
Project description
HTMLement
Why another Python HTML Parser? There is no “HTML Parser” in the “Python” Standard Library. Actually, there is the html.parser.HTMLParser that simply “traverses the DOM tree” and allows me to be notified as each tag is being parsed. Usually, when “parsing HTML” I want to query its elements and extract data from it.
- There are a few third party “HTML parsers” available like “lxml”, “html5lib” and “beautifulsoup”.
“lxml” is the best “parser” available, fast and reliable but since it requires “C libraries”, it’s not always possible to install.
“html5lib” is a “pure-python library” and is designed to conform to the “WHATWG HTML” specification. But it is very slow at parsing HTML.
“beautifulsoup” is also a “pure-python library” but is considered by most to be “very slow”.
The “Object” of this project is to be a “pure-python HTML parser” which is also “faster” than “beautifulsoup”. And like “beautifulsoup”, will also parse invalid html. The most simple way to do this is to use XPath expressions. Python does support a simple (read limited) XPath engine inside its “ElementTree” module. A benefit of using “ElementTree” is that it can use a “C implementation” whenever available.
This “HTML Parser” extends html.parser.HTMLParser to build a tree of ElementTree.Element instances. The returned “root element” natively supports the ElementTree API.
Install
Run
pip install htmlement
-or-
pip install git+https://github.com/willforde/python-htmlement.git
Parsing HTML
Here I’ll be using a sample “HTML document” that will be “parsed” using “htmlement”:
html = """
<html>
<head>
<title>GitHub</title>
</head>
<body>
<a href="https://github.com/marmelo">GitHub</a>
<a href="https://github.com/marmelo/python-htmlparser">GitHub Project</a>
</body>
</html>
"""
# Parse the document
import htmlement
root = htmlement.fromstring(html)
Root is an ElementTree.Element and supports the ElementTree API with XPath expressions. With this I’m easily able to get both the title and all anchors in the document.
# Get title
title = root.find("head/title").text
print("Parsing: %s" % title)
# Get all anchors
for a in root.iterfind(".//a"):
print(a.get("href"))
And the output is as follows:
Parsing: GitHub https://github.com/willforde https://github.com/willforde/python-htmlement
Parsing HTML with a filter
Here I’ll be using a slightly more complex “HTML document” that will be “parsed” using “htmlement with a filter” to fetch only the menu items. This can be very useful when dealing with large “HTML documents” since it can be a lot faster to only “parse the required section” and to ignore everything else.
html = """
<html>
<head>
<title>Coffee shop</title>
</head>
<body>
<ul class="menu">
<li>Coffee</li>
<li>Tea</li>
<li>Milk</li>
</ul>
<ul class="extras">
<li>Sugar</li>
<li>Cream</li>
</ul>
</body>
</html>
"""
# Parse the document
import htmlement
root = htmlement.fromstring(html, "ul", attrs={"class": "menu"})
In this case I’m not unable to get the title, since all elements outside the filter were ignored. But this allows me to be able to extract all “list_item elements” within the menu list and nothing else.
# Get all listitems
for item in root.iterfind(".//li"):
# Get text from listitem
print(item.text)
And the output is as follows:
Coffee Tea Milk
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for htmlement-0.2.3-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c8fd630bdfc5f0f5cf3ccf7779bf737a4874794fd63f0a49d08e4ff9270c0854 |
|
| MD5 | 2097d7b48e05dd064cdd450415a31272 |
|
| BLAKE2b-256 | b0998ce1fe29a48c38a2696f3e43ace4852b645c1b78a5672037d4ae23cafddf |
