Skip to main content

A web parser wrapper on top of lxml and selectolax

Project description

A web content parser using Python lxml


The library is compatible with Python3. Python2 is currently not supported.


Install the package using pip.

pip install webparser-py

**Convert to Document**

Accept the html content document, convert it to the doc element, if we want to convert relative links to absolute links,
we pass the domain url to the absolute links.


from webparser.parser import convert_to_doc

doc = convert_to_doc('HTML content', '')


**class FeedParser()**

Feed parser class is used for parsing the feed through the response content or using a URL.

from webparser.parser import FeedParser

feed = FeedParser() # optional feed URL can be provided.
parsed_links = feed.parse(url='') # url will override constructor feed URL.


Check if the website/URL has a RSS feed link present.

- Check the document with Mimetype of links using XPATH.
- Fuzzy URL search e.g /feed at the end of the website URL. (Attempted if no links for the RSS URL found)

from webparser.parser import has_rss_feed
rss_links = has_rss_feed(doc=html_content, url=website_url)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for webparser-py, version 0.3
Filename, size File type Python version Upload date Hashes
Filename, size webparser_py-0.3-py3-none-any.whl (5.3 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size webparser-py-0.3.tar.gz (4.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page