Skip to main content

Extract text and some metainfo from HTML, coping with malformed pages as well as possible.

Project description

This package was written for a search engine, to allow it to extract the textual content and metadata from HTML pages. It tries to cope with invalid markup and incorrectly specified character sets, and strips out HTML tags (splitting words at tags appropriately). It also discards the contents of script tags and style tags.

As well as text from the body of the page, it extracts the page title, and the content of meta description and keyword tags. It also parses meta robots tags to determine whether the page should be indexed.

The HTML parser used by this module was extracted from the Xapian search engine library (and specifically, from the omindex indexing utility in that library).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for htmltotext, version 0.7.3
Filename, size File type Python version Upload date Hashes
Filename, size htmltotext-0.7.3.tar.gz (27.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page