eScrape Plan

A tool for extracting data from HTML pages in complex and extensible ways according to plans crafted to describe that data.

eScrape Plan starts at the top of an HTML document and compares every tag in a "flattened" sequence against a series of markers, which are HTML tags you expect to find. The content between those markers are stored internally as sections in a document. Each section can then be acted on by extractors, which pull data out of the HTML and store them. Further, after a document has been fully parsed and all of this data has been extracted, the data can be run through outputters to produce artifacts from the data.

Today, eScrape Plan can do things like:

• Parse a long document into smaller sections
• Categorize those sections internally by name/ID
• Run find/replace operations on that content (to do things like normalize newline characters in the source)
• Reorganize this content arbitrarily and produce new HTML files with any arrangement of that content
• Export textual/HTML content into SQLite databases for later querying
• Convert HTML <table> content into SQLite databases

Usage

I'll have better documentation on before too much longer. Until then...

pip install escrape-plan