Extract data from HTML pages that have some kind of a repetitive pattern
Project description
This package tries to find a repetitive pattern in an HTML page that contains some kind of a list (like digest pages). It extracts the sub-html text that creates the pattern, and tries to extract useful information from it. The idea is that in a typical HTML data page that contains a list of items, there will be a repetitive pattern for the human eye (the page format). If this pattern will be the most prevalent in the page (this will be the case in most pages) we will be able to recognize it and “scrape” the relevant data.
