Sitemap scraper for news article selection within a certain time range
Project description
About
This module provides the SitemapRange class and a tool to allow command-line usage sitemap_fetch.py.
The class SitemapRange is meant primarily as a generic building block for creating news aggregating applications where the datasources are spec-compliant news websites.
There are some fault-tolerance features included to deal with some inconsistencies in sitemaps.
Install
To install from pypi:
pip install --user sitemap-range-fetch
Usage
Fetching all news articles on cnn.com in the past 6 days, and format the result as JSON:
sitemap_fetch.py --site "https://cnn.com" --format json --daysago 6
More custom filtering can be done by using the class SitemapRange
Details
This module is provided as is under MIT License.
For extensions, customizations or business inquiries you can get in touch here.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sitemap-range-fetch-0.9.2.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f8ac8f9b18c67fc4d56ee76f5d08a43f00227fb8ec44a6d2785231b8c2811120 |
|
| MD5 | 42ddd4b95b6f4ef4203d01e41d07cef9 |
|
| BLAKE2b-256 | d3786ec948ef5c73e5874ec92be67baffdf6623fb8bfd5f3f16e75c3bafeb7df |
Hashes for sitemap_range_fetch-0.9.2-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 387015753c32fd4d767072288743879592a65b01234a50108dec31df92a23272 |
|
| MD5 | c37c275d18064ab80d2a6a6649f856bf |
|
| BLAKE2b-256 | a546d10a3622b5c0fa53823dfbfa6289c0dcd2a15a2e45e053d7719de2e2b665 |
