NewsGrabber is a Python library that parses Google News sitemap structures into Python objects, enabling developers to easily extract and analyze news-related metadata.
Features
Parses Google News sitemaps into structured Python objects.
Handles sitemap parsing with robust error tolerance.
Lightweight and efficient, leveraging:
lxml for fast XML parsing.
requests for HTTP requests.
python-dateutil for flexible date parsing.
Python 3.11+ compatible.
Installation
Install NewsGrabber via pip:
pip install newsgrabberUsage
Parsing a Google News Sitemap
from newsgrabber import NewsGrabber
grabber = NewsGrabber("https://www.bbc.com/sitemaps/https-sitemap-com-news-1.xml")
grabber.parse()
print("\n".join(x.title for x in grabber.news_urls[:5]))Example Output
BBC Look East: Latest weather forecast for the East
Syria country profile
Namibia country profile
How working parents can get 15 and 30 hours free childcare
South East England weather forecastRequirements
NewsGrabber requires Python 3.11+ and the following dependencies:
lxml>=5.3.0: For XML parsing.
requests>=2.32.3: For HTTP requests.
python-dateutil>=2.1,<3.0.0: For flexible date parsing.
Development and Testing
To set up a development environment:
Clone the repository: bash git clone https://github.com/yibudak/newsgrabber cd newsgrabber
Install dependencies: bash pip install -e .[test]
Run tests: bash pytest
Contributing
Contributions are welcome! If you’d like to contribute, please fork the repository and submit a pull request. Make sure to include tests for any new functionality.
License
This library is licensed under the AGPL-3.0 License.
1 maintainer
Metadata
1 maintainer
Release files for newsgrabber 24.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| newsgrabber-24.1.2.tar.gz | 43.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| newsgrabber-24.1.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 77.3 kB
Release files / newsgrabber-24.1.2.tar.gz
| Download URL | newsgrabber-24.1.2.tar.gz |
|---|---|
| Size | 43.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bd6cd546f7d836f7ce4930ccaff4f151af3783f5158cb06cfca22e97a056a08a
|
|
BLAKE2b-256 checksum How to use checksums |
c4817c8f542af57ea5654000b02f753d0603e41ace7cff13c08507c4ce96eaa1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.0.1 CPython/3.12.8
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 20, 2024.
Transparency logRelease files / newsgrabber-24.1.2-py3-none-any.whl
| Download URL | newsgrabber-24.1.2-py3-none-any.whl |
|---|---|
| Size | 33.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3745bd39c2b45198d64ff9329039d51d819ed73d0743575f3ac5be3491522a00
|
|
BLAKE2b-256 checksum How to use checksums |
2781784afa49991ae24b6bed716ed710e2672a25df20616bb4627717e5e0f386
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.0.1 CPython/3.12.8
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 20, 2024.
Transparency log
