Skip to main content

GatherNews

Gathers unstructured news data and commits it to a SQLite3 database.The goal for GatherNews is to quickly and simply capture news data.

GatherNews allows you to specify which News sites you want to capture by providing the RSS link in “feeds_list.txt” like this:

http://feeds.reuters.com/Reuters/worldNews
http://rss.cnn.com/rss/money_latest.rss

You can then gather the news using the load_db() method:

>>> # Create new tables if any new RSS feed addresses have been added
>>> # Populate all tables with RSS news feeds
>>> # Remove duplicate entries
>>> capture_feeds.load_db()

The examples folder contains working code for each module.

Features

  • Creates tables with a predefined schema

  • Populates each table in the SQLite3 database with articles

  • Removes duplicate articles from each table

  • Includes bug fixes for issues raised about version 0.1.0

  • Faster than version 0.1.0, see the benchmarks!

Installation

To install GatherNews use pip:

$ pip install gathernews

Testing

After installation, you can launch the test suite from outside the source directory (you will need to have nosetests installed):

$ nosetests --exe GatherNews

Bug Resolution

In GatherNews 0.1.0, a bug was introduced that does not allow you to add new RSS feeds to the ‘feeds_list.txt’ after your initial call of the create_tables() method. Before using version 0.2.0 on a preexisting database call the following method.

>>> path = "path/to_your/existing_sqlite3_db/
>>> from gathernews.gRSS import CaptureFeeds
>>> capture_feeds = CaptureFeeds(path)
>>> capture_feeds.fix_create_table_bug()

If this gives you any problems, email me directly (tylers.pile@gmail.com). Thanks.

Documentation

Documentation is available at http://gathernews.readthedocs.org/en/latest/

Contribute

  1. Issue tracker is here: https://github.com/Bonza-Times/GatherNews/issues

  2. Fork it!

Metadata

Release files for GatherNews 0.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for GatherNews 0.2.1
File Size Uploaded Provenance Actions
GatherNews-0.2.1.tar.gz 10.1 kB Without provenance Details for GatherNews-0.2.1.tar.gz

Release files / GatherNews-0.2.1.tar.gz

Download URL GatherNews-0.2.1.tar.gz
Size 10.1 kB
Tags Source
SHA-256 checksum
How to use checksums
aa9fefda00ab2b4a028320e7cadf9c3c83be5961e831a3b9a93383206ae3b3a2
BLAKE2b-256 checksum
How to use checksums
11f0ddb4af6be78e9295c723251bf58cd102a9590f03a256ea5af099680d4567
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for GatherNews 0.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page