Python NewsCollector

As the internet has grown, the available sources of information at our disposal have equally grown. Nowadays, if you want to update yourself with the most important news of the day, you have a vast variety of news sources to choose from. Since we have that many news sources at our disposal, instead of manually going through all their content...

Couldn't we let automation pick the top news stories from various newspapers for us, and nicely combine them into a newsletter?

This is what the Python NewsCollector can do for us!

For a detailed usage guide, please refer to the official NewsCollector Usage Documentation.

Read more about how the algorithm of NewsCollector works in my Medium article.

Description

The Python NewsCollector lets you define a variety of news sources from which it will pick the most relevant articles and bundle these in a nice HTML-based newsletter.

View the full sample newsletter in PDF format here.

The NewsCollector algorithm scrapes the source links provided and compares the articles it found based on their similarity. If it finds multiple articles from different sources covering similar topics, these will be considered as being relevant articles and will be included in the output newsletter.

Basic Usage

You can run the NewsCollector algorithm as follows:

from newscollector import *

newsletter = NewsCollector()
output = newsletter.create()

This will run the full NewsCollector pipeline by scraping the sources from the sources.json file and outputting an HTML newsletter.

The output object will hold the location path of the generated newsletter, so that you can easily retrieve it programmatically:

output
> 'C:\\Output\\Path\\newsletter.html'

CLI Usage

The NewsCollector can also be run directly via the CLI with the following parameters:

newscollector.py [-h] [-s [SOURCES]] [-n [NEWS_NAME]] [-d [NEWS_DATE]] 
                 [-t [TEMPLATE]] [-o [OUTPUT_FILENAME]] [-a [AUTO_OPEN]]
                 [-r [RETURN_DETAILS]]

Output

The NewsCollector will output an HTML newsletter with the most relevant articles it found while scraping the sources provided.

By default, the output newsletter will be created as an HTML file in the installation directory of your package, saved in the folder rendered under the filename newsletter_YYYY-MM-DD.html, where the date is the respective date the NewsCollector scraped its articles from.

View the full sample newsletter in PDF format here.

To adjust the default settings, please refer to Additional Parameters.

Additional Parameters

You can customize the NewsCollector algorithm with the following optional parameters:

newsletter = NewsCollector(sources="sources.json", news_name="Daily News Update", 
                           news_date=date.today(), template='newsletter.html', 
                           output_filename='default', auto_open=False, 
                           return_details=False)

For a detailed usage guide, please refer to the official NewsCollector Usage Documentation.