Easily query articles
Python package that helps you easily retrieve complete web articles.
pip3 install newsdatascraper
from newsdatascraper import Scraper #To first get a single article on a topic new_scraper = Scraper('mock-api-key') articles = new_scraper.fetch_all_articles(query='two sigma', pageSize = 10) """ We support two APIs: NewsApi and GNewsApi To control the API being used change the argument of mode to either 'NEWSPAPER' or 'GNEWS' """ new_scraper = Scraper('mock-api-key', mode = 'GNEWS') articles = new_scraper.fetch_all_articles(query='two sigma', pageSize = 10, dateFrom = "2019-08-04", dateTo = "2019-08-10") #To access individual articles and their properties first_article = articles.articles print(first_article.content) #We also provide helper functions to serialize the data articles.toCsv('test.csv') articles.toPickle('test.pickle') articles.toJson()
Please look at rate limits in the APIs to determine your prefered usage
Working on the Project
pylama -o setup.cfg .
Run tests + code coverage
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.