Skip to main content

PoC for scraping Yahoo News with sentiment analysis

Project description


Scrape financial News from Yahoo and analyse the sentiment (PoC)


With stocknews, you can scrape news data from the Yahoo Financial RSS Feed and store them with the sentiment of the headline and the summary. Depending on the initialization 1 or 2 files are output as csv. No. 1 is the scraped news (optional) and no. 2 is the summary, having the summarized sentiment of news for the given date (see options) and the values.


To install the package, run pip install stocknews


In order to use stocknews to scrape news data and prepare them for your model you simply need this:

from stocknews import StockNews
stocks = ['AAPL', 'MSFT', 'NFLX']
sn = StockNews(stocks, wt_key='MY_WORLD_TRADING_DATA_KEY')
df = sn.summarize()

This returns a pandas DataFrame and saves it to data/data.csv by default (see options)


  • stocks: A list of stocks to check. See for all symbols available
  • news_file='news.csv': filename of the saved news
  • summary_file='data.csv': filename of the saved dataset, including sentiment and value per day and stock
  • save_news=True: save the news file or scrape and analyse on the fly for recent news
  • closing_hour=20: Close of the exchange (NASDAQ in this case). News after closing will be taken for next trading day (skips the weekend as well)
  • closing_minute=0: Same as closing_hour
  • wt_key=None: Your API Key. Get one here. Not needed if read_rss is called directly.



python test


  • add more news sources
  • add more tests



  • Fixed another pathing issue...


  • removed the exit() rolleyes


  • path issues fixed. For real now...


  • fixed some path issues


  • tried to fix too many requests, added a counter for made requests to keep track


  • Suppress ntlk download messages
  • renamed


  • "Initial Release"


  • Testing

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

stocknews-0.9.11.tar.gz (5.4 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page