PoC for scraping Yahoo News with sentiment analysis
Project description
StockNews
Scrape financial News from Yahoo and analyse the sentiment (PoC)
Summary
With stocknews
, you can scrape news data from the Yahoo Financial RSS Feed and store them with the sentiment of the headline and the summary.
Depending on the initialization 1 or 2 files are output as csv. No. 1 is the scraped news (optional) and no. 2 is the summary, having the summarized sentiment of news for the given date (see options) and the values.
Install
To install the package, run pip install stocknews
Usage
In order to use stocknews
to scrape news data and prepare them for your model you simply need this:
from stocknews import StockNews
...
stocks = ['AAPL', 'MSFT', 'NFLX']
sn = StockNews(stocks, wt_key='MY_WORLD_TRADING_DATA_KEY')
df = sn.summarize()
...
This returns a pandas DataFrame and saves it to data/data.csv
by default (see options)
Options
stocks
: A list of stocks to check. See http://eoddata.com/symbols.aspx for all symbols availablenews_file='news.csv'
: filename of the saved newssummary_file='data.csv'
: filename of the saved dataset, including sentiment and value per day and stocksave_news=True
: save the news file or scrape and analyse on the fly for recent newsclosing_hour=20
: Close of the exchange (NASDAQ in this case). News after closing will be taken for next trading day (skips the weekend as well)closing_minute=0
: Same asclosing_hour
wt_key=None
: Your worldtradingdata.com API Key. Get one here. Not needed ifread_rss
is called directly.
Dependencies
pandas
https://pypi.org/project/pandas/feedparser
https://pypi.org/project/feedparser/nltk
https://pypi.org/project/nltk/requests
https://pypi.org/project/requests/numpy
https://pypi.org/project/numpy/
Tests
python setup.py test
ToDo
- add more news sources
- add more tests
Changes
0.9.5
- "Initial Release"
<0.9.5:
- Testing
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.