A web scraper for ticker symbols from yahoo finance
Produces .csv, .xlsx, .json and .yaml files (All files contain same data but in a different format) for stocks, ETF, futures, indexes, mutual funds, currency, warrants and bonds. The ticker symbol, company name and exchange are saved for all symbols. The stock symbols also have a category name.
It gets its data from https://finance.yahoo.com/lookup/. Please note: it is not possible to get all the symbols due to limitations set by Yahoo. About 75%-90% of all symbols are gathered using this script depending on type.
Python 2.7 or Python 3.5+
From python package manager (preferred):
pip install Yahoo-ticker-downloader
python setup.py install
usage: YahooTickerDownloader.py [-h] [-i] [-e] [-E EXCHANGE] [-s SLEEP] [-p] [-m MARKET] type positional arguments: type The type to download, this can be: future mutualfund etf stocks index currency optional arguments: -h, --help show this help message and exit -i, --insecure use HTTP instead of HTTPS -e, --export export immediately without downloading (Only useful if you already downloaded something to the .pickle file) -E EXCHANGE, --Exchange EXCHANGE Only export ticker symbols from this exchange (the filtering is done during the export phase) -s SLEEP, --sleep SLEEP The time to sleep in seconds between requests -p, --pandantic Stop and warn the user if some rare assertion fails -m MARKET, --market MARKET Specify the Region of queried exchanges (us = USA+Canada, dr=Germany, fr=France, hk=Hongkong, gb=United Kingdom, default= all)
The first positional argument must be one of the following: stocks etf future index mutualfund currency warrant bond
For example to download all stock symbols you run it like:
The program takes several hours before it is finished so if you don’t need all symbols it might be a good idea to specify the Region of the exchanges you’re interested in. The program supports suspending and resuming a download. Press CTRL+C to suspend download. Restart the program in the same working directory to resume downloading. It is possible to export a partially downloaded results using the -e flag.
Example of CSV output:
Ticker,Name,Exchange,categoryName,categoryNr ENZ,"Enzo Biochem Inc.",NYQ,"Medical Laboratories & Research",525 ENZN,"Enzon Pharmaceuticals Inc.",NMS,Biotechnology,515 ENZR,"Energizer Resources Inc.",PNK,,0 EOAA.DE,"E.ON AG",GER,"Diversified Utilities",913 KMX,"CarMax Inc.",NYQ,"Auto Dealerships",744 KMY.MU,KIMBERLY-CLARK,MUN,"Personal Products",323 KN1.DU,ECOUNION,DUS,"Business Software & Services",826 KNCAY,"Konica Minolta Holdings Inc.",PNK,,0 KND,"Kindred Healthcare Inc.",NYQ,"Long-Term Care Facilities",523 KNDI,"Kandi Technologies, Corp",NGM,"Auto Manufacturers - Major",330 ...ect
Depending on the type you are downloading, you will get between 3.000 and 100.000+ entries.
Download history for symbols: ystockquote
- Version 2.2.0 (2018-01-31)
- Continue exporting to different formats if one export fails ( #41 )
- Check robots.txt ( anti-feature )
- Version 2.1.1 (2017-08-02)
- A too old requests dependency was listed ( #35 )
- Version 2.1.0 (2017-05-10)
- Added market parameter ( pull request #33 )
- Version 2.0.1 (2017-05-07)
- Fixed issue where all downloads except stock and currency stopped working.
- Version 2.0.0 (2017-05-05)
- Switched over to JSON api
- Version 1.0.0 (2017-04-04)
- Reverted some changes from 0.10.0. Bond is back. Reverted back to English site instead of German.
- Resolved CSV issue again. Closes #23 and #16.
- Merged #26 Workaround Y! b>2000 limit
- Scraper now scrapes a lot more at the expense of runtime.
- Support for python2 is back. Latest python 2 & 3 are supported.
- Removed xls support
- Added xlsx support (#29)
- Version 0.10.1 (2017-02-04)
- More descriptive help message
- Version 0.10.0 (2017-02-02)
- Removed bond downloading option.
- Uses different yahoo source. Fixes #18
- Removed python2 from classifiers. Related to #16
- Version 0.9.0 (unreleased)
- Added a flag to restrict output to specific stock exchanges.
- Version 0.8.1 (2016-08-17)
- Workaround for #7 : downloading interruption
- Solution for #9 : UnicodeEncodeError
- Version 0.7.0 (2016-03-20)
- Added –export option. It will transcode the .pickle file immediately to the desired output formats.
- Version 0.6.0 (unreleased)
- Add 3 retries with an exponential back-off if HTTPError or ChunkedEncodingError is raised when processing _fetchHtml.
- Version 0.5.0 (2015-08-16)
- Allows downloading using a insecure connection.
- The temporarily download file-names now include the ticker type.
- Version 0.4.0 (2014-10-28)
- Warrant symbols can now be downloaded.
- Bond symbols can now be downloaded.
- Version 0.3.0 (2014-08-14)
- Use HTTPS instead of HTTP
- Retry to fetch a page if it contains no symbols (A “fix” for issue #4)
- Renamed all ‘Curreny’ to ‘Currency’
- Relative imports are used
- Fix: .csv file it outputs is encoded in UTF-8 when using python2
- Performance: Considerable reduced memory consumption
- It now outputs .json, .yaml and .xls files in addition to .csv
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size Yahoo-ticker-downloader-2.2.0.tar.gz (10.0 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for Yahoo-ticker-downloader-2.2.0.tar.gz