Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A web scraper for ticker symbols from yahoo finance

Project Description

Produces .csv, .xlsx, .json and .yaml files (All files contain same data but in a different format) for stocks, ETF, futures, indexes, mutual funds, currency, warrants and bonds. The ticker symbol, company name and exchange are saved for all symbols. The stock symbols also have a category name.

It gets its data from https://finance.yahoo.com/lookup/. Please note: it is not possible to get all the symbols due to limitations set by Yahoo. About 75%-90% of all symbols are gathered using this script depending on type.

Requirements

Python 2.7 or Python 3.5+

Install

From python package manager (preferred):

pip install Yahoo-ticker-downloader

From source:

python setup.py install

Example Usage

usage: YahooTickerDownloader.py [-h] [-i] [-e] [-E EXCHANGE] [-s SLEEP] [-p]
                                [-m MARKET]
                                type

positional arguments:
  type                  The type to download, this can be: future mutualfund
                    etf stocks index currency

optional arguments:
  -h, --help            show this help message and exit
  -i, --insecure        use HTTP instead of HTTPS
  -e, --export          export immediately without downloading (Only useful if
                        you already downloaded something to the .pickle file)
  -E EXCHANGE, --Exchange EXCHANGE
                        Only export ticker symbols from this exchange (the
                        filtering is done during the export phase)
  -s SLEEP, --sleep SLEEP
                        The time to sleep in seconds between requests
  -p, --pandantic       Stop and warn the user if some rare assertion fails
  -m MARKET, --market MARKET
                        Specify the Region of queried exchanges (us =
                        USA+Canada, dr=Germany, fr=France, hk=Hongkong,
                        gb=United Kingdom, default= all)

The first positional argument must be one of the following: stocks etf future index mutualfund currency warrant bond

For example to download all stock symbols you run it like:

YahooTickerDownloader.py stocks

The program takes several hours before it is finished so if you don’t need all symbols it might be a good idea to specify the Region of the exchanges you’re interested in. The program supports suspending and resuming a download. Press CTRL+C to suspend download. Restart the program in the same working directory to resume downloading. It is possible to export a partially downloaded results using the -e flag.

Example of CSV output:

Ticker,Name,Exchange,categoryName,categoryNr
ENZ,"Enzo Biochem Inc.",NYQ,"Medical Laboratories & Research",525
ENZN,"Enzon Pharmaceuticals Inc.",NMS,Biotechnology,515
ENZR,"Energizer Resources Inc.",PNK,,0
EOAA.DE,"E.ON AG",GER,"Diversified Utilities",913
KMX,"CarMax Inc.",NYQ,"Auto Dealerships",744
KMY.MU,KIMBERLY-CLARK,MUN,"Personal Products",323
KN1.DU,ECOUNION,DUS,"Business Software & Services",826
KNCAY,"Konica Minolta Holdings Inc.",PNK,,0
KND,"Kindred Healthcare Inc.",NYQ,"Long-Term Care Facilities",523
KNDI,"Kandi Technologies, Corp",NGM,"Auto Manufacturers - Major",330
...ect

Depending on the type you are downloading, you will get between 3.000 and 100.000+ entries.

Further resources

Download history for symbols: ystockquote

Changelog

  • Version 2.1.1 (2017-08-02)
    • A too old requests dependency was listed ( #35 )
  • Version 2.1.0 (2017-05-10)
    • Added market parameter ( pull request #33 )
  • Version 2.0.1 (2017-05-07)
    • Fixed issue where all downloads except stock and currency stopped working.
  • Version 2.0.0 (2017-05-05)
    • Switched over to JSON api
  • Version 1.0.0 (2017-04-04)
    • Reverted some changes from 0.10.0. Bond is back. Reverted back to English site instead of German.
    • Resolved CSV issue again. Closes #23 and #16.
    • Merged #26 Workaround Y! b>2000 limit
    • Scraper now scrapes a lot more at the expense of runtime.
    • Support for python2 is back. Latest python 2 & 3 are supported.
    • Removed xls support
    • Added xlsx support (#29)
  • Version 0.10.1 (2017-02-04)
    • More descriptive help message
  • Version 0.10.0 (2017-02-02)
    • Removed bond downloading option.
    • Uses different yahoo source. Fixes #18
    • Removed python2 from classifiers. Related to #16
  • Version 0.9.0 (unreleased)
    • Added a flag to restrict output to specific stock exchanges.
  • Version 0.8.1 (2016-08-17)
    • Workaround for #7 : downloading interruption
    • Solution for #9 : UnicodeEncodeError
  • Version 0.7.0 (2016-03-20)
    • Added –export option. It will transcode the .pickle file immediately to the desired output formats.
  • Version 0.6.0 (unreleased)
    • Add 3 retries with an exponential back-off if HTTPError or ChunkedEncodingError is raised when processing _fetchHtml.
  • Version 0.5.0 (2015-08-16)
    • Allows downloading using a insecure connection.
    • The temporarily download file-names now include the ticker type.
  • Version 0.4.0 (2014-10-28)
    • Warrant symbols can now be downloaded.
    • Bond symbols can now be downloaded.
  • Version 0.3.0 (2014-08-14)
    • Use HTTPS instead of HTTP
    • Retry to fetch a page if it contains no symbols (A “fix” for issue #4)
    • Renamed all ‘Curreny’ to ‘Currency’
    • Relative imports are used
    • Fix: .csv file it outputs is encoded in UTF-8 when using python2
    • Performance: Considerable reduced memory consumption
    • It now outputs .json, .yaml and .xls files in addition to .csv
Release History

Release History

This version
History Node

2.1.1

History Node

2.1.0

History Node

2.0.1

History Node

2.0.0

History Node

1.0.0

History Node

0.10.1

History Node

0.10.0

History Node

0.8.1

History Node

0.7.0

History Node

0.5.1

History Node

0.5.0

History Node

0.4.3

History Node

0.4.2

History Node

0.4.1

History Node

0.4.0

History Node

0.3.0

History Node

0.2.8

History Node

0.2.7

History Node

0.2.6

History Node

0.2.5

History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

History Node

0.1.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
Yahoo-ticker-downloader-2.1.1.tar.gz (9.6 kB) Copy SHA256 Checksum SHA256 Source Aug 2, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting