Zipline extension to provide bundles of data from Norgate Data into the Zipline algorithmic trading library for the Python programming language
Project description
Project description
Creates an interface between Norgate data and Zipline, the pythonic algorithmic trading library, to create bundles.
This is currently in alpha stage testing to selected users of Norgate Data.
Currently, equities are supported. Futures data support is coming very soon.
Installation
pip install zipline-norgatedata
Requirements
- Python 3.5 (Note: 3.6 and 3.7 will be supported only when Zipline supports this)
- Zipline 1.3
- Microsoft Windows
- An active Norgate Data subscription
Usage
Add the following lines at the top of your extension.py file (typically located at c:\users<username>.zipline)
from pandas import Timestamp
from norgatedata import StockPriceAdjustmentType
from zipline_norgatedata import register_norgatedata_equities_bundle,register_norgatedata_futures_bundle
Then create as many bundles as you desire. These bundles will use one or more watchlists from your Norgate Data installation.
Here are some examples:
bundlename = 'norgatedata-sp500-backtest'
watchlists = ['S&P 500 Current & Past']
stock_price_adjustment_setting = StockPriceAdjustmentType.TOTALRETURN
start_session = Timestamp("1990-01-01",tz='utc')
end_session = Timestamp.now(tz='utc')
calendar_name = 'NYSE'
register_norgatedata_equities_bundle(bundlename,stock_price_adjustment_setting,watchlists,start_session,end_session,calendar_name)
bundlename = 'norgatedata-russell3000-backtest'
watchlists = ['Russell 3000 Current & Past','Russell 3000 indexes']
stock_price_adjustment_setting = StockPriceAdjustmentType.TOTALRETURN
start_session = Timestamp("1990-01-01",tz='utc')
end_session = Timestamp.now(tz='utc')
calendar_name = 'NYSE'
register_norgatedata_equities_bundle(bundlename,stock_price_adjustment_setting,watchlists,start_session,end_session,calendar_name)
bundlename = 'norgatedata-cme-futures'
watchlists = ['CME Futures']
start_session = Timestamp("2000-01-01",tz='utc') # Start date of data ingestion - NOTE: zipline cannot handle dates prior to 1990
end_session = Timestamp.now(tz='utc')
calendar_name = 'us_futures'
register_norgatedata_futures_bundle(bundlename,watchlists,start_session,end_session,calendar_name)
Note: You'll need to create your own watchlist(s) for use with futures as there's no default watchlists for futures. This is done from within the Norgate Data Updater app.
In the above example, we also have a static watchlist called Russell 3000 indexes that contain $RUA and $RUATR. This is useful for trading systems where you want to look at the overall index and not just the constituents.
Metadata
The following fields are available in the metadata dataframe: start_date, end_date, ac_date, symbol, asset_name, exchange, exchange_full, asset_type, norgate_data_symbol, norgate_data_assetid.
Zipline Limitations
- Zipline can only handle equities data from 1990 onwards.
- Zipline can only handle futuress data from 2000 onwards.
- Zipline has unnecessarily complicated futures contracts by restricting symbols to 2 characters. We hope they see the light and allow variable at least 5 characters.
- Zipline doesn't define all futures markets and doesn't provide any extensibility in this area - you will need to add them to site-packages\zipline\finance\constants.py if they are not defined. Be sure to backup this file as it will be overwritten any time you update zipline.
- Zipline assumes that there are bars of every day of trading. If a security doesn't trade for a given day (e.g. it was halted/suspended, or simply nobody wanted to trade it), it will be padded with the previous close repeated in the OHLC fields, with volume set to zero. Consider how this might affect your trading calculations.
Assumptions
- Stocks are automatically set an auto_close_date of the last quoted date
- Futures are automatically set an auto_close_date to the earlier of following: Last trading date (for cash settled futures, and physically delivered futures that only allow delivery after the last trading date), or 1 trading day prior to first notice date for futures that have a first notice date prior to the last trading date.
Ingest data
To ingest data from one of the bundles you've defined in extension.py, you simply run
zipline ingest -b <bundlename>
For example:
zipline ingest -b norgatedata-sp500-backtest
zipline ingest -b norgatedata-russell3000-backtest
zipline ingest -b norgatedata-cme-futures
Incorporating other Norgate Data capbilities into Zipline backtesting
For backtesting against the bundle that was ingested, and using the additional timeseries dataframes and metadata, firstly add a dependency to the top of your code:
import norgatedata
Defining the Universe
For backtesting you need to define your universe. For statstically valid results, this must be survivorship bias-free.
In your run_algorithm's initialize function:
watchlistname = 'Russell 3000 Current & Past'
context.universe = norgatedata.watchlist_symbols(watchlistname)
If you use multiple watchlists, this code snippet may be helpful
symbols = []
watchlists = ['Dow Jones Industrial Average Current & Past', 'Nasdaq 100 Current & Past']
for watchlistname in watchlists:
symbols.extend(norgatedata.watchlist_symbols(watchlistname))
symbols = list(set(symbols)) # Remove dupes
context.universe = symbols
Note: Any watchlists used must also be part of the ingested bundle (e.g. you can't test on the S&P 500 Current & Past stocks if your bundle only ingested the current S&P 500 stocks).
Incorporating timeseries data
To incorporate timeseries data from Norgate
Historical index constituent data
Dividend yield
Support
Thanks
Thanks to Andreas Clenow for his pioneering work in documenting Zipline bundles in his latest book Trading Evolved: Anyone can Build Killer Trading Strategies in Python. We used many of the techniques described in the book to build our bundle code.
Other Zipline observations
Zipline appears to be unable to be installed by Conda v4.7.x. Use the following to downgrade zipline in your Anaconda base setup:
conda config --set allow_conda_downgrades true
conda install conda=4.6.11
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for zipline_norgatedata-0.9.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 45c069352da338acf11d6868c0e1a345eea0b217f4b255d2e493c1d262a25dab |
|
MD5 | 53c733aacba90a1436be1cbf53112e79 |
|
BLAKE2b-256 | 86ea8fb41e2911b229fa4ad790e95932f435830b550e57ebd599bfd125d49792 |