A downloader and parser for IEX DEEP historical data using C++ backend.
Project description
IEX Data Download and Parsing
The IEX exchange offers two types of historical data: DEEP and TOPS, both provided as PCAP files capturing network activity. This package provides a convenient API for extracting data from DEEP files. These files contain various types of messages, such as trade reports, bid submissions, security messages etc. Currently, our package supports parsing two specific types of messages:
- Trade reports - Detailed records of executed trades.
- Price level updates - Information about changes in the bid and ask price levels.
Key benefits:
- High-performance parsing: Significantly faster than the available Python parsers
- Multi-timestamp support: Provides three timestamps for each event:
- Event timestamp (from the matching engine)
- Packet send timestamp
- Packet capture timestamp
These timestamps provide insights into the time delay between event occurrence time and the data reception time, enabling more accurate backtesting. For a more detailed explanation of Market data see market data format.
- Easy-to-use API: Provides a simple interface for extracting data from DEEP files.
- OneTick compatibility: The output format is compatible with OneTick, a popular time-series database for financial data.
Requirements
This requires Linux OS or Windows WSL terminal to run. It also requires additional softwares that are listed at requirements.
How to use
- Install this package using pip.
pip install iex-cppparser
- Compile the cpp files by running the following python script
from iex_cppparser import compile_cpp
compile_cpp.compile()
- Create a
symbols.txt
file to filter the desired symbols
AAPL
MSFT
- Run the following python script
from iex_cppparser import parse_date
download_folder = ...
parsed_folder = ...
parse_date("2023-10-30",download_folder,parsed_folder,"symbols.txt")
For an extensive list of parsing and downloading functions see documentation.
Output
The output CSV file contains parsed trade reports and price level updates.
Acknowledgement
This project extends the work of the authors done as part of IE 421 course at University of Illinois Urbana-Champaign. The previous project repository can be found at this link (a private UIUC repository at the time of this writing).
Citing this Work
To cite this work in academic papers or publications, you can use the following BibTeX format:
@misc{bavisettyvasu,
author = {Bavisetty, Venkata Sai Narayana and Vasu, Karthik},
title = {IEX C++ Parser},
year = {2024},
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{[Repository URL](https://github.com/Sai271828/iex-parser)}}
}
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file iex_cppparser-1.0.3.tar.gz
.
File metadata
- Download URL: iex_cppparser-1.0.3.tar.gz
- Upload date:
- Size: 3.9 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.9.18
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87e9baa7663a291159c1f1daa57232833fd1e62071b3e3cdd91c43744841eacd |
|
MD5 | 5b183ca278f14710345fd51eae819825 |
|
BLAKE2b-256 | 11901719a51572ee7547f2fc82153851aedfcedb69d70136058e6a563e87e51a |
File details
Details for the file iex_cppparser-1.0.3-py3-none-any.whl
.
File metadata
- Download URL: iex_cppparser-1.0.3-py3-none-any.whl
- Upload date:
- Size: 279.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.0 CPython/3.9.18
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 77eeb9ff50fd14edbf28ef17fa9950ea8696c11ff7e373f6ee7cb0f64288c3bb |
|
MD5 | 7cd793b68603933e9286d5ff8a34c940 |
|
BLAKE2b-256 | a088239318d544028af21072adafd65d1bd6b2d144f6625075b0f50810136b75 |