TradeMaster-try 0.0.6

TradeMaster - A platform for algorithmic trading

TradeMaster: An RL Platform for Trading

---

TradeMaster is a first-of-its kind, best-in-class open-source platform for quantitative trading (QT) empowered by reinforcement learning (RL).

It covers the full pipeline for the design, implementation, evaluation and deployment of RL-based trading methods. It contains: 1) a toolkit for efficient data collection, preprocessing and analysis; 2) a high-fidelity data-driven market simulator for mainstream QT tasks (e.g., portfolio management and algorithmic trading); 3) standard implementation of over 10 novel FinRL methods; 4) a systematic evaluation benchmark called PRUDEX-Compass.

Outline

• TradeMaster: An RL Platform for Trading
  • Outline
  • Overview
  • Installation
  • Tutorial
  • Toolkit
  • Results and Visualization
  • Model Zoo
  • Dataset
  • External Data Source
  • How to Use Your Own Data
  • File Structure
  • Publications
  • Contact
  • Join Us
  • Competition

Overview

TradeMaster could be beneficial to a wide range of communities including leading trading firms, startups, financial service providers and personal investors. We hope TradeMaster can make a change for the whole pipeline of FinRL to prevent untrustworthy results and lead successful industry deployment.

Installation

We provide a video tutorial of using docker to build a proper environment of running this project.

To help you better understand the step discribed in the video, Here are the installation tutorials for different operating systems:

• Docker (Recommended)
• MacOS
• Linux
• Windows

Tutorial

We provide tutorials for users to get start with.

Algorithm	Dataset	Code link	Description
Classic RL	FX	tutorial	Classic RL Algorithms for Portfolio Management on FX
DeepScalper	Bitcoin	tutorial	DeepScalper for Algorithm Trading on Crypto
EIIE	DJ30	tutorial	EIIE for Portfolio Management on DJ30
IMIT	DJ30	tutorial	Investor Imitator for Portfolio Management on DJ30
SARL	DJ30	tutorial	SARL for Portfolio Management on DJ30

• Colab Version: Use Google Colab resource to run TradeMaster on Cloud

Toolkit

• CSDI for financial data imputation (link)
• Automatic market style recognition (link)

Results and Visualization

The evaluation module of TradeMaster is mainly based on PRUDEX-Compass, a systematic evaluation toolkit of FinRL methods with 6 axes and 17 measures. We show some results here:

PRUDEX-Compass provides an intuitive visual means to give readers a sense of comparability and positioning of FinRL methods. The inner level maps out the relative strength of FinRL methods in terms of each axis, whereas the outer level provides a compact way to visually assess which set-up and evaluation measures are practically reported to point out how comprehensive the evaluation are for FinRL algorithms.

PRIDE-Star is a star plot to evaluate profitability,risk-control and diversity. It contains the normalized score of 8 measures.

(a) A2C	(b) PPO	(c) SAC

Rank distribution plot is a bar plot, where the i-th column in the rank distribution shows the probability that a given method is assigned rank i in the corresponding metrics.

(a) All 4 datasets	(b) DJ30	(c) FX

Performance profile reports FinRL methods' score distribution of all runs across the different financial markets that are statistically unbiased and more robust to outliers.

(a) All 4 datasets	(b) DJ30	(c) FX

For more information of the usage of this part, please refer to this tutorial and this project

Model Zoo

DeepScalper based on Pytorch (Shuo Sun et al, CIKM 22)

OPD based on Pytorch (Fang et al, AAAI 21)

DeepTrader based on Pytorch (Wang et al, AAAI 21)

SARL based on Pytorch (Yunan Ye et al, AAAI 20)

ETTO based on Pytorch (Lin et al, 20)

Investor-Imitator based on Pytorch (Yi Ding et al, KDD 18)

EIIE based on Pytorch (Jiang et al, 17)

Classic RL based on Pytorch and Ray: PPO A2C SAC DDPG DQN PG TD3

Dataset

Dataset	Data Source	Type	Range and Frequency	Raw Data	Datasheet
SP500	YahooFinance	US Stock	2000/01/01-2022/01/01, 1day	OHLCV	SP500
DJ30	YahooFinance	US Stock	2012/01/01-2021/12/31, 1day	OHLCV	DJ30
FX	Kaggle	FX	2000/01/01-2019/12/31, 1day	OHLCV	FX
Crypto	Kaggle	Crypto	2013/04/29-2021/07/06, 1day	OHLCV	Crypto
SZ50	YahooFinance	CN Securities	2009/01/02-2021-01-01, 1day	OHLCV	SZ50
Bitcoin	Kaggle	Crypto	2021-04-07 11:33-2021-04-19 09:54 , 1min	LOB	Bitcoin

OHLCV: open, high, low, and close prices; volume: corresponding trading volume

External Data Source

Users may download data from the following data source with personal account:

Data Source	Type	Range and Frequency	Request Limits	Raw Data
Alpaca	US Stocks, ETFs	2015-now, 1min	Account-specific	OHLCV
Baostock	CN Securities	1990-12-19-now, 5min	Account-specific	OHLCV
Binance	Cryptocurrency	API-specific, 1s, 1min	API-specific	Tick-level daily data
CCXT	Cryptocurrency	API-specific, 1min	API-specific	OHLCV
IEXCloud	NMS US securities	1970-now, 1 day	100 per second per IP	OHLCV
JoinQuant	CN Securities	2005-now, 1min	3 requests each time	OHLCV
QuantConnect	US Securities	1998-now, 1s	NA	OHLCV
RiceQuant	CN Securities	2005-now, 1ms	Account-specific	OHLCV
Tushare	CN Securities, A share	-now, 1 min	Account-specific	OHLCV
WRDS	US Securities	2003-now, 1ms	5 requests each time	Intraday Trades
YahooFinance	US Securities	Frequency-specific, 1min	2,000/hour	OHLCV

How to Use Your Own Data

TradeMaster supports financial data with open, high, low, close, volume (OHLCV) raw informations as:

We compute 10 technical indicators to describe the financial markets:

Users can adapt their data with prefered features by changing the data loading and feature calculation part with corresponding input and output size. We plan to support limit order book (LOB) and altervative data such as text and graph in the future.

File Structure

|-- agent
|   |-- ClassicRL
|   |-- DeepScalper
|   |-- DeepTrader
|   |-- EIIE
|   |-- Investor_Imitator
|   |-- SARL
|-- config
|   |-- input_config
|   |-- output_config
|-- data
|   |-- download_data.py
|   |-- preprocess.py
|   |-- data
|       |-- BTC
|       |-- dj30
|       |-- exchange
|       |-- sz50
|-- env
|   |-- AT
|   |-- OE
|   |-- PM
|-- experiment
|-- figure
|-- result
|-- tutorial
|   |-- ClassRL_for_PM_on_FX.ipynb
|   |-- DeepScalper_for_AT_on_Bitcoin.ipynb
|   |-- EIIE_for_PM_on_DJ30.ipynb
|   |-- IMIT_for_PM_on_DJ30.ipynb
|   |-- SARL_for_PM_on_DJ30.ipynb
|   |-- Visualization.ipynb
|-- visualization
|   |-- compass
|   |-- exen
|   |-- ocatgon
|   |-- performance_profile
|   |-- rank
|-- README.md
|-- requirement.txt

Publications

Deep Reinforcement Learning for Quantitative Trading: Challenges and Opportunities (IEEE Intelligent Systems 2022)

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities (CIKM 2022)

Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 21)

Reinforcement Learning for Quantitative Trading (Survey)

PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets

Contact

• This repository is developed and maintained by AMI group at Nanyang Technological University
• If you want to make contributions to TradeMaster, please create pull requests.

Join Us

We have positions for software engineer, RA and postdoc. If you are interested in working at the intersection of RL and financial trading, feel free to send an email to [e-mail] with your CV.

Competition

TradeMaster Cup 2022