akademy 0.1.44

akademy: A Reinforcement Learning Framework

Akademy

Akademy is a module containing composable object classes for developing reinforcement learning algorithms focused on quantitative trading and time-series forecasting. This module is a work-in-progress and should, at no time, be assumed to be designed well or be free of bugs.

Overview

Akademy is designed using an Agent-Environment model such that Agent-class objects ingest information from Environment-class objects (Env), produce an Action, which is then applied to the Environment which results in a change in State and possible reward to offer feedback to the agent.

Note: this module does not provide any training routines -- only the object class that can be used to support the implementation of custom training routines.

Getting Started

To install akademy use the following command in the desired Python 3.7+ environment:

pip install akademy

Once installed, developers will have access to Agent, TradeEnv, and Network class objects in which to design Reinforcement Learning algorithms to train models.

Sample training routine:

from akademy.models.envs import TradeEnv
from akademy.models.agents import DQNAgent
from akademy.common.utils import load_spy_daily

# loads the dataset used during training
data = load_spy_daily(count=2500)

# load the Trading Environment
env = TradeEnv(
    data=data,
    window=50,
    asset="spy",
)

# load the agent to train
agent = DQNAgent(
    action_count=env.action_space.n,
    state_shape=env.observation_space.shape
)

# load user-defined training routine
training_routine(
    agent=agent,
    env=env
)

Tests

Unit testing can be run via the following command:

python -m unittest

For detailed information the --verbose flag can be used. For more detailed usage consult the unittest module documentation.

Available Data

This module comes with minimal data for Agents and Environments to train on. The current data available is listed below, along with sources for the most up-to-date versions as well:

1. S&P500

Location: /data/SPY.CSV
Start: 1993-01-29
End: 2023-01-23
Total Rows: 7,454 (excludes header)
Header: Date,Open,High,Low,Close,Adj Close,Volume
Source: https://finance.yahoo.com/quote/SPY/history?p=SPY

note: Any data can be used easily enough via conversion into a Pandas DataFrame object, but must contain information for date and pricing data for open, high, low, and close as well as volume such that each row has at least those 6 features or the latter 5 and an index representative of date.

Notes

Gym vs. Gymnasium

The Gym project by OpenAI has been sunset and now maintained as Gymnasium by the Farama-Foundation. The Env classes present here make use of the newer Gymnasium package which, among other differences, produces an extra item in the step method indicating whether an environment has been truncated. See here

PyTorch

PyTorch requires some additional consideration for setup depending on use-case. Akademy uses an approach whereby CPU-based training and inferences are possible via parameterized function calls. However, GPU use (e.g. CUDA) requires local considerations. [See here] (https://pytorch.org/get-started/locally/) for a more in-depth discussion and guide.

This module currently uses the 1.* version, though a 2.* version release is imminent and an upgrade to that version is planned.