AssetAllocator 0.0.12

Train RL agents to manage a portfolio

Automating Portfolio Allocation with Reinforcement Learning

Docs | Examples

AssetAllocator

Installation

pip install AssetAllocator

Usage

Available Models and their keys

Model Name	Key
Normalized Advantage Function	NAF
REINFORCE	REINFORCE
Deep Deterministic Policy Gradient	DDPG
Twin Delayed Deep Deterministic Policy Gradient	TD3
Advantage Actor Critic	A2C
Soft Actor Critic	SAC
Trust Region Policy Optimization	TRPO
Proximal Policy Optimization	PPO

Running Experiments

We wrote a generic Trainer and Experiment class that can be used to train any of the agents. All you need to do is your hyperparameter dictionaries and the agent name to run an experiment

import torch
from AssetAllocator.experiment import Experiment

device = 'cuda' if torch.cuda.is_available() else 'cpu'

trainer_kw = {'print_every': 1, 'test_runs': 1}
model_kw = {'device': device}

exp = Experiment(trainer_kwargs=trainer_kw, model_kwargs=model_kw)
exp.run('SAC')

exp = Experiment(trainer_kwargs=trainer_kw, model_kwargs=model_kw, timesteps=[1_000_000])
exp.run('SAC')

Hyperparameter Tuning

The Experiment class has support for overriding agent, trainer, and environment parameters. Check the docs for more details about the agent, trainer, and environment and pass in the appropriate dictionaries to the Experiment class. An example can be seen below

trainer_kw = {
    'experiment_name': 'time_to_get_rich', 
    'print_every': 100, 
    'test_runs': 10, 
    'add_softmax'=True, 
    'start_date'='2009-01-01', 
    'end_date'='2022-01-01', 
    'seed'=667, 
    'test_length'=550,
    'test_runs'=1
    }

model_kw = {
    'device': device,
    'hidden_dim'=256, 
    'gamma'=0.9,
    }

exp = Experiment(trainer_kwargs=trainer_kw, model_kwargs=model_kw)
exp.run('A2C')

More Examples

We have provided several example notebooks to help you get started

Dependencies

• gym
• torchvision
• scipy
• torch
• yfinance
• stable_baselines3
• numpy
• tqdm
• pandas
• matplotlib

Contributions

AssetAllocator is open to contributions

Attribution

Logos 1, 2 obtained from flaticon