kioku-rl 0.1.1

Cognitive science-inspired reinforcement learning research library

Kioku

A cognitive science-inspired reinforcement learning research library. Kioku is designed to be modular and give you tools to experiment with classical and modern RL algorithms and build your own.

Features

Kioku currently includes implementations of DQN, A2C and PPO, along with reusable components for memory buffers, schedulers, models and environments. It supports the following agents based on classical and SOTA RL methods:

• DQN
  • Based on: Playing Atari with Deep Reinforcement Learning
  • Uses Q-value function with Polyak averaging and epsilon decay
• DQN with PER
  • Adds a priotized experience replay buffer based on: Prioritized Experience Replay
• A2C
  • Based on: Asynchronous Methods for Deep Reinforcement Learning
  • Uses N-step returns
• PPO
  • Based on: Proximal Policy Optimization Algorithms
  • Uses Generalized Advantage Estimation, N-step returns, mini-batch learning, multiple learning iterations per batch.

Installation

Install via PyPI:

pip install kioku-rl

Usage

Run an example:

python examples/ppo_cartpole.py

or use it in your own programs as a library:

from kioku.schedulers import StaticScheduler
from kioku.environments import GymEnvironment
from kioku.agents import PPOAgent
from kioku.trainers import Trainer
from kioku.utils import set_seed

set_seed(42)

env = GymEnvironment("CartPole-v1")
render_env = GymEnvironment("CartPole-v1", render_mode="human")

agent = PPOAgent(
    observation_size=env.observation_size,
    num_actions=env.action_size,
    learning_rate=StaticScheduler(1e-3)
)

trainer = Trainer(agent, env)
trainer.train(100000)

tester = Trainer(agent, render_env)
results = tester.test(10)

Results

DQN Cartpole:

PPO Cartpole:

PPO Lunar Lander: