Archid

A high-level machine learning framework built on Euclid ML.

Archid provides the algorithms, training loops, environments, and experience systems used to train machine learning models and reinforcement learning agents.

Euclid handles the numerical side of machine learning — tensors, autodiff, neural networks, losses, and optimizers.

Archid handles how those components are used to learn.

Why Archid?

Archid is designed to sit between low-level ML operations and complete, black-box training systems.

• Modular algorithms — swap learning algorithms without rewriting your environment or training code.
• Tabular and deep learning — experiment with simple Q-tables or Euclid neural networks.
• Reusable abstractions — algorithms, environments, and experience can be combined independently.
• Experiment-friendly — designed for implementing and testing new learning methods.

Features

Reinforcement Learning

Archid currently includes:

• Q-Learning
• SARSA
• Expected SARSA
• Double Q-Learning
• Monte Carlo Control
• TD Learning
• DQN
• Q-Actor-Critic
• PPO

RL uses a small set of core abstractions:

algorithm.predict(state)
algorithm.update(experience)

environment.reset()
environment.step(action)

trainer.train(...)
trainer.run(...)

Both tabular and neural-network-based algorithms can use the same training infrastructure.

Supervised Learning

Archid also provides a lightweight supervised-learning trainer for Euclid networks.

trainer = SupervisedLearning(
    loss,
    optimizer,
    network,
)

trainer.train(loader, steps)

The trainer handles the basic training loop:

forward → loss → backward → optimizer.step()

Core API

ReinforcementLearning

ReinforcementLearning(algorithm, environment)

history = trainer.train(
    episodes=500,
    max_steps=100,
    epsilon=0.1,
    seed=0,
)

experience = trainer.run()

Algorithm

Algorithms expose a common interface:

algorithm.predict(state)
algorithm.update(experience)

Environment

Environments implement:

environment.reset()
environment.step(action)
environment.render()
environment.close()

Experience

Experiences contain transitions collected during interaction:

Transition(
    state,
    action,
    reward,
    next_state,
    done,
)

experience = Experience()
experience.add(transition)

Examples

The repository includes examples and experiments covering both reinforcement and supervised learning.

Example	Description
train_cliff_walker.py	Q-Learning on Cliff Walking
benchmark_cliff_walking_rl.py	Compares reinforcement-learning algorithms
ppo_biped.py	PPO on Gymnasium's BipedalWalker-v3
supervised.py	Supervised learning with a Euclid CNN
double_descent.py	Double-descent experiment

Quickstart

Q-Learning

from archid import QLearning, ReinforcementLearning
from examples.cliff_walking_env import CliffWalking

env = CliffWalking()

agent = QLearning(
    actions=lambda state: [0, 1, 2, 3],
    learning_rate=0.1,
    gamma=0.99,
)

trainer = ReinforcementLearning(agent, env)

history = trainer.train(
    episodes=500,
    max_steps=100,
    epsilon=0.1,
    seed=0,
)

experience = trainer.run()

PPO

Deep reinforcement-learning algorithms use Euclid networks:

import euclid

from archid import PPO

actor = euclid.networks.Sequential([
    euclid.layers.Dense(24, 64),
    euclid.layers.Tanh(),
    euclid.layers.Dense(64, 4),
])

critic = euclid.networks.Sequential([
    euclid.layers.Dense(24, 64),
    euclid.layers.Tanh(),
    euclid.layers.Dense(64, 1),
])

See examples/ppo_biped.py for a complete continuous-control example using BipedalWalker-v3.

Architecture

archid/
├── reinforcement_learning/
│   ├── reinforcement_learning.py
│   ├── algorithms/
│   │   ├── algorithm.py
│   │   ├── action_value_based/
│   │   ├── state_value_based/
│   │   └── actor_critic/
│   ├── environment/
│   └── experience/
│
├── supervised_learning/
│   └── supervised_learning.py
│
└── __init__.py

The important separation is:

Archid → learning algorithms and training
Euclid → numerical computation and neural networks

Tabular algorithms do not require Euclid. Deep-learning algorithms use Euclid where neural networks are required.

Installation

Requirements

• Python 3.13+
• NumPy
• Euclid ML

Install from source:

git clone https://github.com/Dra3d30n/Archid.git
cd Archid

pip install numpy "euclid-ml>=0.9.9"
pip install -e .

Archid is currently under active development, so the API may change before the first stable release.

Testing

Run the test suite with:

python tests/test_reinforcement_learning.py
python tests/test_supervised_learning.py
python tests/test_rl_training_loop.py

The tests cover algorithm updates, supervised training, and the reinforcement-learning training loop.

Roadmap

• Tabular RL algorithms
• Cliff Walking benchmarks
• PPO
• Q-Actor-Critic
• Supervised-learning trainer
• Improve PPO and deep-RL implementations
• DQN replay buffer and target networks
• A2C / A3C
• DDPG / TD3 / SAC
• Checkpointing and restoration
• Parallel environments
• Documentation site
• PyPI release

Contributing

Contributions are welcome.

For new algorithms, keep the core interface simple:

predict(state)
update(experience)

Include tests or an example demonstrating the algorithm.

License

MIT © 2026 Dra3d30n

See LICENSE.

Links

• GitHub: https://github.com/Dra3d30n/Archid
• Euclid ML: https://dra3d30n.github.io/Euclid/