reinfors 0.4.0

High-throughput reinforcement-learning search and sampling in Rust, with caller-owned Python networks and training.

Reinfors runs game dynamics, search, episode orchestration, and batch assembly in a parallel Rust backend. Your inference callback is the boundary: it receives pooled NumPy observations and returns model outputs, so the network, framework, optimizer, replay, hardware placement, and distributed topology remain yours.

Game semantics cover single- and multi-agent, zero-sum, cooperative, and general-sum tasks, with turn-taking or simultaneous decisions, explicit chance, and perfect or imperfect information. Algorithms span policy-driven value learning, search-guided learning, and standalone game-theoretic solving.

import numpy as np
import reinfors as rf

game = rf.games.Connect4()
engine = rf.Engine(
    game=game,
    reward=rf.Reward(win=1.0, loss=-1.0),
    policy=rf.policies.Minimax(depth=4),
    learner=rf.learners.TreeStrap(),
    n_games=32,  # parallel episode slots
    seed=0,
)

actions = game.action_space().n

def infer(obs: np.ndarray) -> np.ndarray:
    # The network is the search's evaluation function: it scores every frontier leaf,
    # pooled across games. Replace with PyTorch, JAX, an accelerator service, or any
    # other backend; TreeStrap's targets train it toward its own deeper search results.
    return np.zeros((len(obs), 1, actions), dtype=np.float32)

batch = engine.collect(n_records=1024, infer=infer)
print(batch.obs.shape, batch.targets.shape, batch.telemetry)

Performance

Reinfors' native backend makes it much more performant than Python-loop RL libraries such as Gymnasium. Benchmarking reinfors' car_racing port against Gymnasium's CarRacing-v3 — separate implementations of the same design, with matched action space and 96x96 RGB observation content, stepped in single-threaded loops — reinfors runs ~20x more environment steps per second (medians of three 30s trials):

Reproduce with scripts/bench_carracing_throughput.py, which reports medians, ranges, and machine provenance.

Parallelising multiplies this further, and in reinfors it is trivial — set n_threads — while collect_stream overlaps collection with training as the normal operating mode:

Benchmarked against OpenSpiel's all-C++ libtorch AlphaZero on a matched chess training workload — two-hour rounds at each stack's best measured configuration, then head-to-head play between the resulting models — reinfors sustained 9.9% higher training throughput, and its trained networks scored 0.605 ± 0.020 against OpenSpiel's over 300 paired games (+74 Elo). Protocol, evidence, and full results: reinfors-benchmarks.

Reinfors is not designed to maximize throughput at any cost or to outperform a bespoke, fully fused JAX/XLA pipeline on the fixed workload it specializes for. It targets a practical balance: native simulation, search, and batching, while games, algorithms, networks, and deployment remain modular enough for broad experimentation.

Install

pip install reinfors

Optional adapters and training dependencies are separate:

pip install "reinfors[gym]"   # Gymnasium and PettingZoo adapters
pip install "reinfors[train]" # PyTorch examples

Contributing or building from source? Start with the contributing guide and the development setup guide.

Where next?

• Get started
• Understand sampling and injectable training
• Choose a game, algorithm, or built-in composition
• Run the examples
• Evaluate searched agents with Arena
• Add a Rust game or algorithm
• Read the complete documentation

Stability

reinfors is pre-1.0: any 0.x release may change any API, behavior, or serialized format (including snapshot and config layouts) without deprecation. Pin an exact version (reinfors==0.x.y) and read release notes when upgrading. What does hold at every version: constructors validate their inputs, and no public Python input reaches a Rust panic — both enforced by adversarial test sweeps in CI.

Citation

If you use reinfors in your research, cite it via the repository's CITATION.cff (GitHub's "Cite this repository" button renders it as BibTeX/APA). What reinfors itself builds on is catalogued in References.

License

Licensed under either of MIT or Apache-2.0, at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work shall be dual-licensed as above, without any additional terms or conditions.