kaggriculture-engine

A Rust port of the kaggle-environments Kaggriculture interpreter (kaggriculture.py) that gives bit-identical results. It is about 10,000× faster per episode than the Python original running under kaggle's Environment.

Given the same configuration and the same per-step agent actions, the port produces byte-identical state to the Python original. That includes float formatting, dict key order, RNG draws, rewards and statuses, and which Python exception (if any) a step raises.

Python (PyO3)

uv sync                                   # builds the extension (maturin) into .venv
.venv/bin/maturin develop --release --uv  # rebuild after changing Rust code

import kaggriculture_engine as ke

game = ke.Game({"seed": 42})                 # == make("kaggriculture", configuration=...)
game.step([{"farmer": ["WATER"]}, {"market": [["BUY_SEED", "CARROT", 1]]}])   # == env.step
obs = game.observation(0, structified=True)  # what kaggle hands agent 0 (obs.player, obs["farms"], ...)
game.statuses, game.rewards, game.current_step, game.render(), game.record()

# Whole games: named Rust agents run with the GIL released; any Python
# callable agent(obs[, config]) works too, e.g. the original kaggriculture agents.
finished = ke.run_episode(["starter", my_python_agent], {"seed": 42})
results = ke.run_batch(range(10_000), ("starter", "random:1"))   # parallel, GIL released

step mirrors env.step exactly:

• An action can be any Python value, and each entry can instead be an exception instance, which marks that agent ERROR (a DeadlineExceeded marks it TIMEOUT).
• When the original interpreter would raise, step raises the same exception class (TypeError, ValueError, OverflowError or ZeroDivisionError) and leaves the game unchanged.
• Stepping a finished game raises ke.FailedPrecondition.
• Tuples, numpy scalars, Decimals, sets, bytes and dicts with non-str keys behave as they do in kaggle.

The extension doesn't need kaggle-environments at runtime.

Build and run the CLI

cargo build --release
B=target/release/kaggriculture_engine

# Agent-driven episode (pass | starter | random[:SEED]); one JSON record per state
echo '{"seed": 42}' | $B play - starter random:7 [--render]

# Replay recorded inputs: {"configuration": {...}, "steps": [[in0, in1], ...]}
# where in = {"action": <any JSON>} | {"status": "ERROR"} | {"status": "TIMEOUT"}
$B replay scenario.json [--render]

# Throughput (single thread, then all cores via Rayon)
$B bench --episodes 8000 --agents starter,random:1 [--threads N]

Library use:

use kaggriculture_engine::{Action, AgentInput, Config, Game};
use std::sync::Arc;

let cfg = Arc::new(Config::from_json(&serde_json::json!({"seed": 42}))?);
let mut game = Game::new(cfg.clone());
while !game.done() {
    let a = Action::from_json(&serde_json::json!({"farmer": ["WATER"]}), cfg.max_orders);
    game.step(&[AgentInput::Act(a.clone()), AgentInput::Act(a)])?; // Err = Python raised; state untouched
}

batch::run_batch plays many seeds in parallel. To time a single game end to end (cold and warm, with and without serialising every state), run cargo run --release --example one_game.

Verifying bit-exactness

The Python side needs the project venv (uv sync).

.venv/bin/python tests/parity/run_parity.py      # full sweep (~146 cases, ~30k records)
cargo test --release                             # unit tests + a 14-case parity slice

tests/python/test_bindings.py (.venv/bin/python -m unittest discover -s tests/python) checks the Python bindings against the same reference. It replays fuzzed scenarios through Game.step, drives the original Python agents with the bindings' observations, and checks that Python-only action values and agent exceptions behave identically.

tests/parity/reference.py runs the original kaggriculture.py through the real kaggle Environment (make → env.step). tests/parity/fuzz.py plays both seats with a seeded, state-aware policy, so games reach crops, animals, hands, land, shops, weeds and decay. It also injects malformed input: unhashable items, int()-hostile counts ("abc", "1_0", "٣", 1e400), non-dict actions, agent ERROR/TIMEOUT statuses, and hostile marketParams and configs. The Rust binary replays the exact same JSON text, and every output line must be byte-identical. Mismatching inputs are saved to tests/parity/failures/.

As a check on the harness itself, I planted small bugs (care-bonus arithmetic, round-half-up instead of Python's round-half-even) and the sweep flagged each of them.

Performance (Apple M1, 8 cores; 720-step episodes)

From Python, with starter vs starter:

• ke.run_episode with named agents takes 0.09 ms per game.
• The original Python agents, called through ke.run_episode, take 29 ms per game. That's still 29× faster than kaggle's env.run, and building the observation dicts is now the bottleneck.
• A plain Python loop costs 0.65 µs per Game.step.

The Python figures include kaggle's framework overhead (it deep-copies the state every step), which is how the game is normally run.

Where the time went and what was done about it:

• Rayon runs episodes in parallel (batch.rs). A single step (~150 ns) is far too small to split across threads.
• SIMD (NEON): CPython's random.Random(seed) (MT19937 init_by_array) is a long serial multiply chain. The interpreter builds a new generator every in-game day, and random_agent builds one every turn. All of these seeds are known ahead of time, so pyrandom::LaneBatch seeds 32 of them at once across NEON lanes. That is 28× faster per generator than scalar. It also twists and tempers lazily: the first 227 outputs depend only on the untwisted state. The single-state twist and tempering also use NEON. Other architectures fall back to auto-vectorised lane arrays.
• Price curves are memoised per inventory (sqrt/log calls were half of the step time), the decay scan is skipped until a plant can decay, the shed total is kept incrementally, and inventories are allocation-free ordered maps.
• Steps that cannot raise run in place. A step that might raise (malformed actions, marketParams overrides, …) runs on a copy, matching kaggle's behaviour of discarding the state when the interpreter raises.

Layout

Known divergences (all outside realistic inputs)

• marketParams integers beyond ±2⁶², boardSize > 1024 and turnsPerDay > 2⁵⁸ are rejected at config time. Python would accept them.
• Inputs must be standard JSON. Python's json additionally accepts NaN/Infinity literals and lone surrogates.
• Exceptions are reported by class (TypeError, …), not by message text.
• random_agent takes an explicit seed; the Python original seeds from OS entropy on every call. The parity harness patches Python to use the same seeds. With no configured seed, both sides pick a random episode seed, so those runs aren't comparable.
• Interpreter stdout (warnings) is returned in full; kaggle truncates logs at maxLogLength (10,000 chars per step).
• The renderer prints non-ASCII characters verbatim; CPython escapes the few non-printable ones. This only matters for strings inside marketParams.
• html_renderer (which only serves a bundled HTML file) is not ported.
• Python bindings, callable agents: there is no act-timeout or overage-time accounting, and observations omit kaggle's framework-only remainingOverageTime. Observations use the package's own Struct rather than kaggle's class (same behaviour, different isinstance). Nested dicts in the configuration passed to agents are left as plain dicts.

License

Licensed under the Apache License, Version 2.0.

This project is a port of kaggriculture.py from kaggle-environments (Copyright 2020 Kaggle Inc, Apache-2.0). See NOTICE for attribution, including the CPython algorithms reimplemented for bit-exact compatibility.