gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methods

A Gymnasium-compatible multi-echelon inventory management environment for reinforcement learning and operations research.

This repository contains the standalone environment package only. The paper benchmark agents, trained weights, result tables, and manuscript source live in the companion benchmark repository.

Project Links

• PyPI package: gym-invmgmt
• Standalone environment package: r2barati/gym-invmgmt
• Paper/code repository: r2barati/gym-invmgmt-paper
• Trained checkpoint archive: rezabarati/gym-invmgmt-weights

Tested with:

The environment simulates a configurable supply chain network with realistic logistics — production capacities, pipeline lead times, holding costs, backlog penalties — driven by a composable demand engine supporting non-stationary patterns and endogenous customer goodwill dynamics.

Installation

pip install gym-invmgmt

To upgrade an existing installation:

pip install -U gym-invmgmt

To install the latest development version directly from GitHub:

pip install git+https://github.com/r2barati/gym-invmgmt.git

For development (editable install):

git clone https://github.com/r2barati/gym-invmgmt.git
cd gym-invmgmt
pip install -e .

Release instructions for maintainers are in docs/releasing_to_pypi.md.

Repository Structure

gym-invmgmt/
├── gym_invmgmt/               ← Source code (all Python modules)
│   ├── core_env.py            ←   Gymnasium environment (step, reset, reward)
│   ├── demand_engine.py       ←   Non-stationary demand generation
│   ├── network_topology.py    ←   Graph builder (presets + YAML parser)
│   ├── visualization.py       ←   Network plotting
│   ├── utils.py               ←   Shared helpers (run_episode, compute_kpis)
│   ├── topologies/             ←   YAML network topology definitions
│   │   ├── assembly.yaml
│   │   ├── diamond.yaml
│   │   ├── distribution_tree.yaml
│   │   ├── divergent.yaml
│   │   ├── serial.yaml
│   │   └── w_network.yaml
│   └── wrappers/               ←   Action rounding, episode logging
├── examples/                  ← Runnable example scripts
│   ├── quickstart.py          ←   Minimal env usage
│   ├── run_heuristic.py       ←   Newsvendor base-stock policy
│   ├── run_or_baselines.py    ←   Classical OR policies comparison
│   ├── train_ppo.py           ←   PPO training with SB3
│   ├── benchmark_agents.py    ←   Multi-agent benchmark across scenarios
│   └── generate_videos.py     ←   Dashboard video generator
├── tests/                     ← Test suite
├── docs/                      ← Documentation
│   ├── guides/                ←   Tutorials and walkthroughs
│   └── reference/             ←   Technical reference docs
└── assets/                    ← Images for documentation

Quick Start

import gymnasium as gym
import gym_invmgmt

env = gym.make("GymInvMgmt/MultiEchelon-v0")
obs, info = env.reset(seed=42)

done = False
while not done:
    action = env.action_space.sample()
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

Environments

Both environments default to 30-period episodes with stationary Poisson demand (μ=20).

Environment Details

Network Topology

The multi-echelon network features factories with production capacities, distributors as intermediary holding points, and retailers facing stochastic customer demand:

The serial chain provides a simpler linear topology for focused experiments:

Observation Space

Box(-inf, inf, shape=(obs_dim,)) — a flat vector containing:

• Demand: Current realized demand at each retail link
• Inventory: On-hand inventory at each managed node (distributors + factories)
• Pipeline: In-transit quantities for each supply link, broken out by lead-time position
• Extra Features: Current time period t and demand sentiment s (goodwill multiplier)

Action Space

Box(0, max_order, shape=(n_reorder_links,)) — continuous order quantities for each supply link.

Note for RL practitioners: The upper bound is a conservative theoretical maximum (initial inventory + capacity × horizon). In practice, meaningful orders are much smaller. If using PPO/SAC out-of-the-box, consider wrapping with gymnasium.wrappers.RescaleAction(env, -1, 1) to normalize the action range, or use IntegerActionWrapper for discrete order quantities.

Reward Function

Each step returns the network-wide profit:

Reward = Σ (Revenue − Purchasing Cost − Holding Cost − Operating Cost − Backlog Penalty)

• Revenue: Selling price × units sold at retail
• Purchasing Cost: Unit cost × units ordered from upstream
• Holding Cost: Per-unit cost for on-hand inventory and in-transit pipeline
• Operating Cost: Factory variable cost per unit produced
• Backlog Penalty: Per-unit penalty for unmet demand

Episode Termination

Episodes truncate after num_periods steps (default 30). There is no early termination.

Configuration

All parameters are configurable via gymnasium.make() kwargs or direct CoreEnv() instantiation:

Network Topologies

• scenario='network' — Multi-echelon divergent network (3 factories, 2 distributors, 1 retailer)
• scenario='serial' — Serial supply chain (1 factory → 1 distributor → 1 retailer)
• scenario='custom' — User-defined topology loaded from a YAML config file (see below)

Demand Scenarios

The DemandEngine supports composable non-stationary effects:

Example — shock demand with goodwill:

env = gym.make("GymInvMgmt/MultiEchelon-v0",
    demand_config={
        'type': 'shock',
        'base_mu': 25,
        'use_goodwill': True,
        'shock_time': 15,
        'shock_mag': 2.0,
    },
    num_periods=50,
)

Example — real-world demand data (e.g., M5 competition):

import numpy as np
real_demand = np.loadtxt("my_sales_data.csv")  # one value per period

env = gym.make("GymInvMgmt/MultiEchelon-v0",
    demand_config={
        'type': 'stationary',
        'base_mu': float(np.mean(real_demand)),
        'external_series': real_demand,
    },
    num_periods=len(real_demand),
)

Note: When overriding demand_config via gym.make(), provide the full dictionary — partial dictionaries replace the registered defaults entirely rather than merging recursively.

Fulfillment Modes

• backlog=True (default) — Unmet demand accumulates as backlog, penalized each period
• backlog=False — Unmet demand is lost immediately (lost sales model)

Custom Network Topologies

Beyond the two built-in presets, you can define any topology via a YAML config file:

from gym_invmgmt import make_custom_env

# Load a custom topology — diamond network with parallel factories
env = make_custom_env('gym_invmgmt/topologies/diamond.yaml', num_periods=30)
obs, info = env.reset(seed=42)

The built-in presets (_build_network_scenario(), _build_serial_scenario()) define topologies in Python. The YAML parser (_build_custom_scenario()) reads the same node/edge structure from a config file, auto-detects node roles, and validates the graph.

See gym_invmgmt/topologies/ for ready-to-use YAML files, and the full YAML schema reference for all supported parameters.

Visualization

from gym_invmgmt import CoreEnv

env = CoreEnv(scenario='network')
env.plot_network()              # Simple topology view
env.plot_network(detailed=True) # With costs, lead times, capacities

Wrappers

The package includes two agent-agnostic wrappers:

IntegerActionWrapper

Rounds continuous actions to integers for physical realism (you can't order 3.7 units):

from gym_invmgmt import IntegerActionWrapper
env = IntegerActionWrapper(env)

EpisodeLoggerWrapper

Saves full trajectory matrices (inventory, demand, orders, profit, backlog) as .npz files:

from gym_invmgmt import EpisodeLoggerWrapper
env = EpisodeLoggerWrapper(env, log_dir="./logs", run_name="experiment_1")

Action Scaling for Deep RL

When training with PPO, SAC, or other deep RL algorithms, normalize the action space to [-1, 1]:

from gymnasium.wrappers import RescaleAction

env = CoreEnv(scenario='network')
env = RescaleAction(env, min_action=-1.0, max_action=1.0)
# Agent now outputs actions in [-1, 1], automatically mapped to valid order quantities

Architecture

CoreEnv (Gymnasium Environment)
├── Simulation Dynamics         # Order placement, delivery, demand fulfillment
│   ├── Allocation Logic        # Raw-material, distribution, factory constraints
│   ├── Pipeline Advancement    # Lead-time delayed deliveries & inventory update
│   └── Profit Calculation      # Revenue – procurement – holding – ops – penalty
├── State & Spaces              # Observation vector, action/observation spaces
├── SupplyChainNetwork          # Topology: nodes, edges, lead times, capacities
│   ├── _build_network_scenario()   # Built-in: multi-echelon divergent graph
│   ├── _build_serial_scenario()    # Built-in: serial supply chain
│   └── _build_custom_scenario()    # Custom: loads from YAML config file
├── DemandEngine                # Non-stationary demand generation
│   ├── Composable Effects      # Trend / Seasonal / Shock (can combine)
│   ├── External Series         # Real-world data injection (M5, Favorita, etc.)
│   └── Endogenous Goodwill     # Service-dependent demand feedback
└── Wrappers
    ├── IntegerActionWrapper     # Discrete order rounding
    └── EpisodeLoggerWrapper     # Trajectory recording

Documentation

For detailed mathematical formulations and parameter references:

• MDP Formulation — State space, action space, transition dynamics, and reward function with equations
• Demand Engine — Composable non-stationary effects (trend, seasonal, shock) and endogenous goodwill dynamics
• External Datasets — Using real-world demand data (M5, Favorita, Rossmann) with recommended datasets and examples
• Network Topologies — Complete node/edge parameter tables, both built-in presets and YAML custom topologies
• Comparison with Prior Work — How this project relates to OR-Gym and OR-RL-Benchmarks

Tutorial & Examples

Getting Started Tutorial — A comprehensive walkthrough covering:

Visual Dynamics Guide — See exactly how decisions affect every node:

Citing

If you use this environment in your research, please cite:

@misc{gym-invmgmt,
  author = {Barati, Reza},
  title = {gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methods},
  year = {2026},
  howpublished = {\url{https://github.com/r2barati/gym-invmgmt}},
}

This environment builds on foundational work by ORL Benchmarks (Balaji et al., 2019), OR-Gym (Hubbs et al., 2020), and Perez et al. (2021). See CITATION.cff for full references.

License

MIT License. See LICENSE for details.