POMDPPlanners

POMDPPlanners is a set of reliable implementations of POMDP (Partially Observable Markov Decision Process) planning algorithms and environments in Python. It provides standardized simulation studies for research and production-quality planners for industrial applications — from classic benchmarks like Tiger and RockSample to photorealistic autonomous driving and robotic manipulation.

Rendered by the package itself: the CARLA driving environment (left) and the Isaac Sim / IsaacLab Franka reach environment (right). Realistic environments are integrated from the open-source simulators CARLA and NVIDIA Isaac Lab — credit to their authors.

Main Features

Documentation

Documentation is available online: https://yaacovpariente.github.io/POMDPPlanners/

Installation

Note: POMDPPlanners requires Python 3.10+.

# Clone the repository
git clone https://github.com/yaacovpariente/POMDPPlanners.git
cd POMDPPlanners

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install the package
pip install -e .

Example

Plan with POMCP on the classic Tiger problem in just a few lines:

from POMDPPlanners.environments.tiger_pomdp import TigerPOMDP
from POMDPPlanners.planners.mcts_planners.pomcp import POMCP
from POMDPPlanners.utils.belief_factory import create_environment_belief

env = TigerPOMDP(discount_factor=0.95)
planner = POMCP(environment=env, discount_factor=0.95, depth=20,
                exploration_constant=10.0, n_simulations=1000,
                name="POMCP")
belief = create_environment_belief(env, n_particles=200)

actions, _ = planner.action(belief)
print(f"Recommended action: {actions[0]}")

Running Experiments

The recommended entry point for end-to-end experiments is LocalSimulationsAPI, which runs parallel episodes, applies persistent caching, and returns aggregated statistics (mean return, CVaR, VaR, confidence intervals).

from POMDPPlanners.environments import ContinuousLightDarkPOMDPDiscreteActions
from POMDPPlanners.planners.mcts_planners.pomcpow import POMCPOW
from POMDPPlanners.planners.mcts_planners.pft_dpw import PFT_DPW
from POMDPPlanners.utils.action_samplers import DiscreteActionSampler
from POMDPPlanners.utils.belief_factory import create_environment_belief
from POMDPPlanners.simulations.simulation_apis.local_simulations_api import LocalSimulationsAPI
from POMDPPlanners.core.simulation import EnvironmentRunParams

env = ContinuousLightDarkPOMDPDiscreteActions(discount_factor=0.95)
sampler = DiscreteActionSampler(env.get_actions())

pomcpow = POMCPOW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=10.0, k_o=2.0, k_a=2.0,
                  alpha_o=0.5, alpha_a=0.5, n_simulations=500,
                  action_sampler=sampler, name="POMCPOW")
pft_dpw = PFT_DPW(environment=env, discount_factor=0.95, depth=10,
                  exploration_constant=10.0, n_simulations=500,
                  action_sampler=sampler, name="PFT_DPW")
belief = create_environment_belief(env, n_particles=200)

api = LocalSimulationsAPI()
_, stats = api.run_multiple_environments_and_policies(
    environment_run_params=[EnvironmentRunParams(
        environment=env, belief=belief,
        policies=[pomcpow, pft_dpw], num_episodes=100, num_steps=30)],
    alpha=0.1, confidence_interval_level=0.95,
    experiment_name="LightDark_Evaluation",
)

For hyperparameter search, LocalSimulationsAPI.run_optimize_and_evaluate(...) accepts HyperParameterRunParams with Optuna search ranges and forwards the best configuration to evaluation automatically.

Long-running experiments can report progress to Slack and a local progress database, including detection of crashed or stalled runs — set SLACK_WEBHOOK_URL in your environment and notifications are picked up automatically. See NotificationConfig for details.

Tutorial Notebooks

Self-contained Jupyter notebooks with executable end-to-end examples live in docs/examples/:

Implemented Algorithms

Implemented Environments

Custom environments are first-class: subclass Environment, implement the transition, observation, and reward models, and every planner and the whole experiment framework work with it out of the box. See custom_environment.ipynb.

Belief Representations

Beliefs are pluggable and planner-independent: unweighted and weighted particle filters, batched particle beliefs, Gaussian and Gaussian-mixture beliefs, and GPU-vectorized particle belief updaters for large-scale simulation.

Citing the Project

If you use POMDPPlanners in your research, please cite:

@misc{pariente2026pomdpplannersopensourcepackagepomdp,
      title={POMDPPlanners: Open-Source Package for POMDP Planning}, 
      author={Yaacov Pariente and Vadim Indelman},
      year={2026},
      eprint={2602.20810},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2602.20810}, 
}

Contributing & Support

Questions, bug reports, and feature requests are welcome on the issue tracker.

License

This project is licensed under the MIT License — see the LICENSE.md file for details.