_  __    _  __              __             _      __
   _____ _____ (_)/ /__ (_)/ /_        ____/ /___   _____ (_)____/ /___
  / ___// ___// // //_// // __/______ / __  // _ \ / ___// // __  // _ \
 (__  )/ /__ / // ,<  / // /_ /_____// /_/ //  __// /__ / // /_/ //  __/
/____/ \___//_//_/|_|/_/ \__/        \__,_/ \___/ \___//_/ \__,_/ \___/

Scikit-decide for Python

Scikit-decide is an AI framework for Reinforcement Learning, Automated Planning and Scheduling.

This framework was initiated at Airbus AI Research and notably received contributions through the ANITI and TUPLES projects, and also from ANU.

Main features

• Problem solving: describe your decision-making problem once and auto-match compatible solvers.
  For instance planning/scheduling problems can be solved by RL solvers using GNNs.
• Growing catalog: enjoy a growing list of domains & solvers catalog, supported by the community.
• Open & Extensible: scikit-decide is open source and is able to wrap existing state-of-the-art domains/solvers.
• Domains available:
  • Gym(nasium) environments for reinforcement learning (RL)
  • PDDL (Planning Domain Definition Language) via unified-planning and plado libraries
    • encoding in gym(nasium) spaces compatible with RL
    • graph representations for RL (inspired by Lifted Learning Graph) 🆕
  • RDDL (Relational Dynamic Influence Diagram Language) using pyrddl-gym library.
  • Flight planning, based on openap or in-house Poll-Schumann for performance model
  • Scheduling, based on rcpsp problem from discrete-optimization library
  • Toy domains like: maze, mastermind, rock-paper-scissors
• Solvers available:
  • RL solvers from ray.rllib and stable-baselines3
    • existing algos with action masking
    • adaptation of RL algos for graph observation, based on GNNs from pytorch-geometric 🆕
    • autoregressive models with action masking component by component for parametric actions 🆕
  • Planning solvers from unified-planning library
  • RDDL solvers jax and gurobi-based based on pyRDDLGym-jax and pyRDDLGym-gurobi from pyrddl-gym project
  • Search solvers coded in scikit-decide library:
    • A*, AO*, Improved-LAO*
    • Value Iteration (VI), Policy Iteration (PI)
    • Labeled RTDP, Learning Real-Time A*
    • LDFS (Label-correcting Depth-First Search), Iterative Deepening A*
    • SSiPP (Short-Sighted Planning), FRET (Find, Revise, Eliminate Traps)
    • iDual (LP-based SSP solver), Goal Probability and Cost Iteration (GPCI)
    • Best First Width Search, Iterated Width (IW), Rollout IW (RIW)
    • Monte Carlo Tree Search (MCTS), POMCP
    • DESPOT, SARSOP, Witness (POMDP solvers)
    • RTDP-Bel (belief-space RTDP), HSVI / GoalHSVI
    • SSPReplan, SSPDetHindsight, SSPPlanMerger (determinization approaches)
    • Multi-Agent RTDP, Multi-Agent Heuristic meta-solver (MAHD)
  • (Probabilistic) PDDL (PPDDL) solvers:
    • FF planner
    • FFReplan / PPDDLReplan (replanning with pluggable inner solvers)
    • FFDetHindsight / PPDDLDetHindsight (determinization in hindsight)
    • RFF / PPDDLPlanMerger (plan aggregation into a policy)
  • PDDL heuristics (with their probabilistic extensions):
    • Delete-Relaxation heuristics
    • FF Heuristic
  • PDDL+ parser and simulators with Probabilistic PDDL extensions
    • Lifted applicable action filtering using Clingo
    • Z3-based event synchronization in python using z3-solver
  • Evolution strategy: Cartesian Genetic Programming (CGP)
  • Scheduling solvers from discrete-optimization,
    • itself wrapping ortools, gurobi, toulbar, minizinc, deap (genetic algorithm), didppy (dynamic programming),
    • and coding local search (hill climber, simulated annealing), Large Neighborhood Search (LNS), and genetic programming based hyper-heuristic (GPHH)
• Tuning solvers hyperparameters
  • hyperparameters definition
  • automated study with optuna

Installation

Quick version:

pip install scikit-decide[all]

For more details, see the online documentation.

Documentation

The latest documentation is available online.

Examples

Some educational notebooks are available in notebooks/ folder. Links to launch them online with binder are provided in the Notebooks section of the online documentation.

More examples can be found as Python scripts in the examples/ folder, showing how to import or define a domain, and how to run or solve it. Most of the examples rely on scikit-decide Hub, an extensible catalog of domains/solvers.

Contributing

See more about how to contribute in the online documentation.