Skip to main content

Package with tools for Resource-constrained Markov Decision Processes

Project description

FiMDP - Fuel in Markov Decision Processes

Python Package with Algorithms for Controller Synthesis in Resource-constrained Markov Decision Processes

Documentation Status Binder Build Status

FiMDP is a Python package for analysis and controller synthesis of Consumption Markov Decision Processes (ConsMDPs) — MDPs with resource constraints. The model of ConsMDPs and the basic algorithms implemented in FiMDP are described in our paper presented at CAV2020 [1].

Our related project called FiMDPEnv is a set of environments that work with FiMDP and can create animations like this one.

Multiple agents following energy-aware policy in grid-world.
Multiple agents following energy-aware policies in UUVEnv from FiMDPEnv.


FiMDP can be installed using pip from PyPI

pip install -U fimdp

While the baseline package has minimal dependencies, FiMDP depends on several other tools for extended functionality. Some of the recommended dependencies are:

  • Graphviz: for visualizations in Jupyter notebooks,
  • Storm and Stormpy: for reading PRSIM, JANI, and Storm models,
  • Spot: for support of labeled ConsMDPs and specifications given as deterministic Büchi automata or the recurrence fragment of Linear-time Temporal Logic (LTL).

Usage and documentation (work in progress)

The directory tut contains several notebooks that explain how to use FiMDP. The notebook Basics.ipynb is a good starting point.

For a complete overview of the tool, installation options, source code documentation, and interactive examples refer to FiMDP readthedocs.


Notebooks evaluationg performance of FiMDP are stored in a separate repository FiMDP-Evaluation.


If you have any trouble with the installation, or have any questions, raise an issue or email František (Fanda) Blahoudek or Pranay Thangeda.


[1] Blahoudek F., Brázdil T., Novotný P., Ornik M., Thangeda P., Topcu U. (2020) Qualitative Controller Synthesis for Consumption Markov Decision Processes. In proceeding of CAV 2020. Lecture Notes in Computer Science, vol 12225. Springer.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fimdp-2.0.tar.gz (39.2 kB view hashes)

Uploaded source

Built Distribution

fimdp-2.0-py3-none-any.whl (41.8 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page