fimdp 1.0.2

Package with tools for Resource-constrained Markov Decision Processes

FiMDP - Fuel in Markov Decision Processes

Python Package with Algorithms for Controller Synthesis in Resource-constrained Markov Decision Processes

Full overview of the tool, installation options, documentation, and interactive examples: FiMDP readthedocs.

Citation Info: This work has been accepted to the 32nd International Conference on Computer-Aided Verification (CAV 2020) scheduled to take place July 19-24, 2020. Citation info will be updated once the conference proceedings are available online.

Overview

FiMDP is a Python package designed around a proof-of-concept tool implementing algorithms developed in our work on controller synthesis for resource-constrained problems modeled as Consumption Markov Decision Processes (CMDPs). The algorithms are detailed in the work titled 'Qualitative Controller Synthesis for Consumption Markov Decision Processes' by František Blahoudek, Tomáš Brázdil, Petr Novotný, Melkior Ornik, Pranay Thangeda and Ufuk Topcu.

Installation and Usage

Refer to the installation and usage section in the documentation for detailed instructions.

Binder: Use this link to access the interactive Jupter notebooks on the web without any installation.

Docker: The docker image with FiMDP is published on Docker Hub at this link.

Local Installation: Refer to the installation documentation.

Example Environments

We provide a number of examples in Interactive Jupyter notebooks to explore the features of our tool, to analyze its performance, and also to validate the results presented in our paper. The two primary examples are the problems of an electric vehicle routing with limited capacity and a multi-agent grid world inspired by the Mars 2020 mission. The notebooks can be found in the examples subdirectory and their contents and described in detail here.

Contact

If you have any trouble with the installation, or have any questions, raise an issue or email František Blahoudek or Pranay Thangeda.