quickpomdps - python

quickpomdps is a package to quickly define [PO]MDPs in Python. You can use any of the solvers in POMDPs.jl ecosystem, directly in Python.

A hybrid continuous-discrete light-dark problem definition and QMDP solution (taken from examples/lightdark.py) looks like this:

r = 60
light_loc = 10

def transition(s, a):
    if a == 0:
        return Deterministic(r+1)
    else:
        return Deterministic(min(max(s+a, -r), r))

def observation(s, a, sp):
    return Normal(sp, abs(sp - light_loc) + 0.0001)

def reward(s, a, sp):
    if a == 0:
        return 100.0 if s == 0 else -100.0
    else:
        return -1.0

m = QuickPOMDP(
    states = range(-r, r+2),
    actions = [-10, -1, 0, 1, 10],
    discount = 0.95,
    isterminal = lambda s: s < -r or s > r,
    obstype = Float64,
    transition = transition,
    observation = observation,
    reward = reward,
    initialstate = Uniform(range(-r//2, r//2+1))
)

solver = QMDPSolver()
policy = solve(solver, m)

Installation

pip install quickpomdps

quickpomdps uses the pyjulia package which requires julia to be installed. We recommend using juliaup for this purpose.

Upon invocation of import quickpomds in Python, all Julia dependencies will be installed if they are not already present. Please note that, by default, the Julia dependencies are added to the global environment. If you want to install these dependencies to a local environment instead, export the JULIA_PROJECT with the desired path as documented here.

Development

This package uses python-poetry for dependency management. Thus, it may be installed via one of the may ways supported by poetry, for example,

git clone https://github.com/JuliaPOMDP/quickpomdps
cd quickpomdps
poetry install
poetry run python examples/lightdark.py

Usage

See examples/ and tests/. Documentation can be found at the QuickPOMDPs.jl and POMDPs.jl packages.

Help

If you need help, please ask on the POMDPs.jl discussions page!