Skip to main content

A framework for discrete-time Markov chains analysis.

Project description

PyDTMC is a full-featured, lightweight library for discrete-time Markov chains analysis. It provides classes and functions for creating, manipulating and simulating markovian stochastic processes.

Requirements

PyDTMC supports only Python 3 and the minimum version required is 3.6. In addition, the environment must include the following libraries:

In order to use the plot_graph function, Graphviz and PyDot must be installed too.

Installation & Upgrade

Via PyPI:

$ pip install PyDTMC
$ pip install --upgrade PyDTMC

Via GitHub:

$ pip install git+https://github.com/TommasoBelluzzo/PyDTMC.git@master#egg=PyDTMC
$ pip install --upgrade git+https://github.com/TommasoBelluzzo/PyDTMC.git@master#egg=PyDTMC

Usage

The core element of the library is the MarkovChain class, which can be instantiated as follows:

>>> import numpy as np
>>> p = np.array([[0.2, 0.7, 0.0, 0.1], [0.0, 0.6, 0.3, 0.1], [0.0, 0.0, 1.0, 0.0], [0.5, 0.0, 0.5, 0.0]])
>>> mc = MarkovChain(p, ['A', 'B', 'C', 'D'])
>>> print(mc)

DISCRETE-TIME MARKOV CHAIN

 - TRANSITION MATRIX:

            A       B       C       D
      ------- ------- ------- -------
  A | 0.20000 0.70000 0.00000 0.10000
  B | 0.00000 0.60000 0.30000 0.10000
  C | 0.00000 0.00000 1.00000 0.00000
  D | 0.50000 0.00000 0.50000 0.00000

 - PROPERTIES:

  ABSORBING:   YES
  APERIODIC:   YES
  IRREDUCIBLE: NO
  ERGODIC:     NO

 - COMMUNICATING CLASSES:

          [A,B,D] | [C]
  TYPE:         T |   R
  PERIOD:       1 |   1

Static values of a MarkovChain instance can be retrieved through its properties:

>>> print(mc.size)
4

>>> print(mc.recurrent_states)
['C']

>>> print(mc.transient_states)
['A', 'B', 'D']

>>> print(mc.steady_states)
[array([0., 0., 1., 0.])]

>>>print(mc.fundamental_matrix)
[[1.50943396 2.64150943 0.41509434]
 [0.18867925 2.83018868 0.30188679]
 [0.75471698 1.32075472 1.20754717]]

>>> print(mc.absorption_times)
[4.56603774 3.32075472 3.28301887]

>>> print(mc.topological_entropy)
0.6931471805599457

Dynamic computations on a MarkovChain instance can be performed through its methods:

>>> print(mc.expected_rewards(10, [2, -3, 8, -7]))
[-2.76071635 -12.01665113  23.23460025  -8.45723276]

>>> print(mc.expected_transitions(2))
[[0.085  0.2975 0.     0.0425]
 [0.     0.345  0.1725 0.0575]
 [0.     0.     0.7    0.    ]
 [0.15   0.     0.15   0.    ]]
 
>>> print(mc.walk(10))
['D', 'A', 'B', 'C', 'C', 'C', 'C', 'C', 'C', 'C']

Plotting functions can provide a visual representation of a MarkovChain instance and its attributes. In order to immediately display function outputs, the interactive mode of Matplotlib must be turned on.

>>> plot_eigenvalues(mc)

Eigenplot

>>> plot_graph(mc)

Graphplot

>>> plot_walk(mc, 10, 'sequence')

Walkplot

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

PyDTMC-1.1.0.tar.gz (28.1 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page