PyDTMC

PyDTMC is a full-featured, lightweight library for discrete-time Markov chains analysis. It provides classes and functions for creating, manipulating and simulating markovian stochastic processes.

Requirements

PyDTMC supports only Python 3 and the minimum required version is 3.6. In addition, the environment must include the following libraries:

For a better user experience, it's recommended to install Graphviz and PyDot before using the plot_graph function. In order to build the project documentation, it's necessary to install Sphinx. In order to perform unit tests, it's necessary to install PyTest.

Installation & Upgrade

Via PyPI:

$ pip install PyDTMC
$ pip install --upgrade PyDTMC

Via GitHub:

$ pip install git+https://github.com/TommasoBelluzzo/PyDTMC.git@master#egg=PyDTMC
$ pip install --upgrade git+https://github.com/TommasoBelluzzo/PyDTMC.git@master#egg=PyDTMC

Usage

The core element of the library is the MarkovChain class, which can be instantiated as follows:

>>> p = [[0.2, 0.7, 0.0, 0.1], [0.0, 0.6, 0.3, 0.1], [0.0, 0.0, 1.0, 0.0], [0.5, 0.0, 0.5, 0.0]]
>>> mc = MarkovChain(p, ['A', 'B', 'C', 'D'])
>>> print(mc)

DISCRETE-TIME MARKOV CHAIN
 SIZE:           4
 RANK:           4
 CLASSES:        2
  > RECURRENT:   1
  > TRANSIENT:   1
 ERGODIC:        NO
  > APERIODIC:   YES
  > IRREDUCIBLE: NO
 ABSORBING:      YES
 REGULAR:        NO
 REVERSIBLE:     NO

Below a few examples of MarkovChain instance properties and static computations:

>>> print(mc.is_ergodic)
False

>>> print(mc.recurrent_states)
['C']

>>> print(mc.transient_states)
['A', 'B', 'D']

>>> print(mc.steady_states)
[array([0.0, 0.0, 1.0, 0.0])]

>>> print(mc.is_absorbing)
True

>>> print(mc.fundamental_matrix)
[[1.50943396 2.64150943 0.41509434]
 [0.18867925 2.83018868 0.30188679]
 [0.75471698 1.32075472 1.20754717]]
 
>>> print(mc.kemeny_constant)
5.547169811320755

>>> print(mc.mean_absorption_times())
[4.56603774 3.32075472 3.28301887]

>>> print(mc.absorption_probabilities())
[1.0 1.0 1.0]

>>> print(mc.entropy_rate)
None

Dynamic computations on MarkovChain instances can be performed through their parametrized methods:

>>> print(mc.expected_rewards(10, [2, -3, 8, -7]))
[-2.76071635, -12.01665113, 23.23460025, -8.45723276]

>>> print(mc.expected_transitions(2))
[[0.085, 0.2975, 0.0,    0.0425]
 [0.0,   0.345,  0.1725, 0.0575]
 [0.0,   0.0,    0.7,    0.0   ]
 [0.15,  0.0,    0.15,   0.0   ]]

>>> print(mc.first_passage_probabilities(5, 3))
[[0.5, 0.0,    0.5,    0.0   ]
 [0.0, 0.35,   0.0,    0.05  ]
 [0.0, 0.07,   0.13,   0.045 ]
 [0.0, 0.0315, 0.1065, 0.03  ]
 [0.0, 0.0098, 0.0761, 0.0186]]
 
>>> print(mc.hitting_probabilities([0, 1]))
[1.0, 1.0, 0.0, 0.5]
 
>>> print(mc.walk(10))
['B', 'B', 'B', 'D', 'A', 'B', 'B', 'C', 'C', 'C']

Plotting functions can provide a visual representation of MarkovChain instances; in order to display function outputs immediately, the interactive mode of Matplotlib must be turned on:

>>> plot_eigenvalues(mc)

Eigenplot

>>> plot_graph(mc)

Graphplot

>>> plot_walk(mc, 10, 'sequence')

Walkplot

Release files for PyDTMC 5.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for PyDTMC 5.5.0
File	Size	Uploaded
PyDTMC-5.5.0.tar.gz	58.5 kB	Jun 11, 2021	Details

Built distribution (wheel)

Table of built distributions (wheels) for PyDTMC 5.5.0
File	Interpreter	ABI	Platform	Reset
PyDTMC-5.5.0-py3-none-any.whl 44.0 kB Jun 11, 2021	Python 3	none	any	Details

Total release size: 102.5 kB

Release files / PyDTMC-5.5.0.tar.gz

Download URL	PyDTMC-5.5.0.tar.gz
Size	58.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`6bfa5b64492d92d60c54109ee702f1724a61ded6930d54399ec326dbb19b2467`
BLAKE2b-256 checksum How to use checksums	`e49e3b2abba0dafb423d543270162b118469387efd9f78d776bf057b22987aca`
Upload date	Jun 11, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.6.6`

Release files / PyDTMC-5.5.0-py3-none-any.whl

Download URL	PyDTMC-5.5.0-py3-none-any.whl
Size	44.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`0d860f9a108a3be8667ca4d62a8fdcb14994ccc9fd0da0aa8a197befaa0693d8`
BLAKE2b-256 checksum How to use checksums	`231a31dc40c86874e332f6ad9e52b62464344e5eb3a07c828ce30a1da2c76d18`
Upload date	Jun 11, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.6.6`