ot_markov_distances
Differentiable distances on graphs based on optimal transport
This is the implementation code for
Brugere, T., Wan, Z., & Wang, Y. (2023). Distances for Markov Chains, and Their Differentiation. ArXiv, abs/2302.08621.
Setup
Installing as a library
The ot_markov_distances package can be installed with the following command:
pip install ot-markov-distances
If for some reason you need to use cuda11.8 (ie you are installing torch+cuda118) then use the following command instead
pip install git+https://github.com/YusuLab/ot_markov_distances@cuda118
Dependencies
Python version
This project requires python 3.10 a minima. If your python version is prior to 3.10, you need to update (or to create a new conda environment) to a version above (latest release at the time of writing is 3.12)
Python dependencies
This package manages its dependencies via poetry. I recommend you install it (otherwise if you prefer to manage them manually, a list of the dependencies is available in the file pyproject.toml)
When you have poetry, you can add dependencies using our makefile
$ make .make/deps
or directly with poetry
$ poetry install
TUDataset
If you are planning to reproduce the classification experiment.
The TUDataset package is also needed to run the classification experiment, but it is not available via pip / poetry. To install it, follow the instruction in the tudataset repo, including the “Compilation of kernel baselines” section, and add the directory where you downloaded it to your $PYTHONPATH. eg:
$ export PYTHONPATH="/path/to/tudataset:$PYTHONPATH"
Project structure
.
├── docs #contains the generated docs (after typing make)
│ ├── build
│ │ └── html #Contains the html docs in readthedocs format
│ └── source
├── experiments #contains jupyter notebooks with the experiments
│ └── utils #contains helper code for the experiments
├── ot_markov_distances #contains reusable library code for computing and differentiating the discounted WL distance
│ ├── discounted_wl.py # implementation of our discounted WL distance
│ ├── __init__.py
│ ├── sinkhorn.py # implementation of the sinkhorn distance
│ ├── utils.py # utility functions
│ └── wl.py #implementation of the wl distance by Chen et al.
├── staticdocs #contains the static source for the docs
│ ├── build
│ └── source
└── tests #contains sanity checks
Documentation
The documentation is available online: read the documentation
You can build documentation and run tests using
$ make
Alternatively, you can build only the documentation using
$ make .make/build-docs
The documentation will be available in docs/build/html in the readthedocs format
Running Experiments
Running experiments requires installing development dependencies. This can be done by running
$ make .make/dev-deps
or alternatively
$ poetry install --with dev
Experiments can be found in the experiments/ directory (see Project structure ).
The Barycenter and Coarsening experiments can be found in experiments/Barycenter.ipynb and experiments/Coarsening.ipynb.
The performance graphs are computed in experiments/Performance.ipynb
Classification experiment
The Classification experiment (see the first paragraph of section 6 in the paper) is not in a jupyter notebook, but accessible via a command line.
As an additional dependency it needs tudataset, which is not installable via pip. To install it follow the instructions in the tudataset repo. , including the “Compilation of kernel baselines” section, and add the directory where you downloaded it to your $PYTHONPATH.
Now you can run the classification experiment using the command
$ poetry run python -m experiments.classification
usage: python -m experiments.classification [-h] {datasets_info,distances,eval} ...
Run classification experiments on graph datasets
positional arguments:
{datasets_info,distances,eval}
datasets_info Print information about given datasets
distances Compute distance matrices for given datasets
eval Evaluate a kernel based on distance matrix
options:
-h, --help show this help message and exit
The yaml file containing dataset information that should be passed to the command line is in experiments/grakel_datasets.yaml. Modifying this file should allow running the experiment on different datasets.
FAQ
I have a question about the paper
In this case just send me an email through the email address mentioned in the paper.
I have noticed a bug in the code
Please use the Github “Issues” feature to open a ticket, and post a description of the bug, the error message and a minimal reproducible example . I’ll try to fix it.
Or if you have fixed it, you can submit a Pull Request directly
I cannot install the library
If you followed all the instructions correctly, please create a ticket using Github Issues.
Why do you need python3.10 ?
Because I am using structural pattern matching, and some typing features such as this one .
Release files for ot-markov-distances 1.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| ot_markov_distances-1.0.4.tar.gz | 18.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| ot_markov_distances-1.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 35.9 kB
Release files / ot_markov_distances-1.0.4.tar.gz
| Download URL | ot_markov_distances-1.0.4.tar.gz |
|---|---|
| Size | 18.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c23e555f25dc186e4dbacf04f837ae4a225b2715959217ee8b471a1635f2bd1b
|
|
BLAKE2b-256 checksum How to use checksums |
3a5b5ddbb9a13f0bbc1a029dfb617b1d17f724dd0ebb43a4a996ffa499dc81b0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.7
|
Release files / ot_markov_distances-1.0.4-py3-none-any.whl
| Download URL | ot_markov_distances-1.0.4-py3-none-any.whl |
|---|---|
| Size | 17.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9797481e70da3211c5a4e217cfdf071a6812ab0edc975ad496d3f2ebb5a856ca
|
|
BLAKE2b-256 checksum How to use checksums |
ace5610cf06f115e9936955bc0d07e1b8e27530126f4b75f811b0c3fa235af1a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/4.0.2 CPython/3.11.7
|