deeprob-kit 1.1.0

A Python Library for Deep Probabilistic Modeling

Abstract

DeeProb-kit is a general-purpose Python library providing a collection of deep probabilistic models (DPMs) which are easy to use and extend. It also includes efficiently implemented learning techniques, inference routines and statistical algorithms. The availability of a representative selection of the most common DPMs in a single library makes it possible to combine them in a straightforward manner, a common practice in deep learning research nowadays, which however is still missing for certain class of models. Moreover, DeeProb-kit provides high-quality fully-documented APIs, and it will help the community to accelerate research on DPMs as well as improve experiments' reproducibility.

Features

• Inference algorithms for SPNs. ^{1 4}
• Learning algorithms for SPNs structure. ^{1 2 3 4 5}
• Chow-Liu Trees (CLT) as SPN leaves. ^{12 13}
• Batch Expectation-Maximization (EM) for SPNs with arbitrarily leaves. ^{14 15}
• Structural marginalization and pruning algorithms for SPNs.
• High-order moments computation for SPNs.
• JSON I/O operations for SPNs and CLTs. ⁴
• Plotting operations based on NetworkX for SPNs and CLTs. ⁴
• Randomized And Tensorized SPNs (RAT-SPNs). ⁶
• Deep Generalized Convolutional SPNs (DGC-SPNs). ¹¹
• Masked Autoregressive Flows (MAFs). ⁷
• Real Non-Volume-Preserving (RealNVP) flows. ⁸
• Non-linear Independent Component Estimation (NICE) flows. ⁹

The collection of implemented models is summarized in the following table. The supported data dimensionality for each model is showed in the Input Dimensionality column. Moreover, the Supervised column tells which model is suitable for a supervised learning task, other than density estimation task.

Legend — D: one-dimensional size, C: channels, H: height, W: width.

Model	Description	Input Dimensionality	Supervised
Binary-CLT	Binary Chow-Liu Tree (CLT)	D	❌
SPN	Vanilla Sum-Product Network	D	✔
MSPN	Mixed Sum-Product Network	D	✔
XPC	Random Probabilistic Circuit	D	✔
RAT-SPN	Randomized and Tensorized Sum-Product Network	D	✔
DGC-SPN	Deep Generalized Convolutional Sum-Product Network	(C, D, D)	✔
MAF	Masked Autoregressive Flow	D	❌
NICE	Non-linear Independent Components Estimation Flow	D and (C, H, W)	❌
RealNVP	Real-valued Non-Volume-Preserving Flow	D and (C, H, W)	❌

Installation

The library can be installed either from PIP repository or by source code.

# Install from PIP repository
pip install deeprob-kit

# Install from `main` git branch
pip install -e git+https://github.com/deeprob-org/deeprob-kit.git@main#egg=deeprob-kit

Project Directories

The documentation is generated automatically by Sphinx using sources stored in the docs directory.

A collection of code examples and experiments can be found in the examples and experiments directories respectively. Moreover, benchmark code can be found in the benchmark directory.

Related Repositories

• SPFlow
• RAT-SPN
• Random-PC
• LibSPN-Keras
• MAF
• RealNVP

References

1. Peharz et al. On Theoretical Properties of Sum-Product Networks. AISTATS (2015).

2. Poon and Domingos. Sum-Product Networks: A New Deep Architecture. UAI (2011).

3. Molina, Vergari et al. Mixed Sum-Product Networks: A Deep Architecture for Hybrid Domains. AAAI (2018).

4. Molina, Vergari et al. SPFLOW : An easy and extensible library for deep probabilistic learning using Sum-Product Networks. CoRR (2019).

5. Di Mauro et al. Sum-Product Network structure learning by efficient product nodes discovery. AIxIA (2018).

6. Peharz et al. Probabilistic Deep Learning using Random Sum-Product Networks. UAI (2020).

7. Papamakarios et al. Masked Autoregressive Flow for Density Estimation. NeurIPS (2017).

8. Dinh et al. Density Estimation using RealNVP. ICLR (2017).

9. Dinh et al. NICE: Non-linear Independent Components Estimation. ICLR (2015).

10. Papamakarios, Nalisnick et al. Normalizing Flows for Probabilistic Modeling and Inference. JMLR (2021).

11. Van de Wolfshaar and Pronobis. Deep Generalized Convolutional Sum-Product Networks for Probabilistic Image Representations. PGM (2020).

12. Rahman et al. Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees. ECML-PKDD (2014).

13. Di Mauro, Gala et al. Random Probabilistic Circuits. UAI (2021).

14. Desana and Schnörr. Learning Arbitrary Sum-Product Network Leaves with Expectation-Maximization. CoRR (2016).

15. Peharz et al. Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits. ICML (2020).