Parallel random matrix tools and random matrix theory deep learning applications. Generate matrices from Circular Unitary Ensemble (CUE), Circular Ortogonal Ensemble (COE) and Circular Symplectic Ensemble (CSE). Additional spectral analysis utilities are also implemented, such as computation of spectral density and spectral ergodicity for complexity of deep learning architectures.
Project description
Bristol
Parallel random matrix tools and random matrix theory deep learning applications. Generate matrices from Circular Unitary Ensemble (CUE), Circular Ortogonal Ensemble (COE) and Circular Symplectic Ensemble (CSE). Additional spectral analysis utilities are also implemented, such as computation of spectral density and spectral ergodicity for complexity of deep learning architectures.
Features
- Generation of Circular Ensembles: CUE, COE and CSE.
- Random matrices: Reproducibility both in serial and parallel processing.
- Eigenvalue Spectra, spectral densitiy.
- Kullbach-Leibler divergence and spectral ergodicity measure functionality.
- Cascading Periodic Spectral Ergodicity (cPSE)
Installation
Install with pip from pypi.
pip install bristol
To use the latest development version
pip install -upgrade git+https://github.com/msuzen/bristol.git
Documentation
Complexity of a deep learning model: cPSE
You need to put your model as pretrained model format of PyTorch. An example for vgg,
and use cPSE.cpse_measure
function simply:
from bristol import cPSE
import torchvision.models as models
netname = 'vgg11'
pmodel = getattr(models, netname)(pretrained=True)
(d_layers, cpse) = cPSE.cpse_measure(pmodel)
This would give cpse
a single number expressing the complexity of your network and d_layers
evolution of
periodic spectral ergodicity
withing layers as a vector, order matters.
Complex Circular Ensembles and prototype notebooks
-
Basics of circular ensembles ipynb.
-
Computing spectral ergodicity for generated matrices ipynb. This is to reproduce the main figure from arXiv:1704.08693.
-
The concept of cascading periodic ergodicity (cPSE) ipynb This is only to reproduce paper's results from arXiv:1911.07831.
Contact
- Please create an issue for any type of questions or contact
msuzen
.
References
-
Berry, M V & Pragya Shukla 2013, Hearing random matrices and random waves, New. J. Phys. 15 013026 (11pp) berry456
-
Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices, Mehmet Süzen, Cornelius Weber, Joan J. Cerdà, arXiv:1704.08693
-
Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search, Mehmet Süzen, Cornelius Weber, Joan J. Cerdà, arXiv:1911.07831
Citation
If you use the ideas or tools from this package please do cite our manuscripts.
@article{suezen2017a,
title={Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices},
author={Mehmet Süzen and Cornelius Weber and Joan J. Cerdà},
year={2017},
eprint={1704.08303},
archivePrefix={arXiv},
primaryClass={stat.ML}
}
@article{suezen2019a,
title={Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search},
author={Mehmet Süzen and Cornelius Weber and Joan J. Cerdà},
year={2019},
eprint={1911.07831},
archivePrefix={arXiv},
primaryClass={stat.ML}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.