A library for datasets containing heterogeneous data
Project description
Connectome is a framework for datasets management with strong emphasis on simplicity, composability and reusability.
Features
- Self-consistency: connectome encourages data transformations that keep entries' fields consistent
- Caching: transformations' caching works out of the box and supports both caching to RAM and to Disk
- Automatic cache invalidation: connectome tracks all the changes made to a dataset and automatically invalidates the cache when something changes, making sure that your cache is always consistent with the data
- Invertible transformations: write consistent pre- and post- processing to build production-ready pipelines
Install
The simplest way is to get it from PyPi:
pip install connectome
Or if you want to try the latest version from GitHub:
git clone https://github.com/neuro-ml/connectome.git
cd connectome
pip install -e .
# or let pip handle the cloning:
pip install git+https://github.com/neuro-ml/connectome.git
Getting started
The docs are located here
Also, you can check out our Intro to connectome
series of
tutorials here
Acknowledgements
Some parts of our automatic cache invalidation machinery vere heavily inspired by the cloudpickle project.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
connectome-0.10.0.tar.gz
(47.8 kB
view hashes)