Skip to main content

A library for datasets containing heterogeneous data

Project description

docs codecov pypi License PyPI - Downloads

Connectome is a framework for datasets management with strong emphasis on simplicity, composability and reusability.

Features

  • Self-consistency: connectome encourages data transformations that keep entries' fields consistent
  • Caching: transformations' caching works out of the box and supports both caching to RAM and to Disk
  • Automatic cache invalidation: connectome tracks all the changes made to a dataset and automatically invalidates the cache when something changes, making sure that your cache is always consistent with the data
  • Invertible transformations: write consistent pre- and post- processing to build production-ready pipelines

Install

The simplest way is to get it from PyPi:

pip install connectome

Or if you want to try the latest version from GitHub:

git clone https://github.com/neuro-ml/connectome.git
cd connectome
pip install -e .

# or let pip handle the cloning:
pip install git+https://github.com/neuro-ml/connectome.git

Getting started

The docs are located here

Also, you can check out our Intro to connectome series of tutorials here

Acknowledgements

Some parts of our automatic cache invalidation machinery vere heavily inspired by the cloudpickle project.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

connectome-0.10.0.tar.gz (47.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page