Skip to main content

Practical dataset management

Project description

Sefara is a Python library for managing your datasets. It provides a way to specify once what your datasets are (usually fileystem paths) and any metadata (e.g. which experiment they came from), then refer to them conveniently in analysis scripts and notebooks.

Sefara doesn’t assume anything about what your datasets are, what format they’re in, or are how they are accessed.

Quick example

Define a “resource collection” by making a file like this, which we’ll call datasets.sefara.py:

from sefara import export

export(
    "my_first_dataset.hdf5",
    path="/path/to/file1.hdf5",
    tags=["first", "important"],
)
export(
    "my_second_dataset.csv",
    path="/path/to/file2.csv",
    tags=["second", "unimportant"],
)

Then, use Sefara to open it in Python:

>>> import sefara
>>> datasets = sefara.load("datasets.sefara.py")
>>> print(datasets.filter("tags.important")[0].path)
/path/to/file1.hdf5

Documentation

Available at: http://timodonnell.github.io/sefara/docs/html

Installation

pip install sefara

To run the tests:

nosetests

To build the documentation:

pip install -e .
pip install Sphinx
cd docs
make clean setup rst html

The docs will be written to the _build/html directory.

Project details


Release history Release notifications

This version
History Node

0.2.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
sefara-0.2.1.tar.gz (18.5 kB) Copy SHA256 hash SHA256 Source None Jul 6, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page