Skip to main content

Python library and CLI for de-coupled data science project integration and management.

Project description

Projit

Project Integrator for Decoupled Data Science

Status: Beta Functional.

License: MIT Tests Documentation Status PyPI

Projit is a utility to help data scientists manage projects that contain multiple experiments and components that need to interact in a de-coupled manner. Use it to define and manage project structure, properties, data, experiments & collaboration.

The goal of this project is to allow data scientists and teams to work on projects in a structured and standardized way. The projit utility allows you to establish a project with a centralised meta-data repository. This meta-data is used by the application and package to facilitate loosely coupled communication between scripts for experiments, to track results and parameters.

For example, projit provides a python library that can be used inside experiments and scripts so references to training, evaluation and test data sets can accessed without passing around and maintaing paths.

In addition the project can be initialised according to a standardized layout so that the diectory structure is familiar to all team members.

This project was inspired by a combination of other projects:

Released and distributed via setuptools/PyPI/pip for Python 3.

Additional details and usage instructions available in the documentation

Notes

Initial implementation is focused allowing the user to initialise a project, and then modify it using a python package that can be called independently in scripts across the project structure. This creates a central authority for updating and retrieving info about data sets and experimental results.

Usage

You can use this application in multiple ways

Use the runner:

./projit-runner.py init "Test Project"

Invoke the directory as a package:

python -m projit init "Test Project"

Or simply install the package and use the command line application directly as shown in the process below:

Installation

Installation from the source tree:

git clone https://github.com/john-hawkins/projit
cd projit
python setup.py install

(or via pip from PyPI):

pip install projit

Now, the projit command is available:

projit init "Test Project"

This will initialise the current directory as a Data Science Project using the default template. Please refer to the documentation for more detail on projit commands.

Output

By adding experiments and results to a projit project you can examine them. Example output in the table below:

Results on [test]

experiment MAE MAPE
mytest 11230.46 13.46
mytest2 nan 15.86

Acknowledgements

Python package built using the bootstrap cmdline template by jgehrcke

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

projit-0.1.10.tar.gz (15.9 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page