Skip to main content

Utilities of MLOps for INRIA

Project description

common-mlops

This is a set of utilities for end-to-end lifecycle of Machine Learning applications in Python for Inria-Chile. Could be used like a framework and tooling for rapid development of data science and MLOps functions.

Here’s the folder structure for the repository:

common-mlops/
|
├── data/
│   ├── prepared/
│   └── raw/
│   └── metrics/
├── model/
├── mlops/
|   └── *.py
├── test/
|   └── test_*.py
├── venv/
├── check_credentials.sh
├── install.sh
├── LICENSE
├── README.md
├── requirements.txt
├── s3*.py
└── setup.py

There are the folloging folders in our repository:

  • data/ is all versions of the dataset.
  • data/raw/ is the data obtained from an external source.
  • data/prepared/ is for data modified internally.
  • data/metrics/ is for tracking the performance metrics of our models.
  • model/ is for machine learning models.
  • mlops/ is the source code. Python classes used by the Python scripts.
  • test/ is the pyunit of source code.
  • venv/ is the virtual environment.
  • check_credentials.sh bash for check the AWS credentials.
  • install.sh bash for check the installation.
  • LICENSE
  • requirements.txt the dependencies.
  • s3*.py Python scripts for bash executions.
  • setup.py the configuration of the package.

Installation Linux

Perform the following steps in order to install this program. Replace 'access_key_id' and 'secret_access_key' by the account AWS Credentials associated to poc-inriacl.

git clone https://gitlab.com/Inria-Chile/common-mlops.git
./common-mlops/install.sh access_key_id secret_access_key
pip3 uninstall common-mlops
# Version of test
pip3 install -i https://test.pypi.org/simple/ common-mlops
# Version of production
pip3 install common-mlops

Updates in test.pypi.org

Perform the following steps in order to publish new versions to test.pypi.org:

sudo rm -rf build common_mlops.egg-info dist
bumpversion --current-version 1.0.0 minor setup.py mlops/__init__.py
python3 setup.py sdist bdist_wheel
# Version of test
twine check dist/*
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
# Version of production
twine upload dist/*

Tests

Run the unit tests:

python3 -m unittest

Usage

See the example of usage in:

Docker

Build the image:

docker build -t common-mlops .
docker pull common-mlops

Run the image:

docker run -v$HOME/.aws:/root/.aws:ro -e S3_ACCESS_KEY_ID=access_key_id -e S3_SECRET_ACCESS_KEY=secret_access_key -e AWS_ACCESS_KEY_ID=access_key_id -e AWS_SECRET_ACCESS_KEY=secret_access_key common-mlops .

References

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

common-mlops-0.1.1.tar.gz (14.6 kB view details)

Uploaded Source

Built Distribution

common_mlops-0.1.1-py3-none-any.whl (16.7 kB view details)

Uploaded Python 3

File details

Details for the file common-mlops-0.1.1.tar.gz.

File metadata

  • Download URL: common-mlops-0.1.1.tar.gz
  • Upload date:
  • Size: 14.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.8.5

File hashes

Hashes for common-mlops-0.1.1.tar.gz
Algorithm Hash digest
SHA256 33c6a5a66933fcf2af55531c3c96e6c63c3a0f17313ea29f095ac2df9ae2c6fe
MD5 94680e2cb2a26e53d7e9401ad1b7d44a
BLAKE2b-256 115f2e095da2fb6acf320d2a496f17da904eb23fd45f24ea9a8c750f8b5f71b9

See more details on using hashes here.

File details

Details for the file common_mlops-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: common_mlops-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 16.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.8.5

File hashes

Hashes for common_mlops-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 af9d3028a776f22bfdb8723a515d9c32fe318ccd1fbfce67ae00de034c83a13d
MD5 800e67f375c76d08b65721197d8805a0
BLAKE2b-256 b908e7cf1f765c4cd5a40109715809d850d76612f339ba5e2a61ac3b8cfeff5e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page