mlf-core 1.7.7

Reproducible machine learning pipelines using mlflow.

.. image:: https://user-images.githubusercontent.com/21954664/84388841-84b4cc80-abf5-11ea-83f3-b8ce8de36e25.png
:target: https://mlf-core.com
:alt: mlf-core logo

|

========
mlf-core
========

.. image:: https://github.com/mlf-core/mlf-core/workflows/Build%20mlf-core%20Package/badge.svg
:target: https://github.com/mlf-core/mlf-core/workflows/Build%20mlf-core%20Package/badge.svg
:alt: Github Workflow Build mlf-core Status

.. image:: https://github.com/mlf-core/mlf-core/workflows/Run%20mlf-core%20Tox%20Test%20Suite/badge.svg
:target: https://github.com/mlf-core/mlf_core/workflows/Run%20mlf-core%20Tox%20Test%20Suite/badge.svg
:alt: Github Workflow Tests Status

.. image:: https://img.shields.io/pypi/v/mlf-core.svg
:target: https://pypi.python.org/pypi/mlf-core
:alt: PyPI

.. image:: https://img.shields.io/discord/742367395196305489?color=passing
:target: https://discord.gg/Mv8sAcq
:alt: Discord

.. image:: https://readthedocs.org/projects/mlf-core/badge/?version=latest
:target: https://mlf-core.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status

.. image:: https://flat.badgen.net/dependabot/thepracticaldev/dev.to?icon=dependabot
:target: https://flat.badgen.net/dependabot/thepracticaldev/dev.to?icon=dependabot
:alt: Dependabot Enabled


Fully GPU deterministic machine learning project templates using MLflow_.

* Free software: Apache2.0
* Documentation: https://mlf-core.readthedocs.io.

.. image:: https://user-images.githubusercontent.com/21954664/94257992-7a140e00-ff2c-11ea-8059-216a31c62ef1.gif
:target: https://user-images.githubusercontent.com/21954664/94257992-7a140e00-ff2c-11ea-8059-216a31c62ef1.gif
:alt: mlf-core create gif

Features
--------

* Jumpstart your machine learning project with fully fledged, multi GPU enabled mlflow project templates
* Pytorch, Tensorflow, XGBoost supported
* mlflow templates are fully GPU deterministic with system-intelligence
* Conda and Docker support out of the box

.. figure:: https://user-images.githubusercontent.com/21954664/98472352-c2dd0900-21f2-11eb-9fe3-929b2a21bd4c.png
:scale: 100 %
:alt: mlf-core summary

mlf-core enables deterministic machine learning. MLflow and a provided Read the Docs setup ensure that all hyperparameters, metrics and model details are well documented.
Reproducible environments are provided with the use of Conda and Docker. Finally, the mlf-core ecosystem ensures that all library specific settings required for determinism are enabled,
no non-deterministic algorithms are used and that the used hardware is tracked.

Credits
-------

Primary idea and main development by `Lukas Heumos <https://github.com/zethson/>`_.
This package was created with cookietemple_ based on a modified `audreyr/cookiecutter-pypackage`_ project template using Cookiecutter_.

.. _MLflow: https://mlflow.org
.. _cookietemple: https://cookietemple.com
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage


.. _changelog_f:

==========
Changelog
==========

This project adheres to `Semantic Versioning <https://semver.org/>`_.

1.7.7 (2020-11-29)
------------------

**Added**

* Support for deploying the documentation on Github Pages. By default the Documentation is pushed to the gh-pages branch.
Simply enable Github pages (repository settings) with the gh-pages branch and your documentation will build on ``https://username.github.io/repositoryname``

**Fixed**

* Workflows are now also triggered on PR

**Dependencies**

**Deprecated**


1.7.6 (2020-11-22)
------------------

**Added**

**Fixed**

* Github project creation support due to Github's new main branch

**Dependencies**

**Deprecated**

1.7.5 (2020-11-18)
------------------

**Added**

**Fixed**

sync workflow set-env

**Dependencies**

**Deprecated**


1.7.4 (2020-11-11)
------------------

**Added**

**Fixed**

* Sync now compares against the development branch and not the master branch.

**Dependencies**

**Deprecated**


1.7.3 (2020-11-09)
------------------

**Added**

**Fixed**

* Added CHANGELOG.rst to blacklisted files

**Dependencies**

**Deprecated**


1.7.2 (2020-11-07)
------------------

**Added**

**Fixed**

* Removed redundant print in xgboost

**Dependencies**

**Deprecated**


1.7.1 (2020-11-07)
------------------

**Added**

**Fixed**

* mlf-core sync does now correctly find attributes

**Dependencies**

**Deprecated**


1.7.0 (2020-11-06)
------------------

**Added**

* fix-artifact-paths which replaces the artifact paths with the paths of the current system
* More structured documentation

**Fixed**

* Now using GPUs by default only when GPUs are available for XGBoost templates

**Dependencies**

**Deprecated**


1.6.1 (2020-11-06)
------------------

**Added**

* Workflows for package-prediction
* Documentation for package-prediction

**Fixed**

**Dependencies**

**Deprecated**


1.6.0 (2020-11-02)
------------------

**Added**

* New package templates (package-prediction) for Pytorch, Tensorflow and XGBoost

**Fixed**

**Dependencies**

**Deprecated**


1.5.0 (2020-10-29)
------------------

**Added**

* Check for non-deterministic functions for mlflow-tensorflow linter
* Check for all_reduce for mlflow-xgboost templates
* Check for OS for system-intelligence runs. If not Linux -> don't run system-intelligence
* .gitattributes to templates, which ignores mlruns
* Documentation on creating releases

**Fixed**

* Sync now operates correctly with the correct PR URL

**Dependencies**

**Deprecated**


1.4.4 (2020-10-22)
------------------

**Added**

**Fixed**

* Conda report generation

**Dependencies**

**Deprecated**


1.4.3 (2020-09-17)
------------------

**Added**

**Fixed**

* Internal Github workflows
* Docker documentation

**Dependencies**

**Deprecated**

1.4.2 (2020-09-11)
------------------

**Added**

**Fixed**

* Accidentally left a - in the train_cpu.yml of mlflow-pytorch
* mlflow-pytorch and mlflow-tensorflow now only train for 2 epochs on train_cpu.yml

**Dependencies**

**Deprecated**


1.4.1 (2020-09-10)
------------------

**Added**

**Fixed**

* Github username must now always be lowercase, since Docker does not like uppercase letters
* Fixed train_cpu workflows to use the correct containers

**Dependencies**

**Deprecated**

1.4.0 (2020-08-28)
------------------

**Added**

* model.rst documentation for all templates
* added support for verbose output

**Fixed**

* Publish Docker workflows now use the new Github registry
* Default Docker container names are now ```image: ghcr.io/{{ cookiecutter.github_username }}/{{ cookiecutter.project_slug_no_hyphen }}:{{ cookiecutter.version }}```

**Dependencies**

**Deprecated**


1.3.0 (2020-08-27)
------------------

**Added**

* automatically mounting /data now in all mlflow templates (#56)
* mlflow-xgboost xgboost from 1.1.1 to 1.2.0

**Fixed**

* mlf_core.py now uses project_slug; adapted linter accordingly (#55)
* Removed dask-cuda from mlflow-xgboost

**Dependencies**

**Deprecated**


1.2.2 (2020-08-21)
------------------

**Added**

**Fixed**

* A couple of parameters were not with hyphen -> now default behavior

**Dependencies**

**Deprecated**


1.2.1 (2020-08-21)
------------------

**Added**

**Fixed**

* flake8 for mlflow-pytorch

**Dependencies**

**Deprecated**


1.2.0 (2020-08-21)
------------------

**Added**

* Option --view to config to view the current configuration
* Option --set_token to sync to set the sync token again

**Fixed**

* #41 https://github.com/mlf-core/mlf-core/issues/41 -> mlflow-pytorch multi GPU Support

**Dependencies**

**Deprecated**


1.1.0 (2020-08-19)
------------------

**Added**

* Publish Docker workflow. Publishes to Github Packages per default, but can be configured.
* Linting function, which checks mlflow-pytorch for any used atomic_add functions.
* system-intelligence 1.2.2 -> 1.2.3
* Support for both, MLF-CORE TODO: and TODO MLF-CORE: statements

**Fixed**

* Default project version from 0.1.0 to 0.1.0-SNAPSHOT.
* Outdated screenshots
* Nightly versions now warn instead of wrongly complaining about outdated versions.
* Sync actor, but not yet completely for organizations
* A LOT of documentation
* Now using project_slug_no_hyphen to facilitate the creation of repositories with - characters.
* Removed boston dataset from XGBoost and XGBoost_dask
* Renamed all parameters to use hyphens instead of underscores

**Dependencies**

**Deprecated**


1.0.1 (2020-08-11)
------------------

**Added**

**Fixed**

* Sync workflow now uses the correct secret

**Dependencies**

**Deprecated**


1.0.0 (2020-08-11)
------------------

**Added**

* Created the project using cookietemple
* Added all major commands: create, list, info, lint, sync, bump-version, config, upgrade
* Added mlflow-pytorch, mlflow-tensorflow, mlflow-xgboost, mlflow-xgboost_dask templates

**Fixed**

**Dependencies**

**Deprecated**