Minerva

Minerva is a Pytorch-Lightning-based framework for training machine learning models for researchers. Check documentation and examples in Minerva Documentation.

Description

This project aims to provide a robust and flexible framework for researchers working on machine learning projects. It includes various utilities and modules for data transformation, model creation, analysis metrics, and reproducibility. Minerva is designed to be modular and extensible, allowing researchers to easily add new features and functionalities.

Features

Minerva offers a wide range of features to help you with your machine learning projects:

• Model Creation: Minerva offers a variety of models and architectures to choose from.
• Training and Evaluation: Minerva provides tools to train and evaluate your models, including loss functions, optimizers, and evaluation metrics.
• Data Transformation: Minerva provides tools to preprocess and transform your data, including data loaders, data augmentation, and data normalization.
• Modular Design: Minerva is designed to be modular and extensible, allowing you to easily add new features and functionalities.
• Reproducibility: Minerva ensures reproducibility by providing tools for versioning, configuration, and logging of experiments.
• Self-Supervised Learning (SSL) Support: Minerva supports Self-Supervised Learning (SSL) for training models with limited labeled data.
• Development Environment: Minerva provides a development environment with all dependencies pre-installed and configured for you.

Near Future Features

• Hyperparameter Optimization: Minerva will offer tools for hyperparameter optimization powered by Ray Tune.
• PyPI Package: Minerva will be available as a PyPI package for easy installation.
• Pre-trained Models: Minerva will offer pre-trained models for common tasks and datasets.
• Experiment Management: Minerva will offer tools for managing and tracking experiments using well-known tools like MLflow.

Installation

Minerva is available on PyPI. We recommend installing it with uv, a fast Python package and project manager, though pip works as well.

Install with uv (Recommended)

If you don't have uv, see the uv installation guide.

uv pip install minerva

Install with pip

pip install minerva

Install Locally

1. Clone the repository:

git clone https://github.com/discovery-unicamp/Minerva.git
cd Minerva

2. Install dependencies — with uv (recommended):

uv pip install .

Or with pip:

pip install .

VSCode Development Environment using DevContainer

Check the Using Minerva DevContainer for developing with Minerva guide for instructions on how to set up a development environment using Visual Studio Code and Docker.

Modules

Once installed, Minerva is just like any other Python package. You can import its modules and classes in your Python code using:

import minerva

The main modules available in Minerva are:

• minerva.analysis: Tools for analyzing and visualizing model performance.
• minerva.callback: Callbacks for monitoring and logging training progress (Pytorch-Lightning based).
• minerva.data: Readers, Datasets and Data Modules.
• minerva.losses: Loss functions.
• minerva.models: Models and Architectures for supervised and self-supervised learning. Check out the available models and SSL techniques at minerva.models README.md file.
• minerva.pipeline: Tools for training and evaluating models in a reproducible way.
• minerva.samplers: Data samplers to allow for more complex sampling strategies during data loading.
• minerva.transforms: Data transformations and augmentations.
• minerva.utils: Utility functions and classes.

Usage

For more information on how to use Minerva, check the documentation for example notebooks.

Testing

First, install the development dependencies.

With uv (recommended):

uv sync --extra dev

With pip:

pip install -e ".[dev]"

Unit Tests

With uv:

uv run pytest tests/

With pip (after activating your environment):

pytest tests/

Coverage

With uv:

uv run pytest --cov=minerva --cov=tests --cov-report=term --cov-report=html

With pip:

pytest --cov=minerva --cov=tests --cov-report=term --cov-report=html

License

This project is primarily licensed under the MIT License. See the LICENSE file for details. Some files were derived from thid-party libraries and retain their original license. We provide the licenses from the original repositories and included the appropriate copyright notices in the derived files' header. See the THID PARTY LICENSES file for details.

Contact

For any questions or concerns, please open an issue on our GitHub issue tracker.

Contribute

If you want to contribute to this project make sure to read our Code of Conduct and Contributing guide pages.

Acknowledgements

This project is maintained by Gabriel Gutierrez, Otávio Napoli, Fernando Gubitoso Marques, and Edson Borin.