MELITE — Multi-Model Classifier Evaluator
Description
MELITE is a Python package and command-line tool for evaluating and comparing classifiers on numeric tabular datasets. It separates hyperparameter tuning from classifier evaluation, preserves the evidence used for selection, and exports the selected model as a reusable artifact for downstream inference.
MELITE operates at the tabular modeling level. Its learning algorithms consume
numeric feature matrices (X) and target labels (y), regardless of how those
features were produced. Inputs may therefore originate from fingerprints,
descriptors, dimensionality-reduction methods, clinical variables, experimental
measurements, industrial features, or other numeric representations.
Purpose
MELITE is designed to make classifier comparison and selection explicit, reproducible, and auditable. Its workflow separates stages that are often mixed together in small classification workflows:
- hyperparameter tuning;
- classifier evaluation;
- comparison and selection;
- final fitting on all available data;
- model export and inference.
This separation ensures that, within each outer cross-validation split, the data used to evaluate a tuned classifier are held out from the hyperparameter search that produced it, while preserving the evidence needed to understand how the competing classifiers performed.
Why Use MELITE?
- Controlled evaluation. Hyperparameter tuning is kept separate from the evidence used to compare classifiers.
- Evidence preservation. Aggregate and fold-level evaluation results are retained for every evaluated classifier, not only for the selected one.
- Explicit selection. Classifier selection follows a predefined criterion based on cross-validation evidence rather than an informal choice after training.
- Domain-agnostic inputs. MELITE works with numeric tabular data without assuming how the features were generated.
- Reusable artifacts. After selection, the chosen classifier can be fitted on all available data and saved as a model artifact for prediction.
- CLI and Python interfaces. MELITE can be used through its command-line workflow and through a focused public Python API.
What MELITE Does
| MELITE does | MELITE does not |
|---|---|
Evaluate multiple classifiers on prepared numeric X and y. |
Generate domain-specific features or descriptors. |
| Tune supported classifiers within the evaluation design. | Act as a general AutoML framework. |
| Preserve aggregate and fold-level evaluation evidence. | Generate PCA, UMAP, fingerprints, or other feature representations. |
| Select the best active classifier by mean outer-CV F1-macro. | Process raw domain-specific inputs. |
Fit and export the selected model as a .pkl artifact. |
Perform automatic feature engineering or feature selection. |
| Run inference from exported model artifacts. | Guarantee a stable 1.0 API yet. |
Evaluation Contract
For a registered dataset, MELITE follows the contract below:
Xis a two-dimensional numeric feature matrix andyprovides the target labels for the same samples.- Each active classifier is evaluated under the configured outer cross-validation design.
- For tunable classifiers, hyperparameter search occurs only within the training portion of each outer split.
- Evaluation evidence is obtained from the held-out folds of repeated stratified outer cross-validation.
- Mean outer-CV F1-macro is used to select the best active classifier for each dataset.
- Aggregate and per-fold evidence are preserved for every evaluated classifier.
- After selection, MELITE performs the final full-data fitting stage. For a
tunable classifier,
melite runfirst performs a final full-data hyperparameter search and records the resulting parameters inresults.csv; Stacking is fitted directly. melite exportreconstructs the selected classifier from the persisted result, fits it on all available data, and serializes the final model artifact. It performs no additional hyperparameter search, cross-validation, or classifier selection.- Smoke mode is intended for fast execution checks, not final classifier selection.
Installation
Package Users
Install MELITE in a supported Python environment:
python -m pip install melite
Verify the installation:
melite --version
Contributors and Developers
Clone the repository and install in editable mode with development dependencies:
git clone https://github.com/NanoBiostructuresRG/melite.git
cd melite
conda create -n melite_env python=3.11
conda activate melite_env
python -m pip install -e ".[dev]"
To build the documentation locally, install the docs extra as well:
python -m pip install -e ".[dev,docs]"
mkdocs serve
Quick Start
The bundled example creates a ready-to-run synthetic numeric CSV dataset and example configuration for a short evaluation workflow, so you can verify that MELITE is installed and working. Cloning the repository is not required.
melite example
melite run --smoke --config melite_example/config.toml
melite example creates ./melite_example/ in the current directory.
Next steps
- Prepare and use your own datasets.
- Configure MELITE.
- Run a full evaluation.
- Export a model and use the Python prediction API.
Workflow
CLI Workflow
The command-line interface provides the canonical end-to-end MELITE workflow:
- Register one or more numeric datasets in a TOML configuration file.
- Choose the active classifiers.
- Run
melite runto generate evaluation evidence and selected results. - Inspect
results.csv,evaluations.csv,evaluation_folds.csv,optimization_searches.csv,optimization_provenance.json, and the dataset-level F1-macro evidence figures. - Run
melite exportfor the selected result you want to preserve as a model artifact. - Use the exported
.pklartifact throughmelite.predict()for inference.
Python Workflow
The Python API is intentionally component-oriented. It exposes configuration, dataset loading, evaluation-evidence plotting, artifact-based prediction, and version metadata as public symbols.
MELITE does not expose the full evaluation orchestration as a stable high-level Python workflow API. For reproducible end-to-end execution, use the CLI and a version-controlled TOML configuration.
Supported Classifiers
MELITE currently supports six classifier keys:
| Key | Classifier | Tunable | Active by default |
|---|---|---|---|
svc |
Support Vector Classifier | Yes | Yes |
lr |
Logistic Regression | Yes | No |
gnb |
Gaussian Naive Bayes | Yes | No |
rf |
Random Forest | Yes | Yes |
xgb |
XGBoost | Yes | Yes |
stack |
Stacking classifier | No | No |
The first five keys are base classifier families. stack is a non-tunable,
opt-in composite classifier. MELITE supports this fixed set of six keys and
does not expose public registration of custom classifiers.
The default configuration is:
[classifiers]
active = ["svc", "rf", "xgb"]
The [classifiers].active list accepts any supported subset in evaluation
order, including a single classifier. For example, active = ["svc"],
active = ["lr"], active = ["gnb"], and active = ["stack"] each select
one supported classifier. The lr, gnb, and stack classifiers remain inactive
unless explicitly selected.
Every exported classifier artifact supports both predict() and
predict_proba(). SVC remains one classifier family: MELITE preserves direct
SVC class predictions and its decision function while supplying calibrated
probabilities internally for exported inference. Logistic Regression,
Gaussian Naive Bayes, Random Forest, and XGBoost use their native probability
support. Logistic Regression uses a StandardScaler -> LogisticRegression
pipeline and is tunable in C; Gaussian Naive Bayes uses GaussianNB directly
and is tunable in var_smoothing. Random Forest and XGBoost remain unscaled.
The opt-in stacking classifier combines SVC, Random Forest, and XGBoost as base
estimators with Logistic Regression as the final estimator and uses
probability-based base-estimator outputs.
Tunable classifiers use Optuna with its TPE sampler for hyperparameter
optimization. The public n_trials setting explicitly declares the budget per
search, with a normal default of 100 trials. Searches are sequential and seeded
from the canonical RANDOM_STATE, supporting reproducible optimization under
the same configuration and software environment.
Input Format
Dataset Registry
Datasets are registered under user-defined [datasets.<dataset_id>] entries.
For example:
[datasets.morgan_r2_2048]
path = "data/morgan_r2_2048.npz"
label_path = "raw/labels.npy"
family = "fingerprints"
method = "Morgan"
variant = "r2_2048"
description = "Morgan fingerprints, radius 2, 2048 bits"
Here, family is optional dataset metadata used to describe the feature
representation. It is unrelated to the classifier selected or evaluated by
MELITE.
Each dataset must define path and the label source required by its format:
label_column for CSV datasets or label_path for NPZ datasets.
Optional metadata fields such
as family, method, variant, level, and description are preserved for
traceability and do not trigger dataset-specific execution logic.
The legacy [benchmark] configuration section remains supported for backward
compatibility. New configurations should use the dataset registry.
Array Requirements
A registered .npz dataset must contain an X array. MELITE validates that:
Xis two-dimensional;Xis numeric;- the number of rows in
Xmatches the number of labels iny; - an embedded
yarray, when present, matches the configured label vector.
A typical input layout is:
raw/
└── labels.npy
data/
├── morgan_r2_2048.npz
├── rdkit_descriptors.npz
├── PCA85.npz
└── UMAP90.npz
The filenames and feature families are examples only. MELITE does not require PCA, UMAP, fingerprints, descriptors, or any other specific feature-generation method.
Main Outputs
A standard MELITE workflow produces evaluation artifacts and, when requested, a final exported model:
output/
├── results.txt
├── results.csv
├── evaluations.csv
├── evaluation_folds.csv
├── optimization_searches.csv
├── optimization_provenance.json
├── figures/
│ └── evaluation_f1_macro_<dataset>.png
└── Model_<classifier>_<dataset>.pkl
The artifacts have distinct roles:
results.txt— human-readable summary of the selected results.results.csv— selected classifier result for each dataset and the persisted parameter source used bymelite export.evaluations.csv— aggregate evaluation evidence for every active classifier.evaluation_folds.csv— outer-CV evidence for every dataset, classifier, and outer split.optimization_searches.csv— one row per completed outer or final optimization search.optimization_provenance.json— the effective optimization and evaluation contract for the run.figures/evaluation_f1_macro_<dataset>.png— visualization of the outer-CV F1-macro evidence used for classifier selection.Model_<classifier>_<dataset>.pkl— final full-data fitted model created bymelite export.
See the Output Data Contract for the column-level schemas and semantics.
The evaluation figure is generated from already-computed outer-CV evidence. It does not trigger additional fitting, tuning, cross-validation, or selection.
Configuration
MELITE uses TOML configuration files to keep execution choices explicit and reproducible. Configuration controls, among other settings:
- registered datasets and their metadata;
- active classifiers;
- random state;
- inner and outer cross-validation settings;
- optimization trial budget (
n_trials); - input and output paths.
Use --config to supply a project-specific configuration:
melite run --config my_config.toml
melite export --config my_config.toml --row 0
A complete configuration template is available at
examples/melite_run.toml.
Smoke mode can be requested independently from the configuration:
melite run --smoke --config my_config.toml
See the full configuration reference in the project documentation.
Development
Running Tests
Run the test suite:
python -m pytest tests -q
Build the documentation in strict mode:
mkdocs build --strict
Build and check the distributions:
python -m build --no-isolation
python -m twine check dist/*
Run the installed-wheel smoke test:
python scripts/smoke_install_wheel.py
Contributing
Contributions are welcome. Please open an issue before submitting a pull request. Follow the existing code style: NumPy-style docstrings, type hints, and SPDX license headers in all source files.
See CONTRIBUTING.md for full guidelines, including the development setup and the pull request target branch. Please also read our Code of Conduct.
Documentation
The full documentation is published at:
https://nanobiostructuresrg.github.io/melite/
Citation
If you use MELITE in your research, please cite it using the metadata in CITATION.cff or the format below:
Contreras-Torres, F. F., & Murrieta, A. C. (2026). MELITE — Multi-Model Classifier Evaluator. Zenodo. https://doi.org/10.5281/zenodo.20382752
Authors
- Flavio F. Contreras-Torres — Tecnológico de Monterrey
- Ana C. Murrieta — Tecnológico de Monterrey
License
This project is licensed under the terms of the
GNU Lesser General Public License v3.0 or later.
SPDX identifier: LGPL-3.0-or-later.
Metadata
Release files for melite 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| melite-0.3.1.tar.gz | 116.6 kB | With provenance | Details for melite-0.3.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| melite-0.3.1-py3-none-any.whl | Python 3 | none | any | Details for melite-0.3.1-py3-none-any.whl |
Total release size: 190.5 kB
Release files / melite-0.3.1.tar.gz
| Download URL | melite-0.3.1.tar.gz |
|---|---|
| Size | 116.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
071e64ac7c9067dffefd6736fa89eb249ea5beb333f4a65e444a687d7a70e02c
|
|
BLAKE2b-256 checksum How to use checksums |
4f902eba523c240b4f625b23c2d441f9c3aea95e028dc0a1236ef5d8238abb06
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logRelease files / melite-0.3.1-py3-none-any.whl
| Download URL | melite-0.3.1-py3-none-any.whl |
|---|---|
| Size | 73.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0778762e9092eab04764e2edac1cdae5b5500b77b56e1686fd80a8c7943c7a09
|
|
BLAKE2b-256 checksum How to use checksums |
5a9efd93251d0b436bb4ac393c5e49aeaa307b5d5bc30d1563d396ef5ce9b6b3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logSecurity for melite 0.3.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for melite-0.3.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.