improvelib 0.1.0.dev20250107

Open-source package for model standardization and comparison in Python

improvelib

improvelib is a comprehensive toolset designed to enable researchers to consistently compare the performance of new AI models against established benchmarks. It ensures that advancements in model accuracy, efficiency, and robustness are measured and reported in a standardized, reproducible way across cancer research and other fields. As an open-source project, we invite contributions from the community to promote collaboration, share best practices, introduce new metrics, and continuously enhance improvelib. The ultimate goal of improvelib is to be user-friendly and accessible, making it routine for researchers to rigorously and comprehensively compare new models with prior models.

Installation

pip install improvelib

improvelib uses Python >= 3.6 and requires the following dependencies:

• pandas
• requests
• tqdm
• typing_extensions
• pyyaml
• scikit-learn

Documentation

For a detailed guide on how to use the improvelib library, including a tutorial using an example model, LightGBM, see https://jdacs4c-improve.github.io/docs.

Examples

Two repositories demonstrating the use of the improvelib library for drug response prediction:

• https://github.com/JDACS4C-IMPROVE/GraphDRP -- GraphDRP (deep learning model based on graph neural network)
• https://github.com/JDACS4C-IMPROVE/LGBM -- LightGBM model