ChallengerTeco

challengerteco is a Python library designed to streamline the process of training, evaluating, and comparing machine learning models in a distributed environment. It provides a flexible and configurable pipeline for data scientists and ML engineers to run model "challenger" competitions.

Description

This project encapsulates the end-to-end machine learning workflow, from data preparation to model evaluation. It is built to work with large datasets and can be executed on different platforms like Google Cloud Platform (GCP) or on-premise clusters.

The main goal of this library is to provide a standardized way to test different models and hyperparameters, making it easier to select the best-performing model for a given task.

Features

• Multi-platform support: Runs on Google Cloud Platform and on-premise environments.
• Multiple ML Models: Supports popular classification algorithms like:
  • Random Forest
  • Gradient Boosting
  • Logistic Regression
  • LightGBM
  • XGBoost
• Data Preprocessing: Includes steps for:
  • Target balancing (undersampling)
  • Feature scaling
  • Correlation analysis and removal
  • Data type casting
• Hyperparameter Tuning: Uses RandomizedSearchCV and GridSearchCV to find the best model parameters.
• Model Evaluation: Evaluates models using AUC and other metrics.
• Persistence: Saves trained models and evaluation results.

Classes

query_manager

This class is a wrapper for database interactions. It abstracts the boilerplate code needed to connect to and query different data sources.

• execute_query(query): Executes a SQL query.
• to_table(df, mode, nombre): Saves a DataFrame to a table.
• from_query(query): Creates a DataFrame from a SQL query.

Challenger

The main class of the library. It orchestrates the entire ML pipeline. It takes a large number of configuration parameters to control every aspect of the process.

The main method is EjecutarChallenger(), which runs the complete challenger process.

Installation

pip install challengerteco

Usage

The intended use is to create a JSON configuration file and instantiate the Challenger class from it.

config.json

{
    "BALANCEAR_TARGET": true,
    "ELIMINAR_CORRELACIONES": true,
    "CASTEAR_BIGINT": true,
    "REDONDEAR_DECIMALES": true,
    "CON_SCALER": true,
    "TIENE_TESTING": true,
    "CORRER_RF": true,
    "CORRER_GB": true,
    "CORRER_LR": true,
    "CORRER_LGBM": true,
    "CORRER_XGB": true,
    "CORRER_PRODUCTIVO": false,
    "CAMPO_CLAVE": "client_id",
    "TARGET": "purchase",
    "modelo": "propensity_to_buy",
    "ABT_VARIABLES": "var1, var2, var3",
    "ABT_TABLA": "my_project.my_dataset.abt_table",
    "TGT_TABLA": "my_project.my_dataset.target_table",
    "TGT_VARIABLES": "target_variable",
    "TGT_BALENCEO": 20,
    "DECIMALES_VARIABLES_NUMERICAS": 4,
    "COTA_CORRELACIONES": 0.9,
    "REGISTROS_X_PARTICION": 100000,
    "PORCENTAJE_TRAINING": 0.8,
    "GB_param_test": {},
    "LGBM_param_test": {},
    "XGB_param_test": {},
    "RF_param_test": {},
    "LR_param_test": {},
    "PERIODO": "202301",
    "PERIODO_TRAIN1": "202207",
    "PERIODO_TRAIN2": "202208",
    "PERIODO_TRAIN3": "202209",
    "PERIODO_TRAIN4": "202210",
    "PERIODO_TRAIN5": "202211",
    "PERIODO_TRAIN6": "202212",
    "PERIODO_TEST1": "202301",
    "PERIODO_TEST2": "202302",
    "PERIODO_TEST3": "202303",
    "MODELO_PRODUCTIVO": "RF",
    "MODELO_PRODUCTIVO_param_test": {},
    "GRABAR_BINARIOS": true,
    "Tabla_Performance_Modelos": "my_project.my_dataset.performance_table"
}

main.py

from challengerteco import Challenger
from pyspark.sql import SparkSession
# You will need to provide your own bq and spark objects
# bq = bigquery.Client()
# spark = SparkSession.builder.getOrCreate()

challenger = Challenger.instanciar_desde_json('config.json')
challenger.bq = bq
challenger.spark = spark
challenger.reset_query_manager()
challenger.EjecutarChallenger()

TODOs

The following TODOs were found in the code and could be addressed in future versions:

• Clean up library imports.
• Add assertions to validate input parameters.
• Improve error handling and logging.
• Refactor duplicated code in RandomForest, GradientBoosting, and LogisticRegression methods.
• Review memory pressure in pandas-based methods.
• Improve output formatting for better readability.
• Remove hardcoded values and unused variables.
• Update deprecated commands (hdfs dfs -copyFromLocal).
• Add docstrings to methods.

This README.md provides a good overview of the project. It explains what the project does, its main features, how to install and use it, and what can be improved.