Scikit-transformers : Scikit-learn + Custom transformers

About

scikit-transformers is a very usefull package to enable and provide custom transformers such as LogColumnTransformer, BoolColumnTransformers and others fancy transformers.

It was created to provide a simple way to use custom transformers in scikit-learn pipelines, and allow to use them in a scikit-learn model, using GridSearchCV for testing and tuning hyperparameters.

The starting point was to provide a simple LogColumnTransformer, which is a simple wrapper around the numpy log function, making possible to use a skew threshold to apply the log transformation only on columns with a skew superior to a given threshold.

With scikit-transformers, it is now possible to use this LogColumnTransformer in transformer in a GridSearchCV using a skew threshold as hyperparameter to find what columns are good to log or not.

LogColumnTransformer is one of the many transformers implemented in scikit-transformers.

Installation

Using regular pip and venv tools :

python3 -m venv .venv
source .venv/bin/activate
pip install scikit-transformers

Usage

For a very basic usage :

import pandas as pd

from sktransf.trasnformer import LogColumnTransformer

df = pd.DataFrame(
    { "a": range(10),
      "b": range(10)
    }
)

logger = LogColumnTransformer()
logger.fit_transform(df)
df_transf = logger.transform(df)

Using common transformers :

import pandas as pd

from sktransf.transformer import LogColumnTransformer, BoolColumnTransformer
from sktransf.selector import DropUniqueColumnSelector

df = pd.DataFrame(
    { "a": range(10),
      "b": range(10)
    }
)

df_bool = BoolColumnTransformer().fit_transform(df)
df_unique = DropUniqueColumnTransformer().fit_transform(df)
df_logged = LogColumnTransformer().fit_transform(df)

Using a pipeline with a scikit-learn model :

import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

from sktransf.transformer import LogColumnTransformer, BoolColumnTransformer
from sktransf.selector import DropUniqueColumnSelector

pipe = Pipeline([
    ('bool', BoolColumnTransformer()),
    ('unique', DropUniqueColumnTransformer()),
    ('log', LogColumnTransformer()),
    ('model', LinearRegression())
])

X = pd.DataFrame(
    { "a": range(10),
      "b": range(10)
    }
)

y = range(10)

pipe.fit(X, y)

y_pred = pipe.predict(X)

Documentation

For more specific information, please refer to the notebooks:

• Transformers :
  • LogColumnTransformer notebook
  • BoolColumnTransformer notebook
• Selectors :
  • DropUniqueColumnSelector notebook
  • DropSkuColumnSelector notebook
• Pipelines :
  • Pipelines notebook

A complete documentation is be available on the github page.

Changelog, Releases and Roadmap

Please refer to the changelog page for more information.

Contributing

Pull requests are welcome.

For major changes, please open an issue first to discuss what you would like to change.

For more information, please refer to the contributing page.

License

GPLv3