preprocessy 1.0.0a0

Data Preprocessing library that provides customizable pipelines.

Preprocessy

Preprocessy is a library that provides data preprocessing pipelines for machine learning. It bundles all the common preprocessing steps that are performed on the data to prepare it for machine learning models. It aims to do so in a manner that is independent of the source and type of dataset. Hence, it provides a set of functions that have been generalised to different types of data.

The pipelines themselves are composed of these functions and flexible so that the users can customise them by adding their processing functions or removing pipeline functions according to their needs. The pipelines thus provide an abstract and high-level interface to the users.

Pipeline Structure

The pipelines are divided into 3 logical stages -

Stage 1 - Pipeline Input

Input datasets with the following extensions are supported - .csv, .tsv, .xls, .xlsx, .xlsm, .xlsb, .odf, .ods, .odt

Stage 2 - Processing

This is the major part of the pipeline consisting of processing functions. The following functions are provided out of the box as individual functions as well as a part of the pipelines -

• Handling Null Values
• Handling Outliers
• Normalisation and Scaling
• Label Encoding
• Correlation and Feature Extraction
• Training and Test set splitting

Stage 3 - Pipeline Output

The output consists of processed dataset and pipeline parameters depending on the verbosity required.

Project Structure

.
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── datasets
├── evaluations
├── preprocessy
├── requirements.txt
├── requirements_dev.txt
├── setup.py
├── tests
└── venv

5 directories, 6 files

• preprocessy - Contains the different pipeline and function classes

• tests - Contains all the unit and integration tests

• datasets - Contains sample datasets for development purposes

• evaluations - Contains jupyter notebooks with example implementations and performance measurements

Requirements

pandas
scikit-learn # required for feature selection

For development requirements see Contributing Guidelines

Contributing

Please read our Contributing Guide before submitting a Pull Request to the project.

Support

Feel free to contact any of the maintainers. We're happy to help!

Roadmap

Check out our roadmap to stay informed of the latest features released and the upcoming ones. Feel free to give us your insights!

Documentation

Currently, documentation is under development. All contributions are welcome! Please see our Contributing Guide.

License

See the LICENSE file for licensing information.