proxiflow 0.1.3

Data Preprocessing flow tool in python

ProxiFlow

ProxiFlow is a data preparation tool for machine learning that performs data cleaning, normalization, and feature engineering.

Documentation

Read the full documentation here.

Usage

To use ProxiFlow, install it via [pip]{.title-ref} (from test PyPi):

pip install proxiflow

You can then call it from the command line:

proxiflow --config-file myconfig.yaml --input-file mydata.csv --output-file cleaned_data.csv

Here's an example of a YAML configuration file:

data_cleaning:
  remove_duplicates: True
  handle_missing_values:
    drop: True

data_normalization: # mandatory
  min_max: #mandatory but values are not mandatory. It can be left empty
    # Specify columns:
    - Age # not mandatory
  z_score:
    - Price 
  log:
    - Floors

feature_engineering:
  ...

The above configuration specifies that duplicate rows should be removed and missing values should be dropped.

API

ProxiFlow can also be used as a Python library. Here's an example:

import polars as pl
from proxiflow.config import Config
from proxiflow.core import Cleaner

# Load the data
df = pl.read_csv("mydata.csv")

# Load the configuration
config = Config("myconfig.yaml")

# Preprocess the data
dfl = Cleaner(config)
cleaned_df = dfl.clean_data(df)

# Write the output data
cleaned_df.write_csv("cleaned_data.csv")

TODO

• [x] Data cleaning
• [x] Data normalization
• [ ] Feature engineering

Note: only data cleaning is currently implemented; data normalization and feature engineering are TODO features.