upgini 0.10.0a35

Automated feature discovery & enrichment library for supervised machine learning on tabular data

Upgini : automated feature discovery & enrichment library for supervised machine learning on tabular data

Automatically find and enrich ML model with relevant external features from scraped data and public datasets to improve machine learning model accuracy

Live DEMO in Colab » | Upgini.com | Sign In | Slack Community

❔ Overview

Upgini is a Python library for an automated features search to boost accuracy of supervised ML models on tabular data. It enriches your dataset with intelligently crafted features from a broad range of curated data sources, including public datasets and scraped data. The search is conducted for any combination of public IDs contained in your tabular dataset: IP, date, etc. Only features that could improve the prediction power of your ML model are returned.
Motivation: for most supervised ML models external data & features boost accuracy significantly better than any hyperparameters tuning. But lack of automated and time-efficient search tools for external data blocks massive adoption of external features in ML pipelines.
We want radically simplify features search and delivery for ML pipelines to make external data a standard approach. Like a hyperparameter tuning for machine learning nowadays.

🚀 Awesome features

⭐️ Automatically find only features that give accuracy improvement for ML algorithm according to metrics: ROC AUC, RMSE, Accuracy. Not just correlated with target variable data or features, which 9 out of 10 cases gives zero accuracy improvement for production ML cases
⭐️ Calculate accuracy metrics and uplifts if you'll enrich your existing ML model with found external features, right in search results
⭐️ Check the stability of accuracy gain from external data on out-of-time intervals and verification datasets. Mitigate risks of unstable external data dependencies in ML pipelines
⭐️ Scikit-learn compatible interface for quick data integration with your existing ML pipelines
⭐️ Curated and updated data sources, including public datasets and scraped data
⭐️ Support for several search key types (including SHA256 hashed email, IPv4, phone, date/datetime), more to come...
⭐️ Supported supervised ML tasks:

• ☑️ binary classification
• ☑️ multiclass classification
• ☑️ regression
• 🔜 time series prediction
• 🔜 recommender system

🏁 Quick start with Kaggle example

🏎 Live demo with Kaggle competition data

Live demo notebook kaggle_example.ipynb inside your browser:

   

🐍 Install from PyPI

%pip install -Uq upgini catboost

🐳 Docker-way

Clone $ git clone https://github.com/upgini/upgini or download upgini git repo locally and follow steps below to build docker container 👇
Build docker image

• ... from cloned git repo:

cd upgini
docker build -t upgini .

• ...or directly from GitHub:

DOCKER_BUILDKIT=0 docker build -t upgini [e-mail]:upgini/upgini.git#main

Run docker image:

docker run -p 8888:8888 upgini

Open http://localhost:8888?token=<your_token_from_console_output> in your browser

Kaggle notebook

Jupyter notebook with a Kaggle example: kaggle_example.ipynb. The problem being solved is a Kaggle competition Store Item Demand Forecasting Challenge. The goal is to predict future sales of different goods in different stores based on a 5-year history of sales. The evaluation metric is SMAPE.

Competition dataset was split into train (2013-2016 year) and test (2017 year) parts. FeaturesEnricher was fitted on train part. And both datasets were enriched with external features. Finally, ML algorithm was fitted both of the initial and the enriched datasets to compare accuracy improvement. With a solid improvement of the evaluation metric achieved by the enriched ML model.

💻 How it works?

1. 🔑 Get access - API key

For full access beyond demo example, you'll need API key from User profile page https://profile.upgini.com
Pass API key via api_key parameter in FeaturesEnricher class constructor or export as environment variable:
... in python

import os
os.environ["UPGINI_API_KEY"] = "your_long_string_api_key_goes_here"

... in bash/zsh

export UPGINI_API_KEY = "your_long_string_api_key_goes_here"

2. 💡 Reuse existing labeled training datasets for search

To simplify things, you can reuse your existing labeled training datasets "as is" to initiate the search. Under the hood, we'll search for relevant data using:

• search keys from training dataset to match records from potential external data sources and features
• labels from training dataset to estimate relevancy of feature or dataset for your ML task and calculate metrics
• your features from training dataset to find external datasets and features only give accuracy improvement to your existing data and estimate accuracy uplift (optional)

Load training dataset into pandas dataframe and separate features' columns from label column:

import pandas as pd
# labeled training dataset - customer_churn_prediction_train.csv
train_df = pd.read_csv("customer_churn_prediction_train.csv")
train_ids_and_features = train_df.drop(columns="label")
train_label = train_df["label"]

3. 🔦 Choose at least one column as a search key

Search keys columns will be used to match records from all potential external data sources 👓. Define at least one search key with FeaturesEnricher class initialization.

from upgini import FeaturesEnricher, SearchKey
enricher = FeaturesEnricher (
    search_keys={"subscription_activation_date": SearchKey.DATE},
    keep_input=True )

✨ Search key types we support (more is coming!)

Our team works hard to introduce new search key types, currently we support:

Search Key Meaning Type	Description	Example
SearchKey.EMAIL	e-mail	[e-mail]
SearchKey.HEM	sha256(lowercase(email))	0e2dfefcddc929933dcec9a5c7db7b172482814e63c80b8460b36a791384e955
SearchKey.IP	IP address (version 4)	192.168.0.1
SearchKey.PHONE	phone number, E.164 standard	443451925138
SearchKey.DATE	date	2020-02-12  (ISO-8601 standard) 12.02.2020  (non standard notation)
SearchKey.DATETIME	datetime	2020-02-12 12:46:18 12:46:18 12.02.2020 unixtimestamp

⚠️ Requirements for search initialization dataset

We do dataset verification and cleaning under the hood, but still there are some requirements to follow:

• Pandas dataframe representation
• Correct label column types: integers or strings for binary and multiclass labels, floats for regression
• At least one column defined as a search key
• Min size after deduplication by search key column and NAs removal: 1000 records
• Max size after deduplication by search key column and NAs removal: 1 000 000 records

4. 🔍 Start your first data search!

The main abstraction you interact is FeaturesEnricher. FeaturesEnricher is a Scikit-learn compatible estimator, so you can easily add it into your existing ML pipelines. First, create instance of the FeaturesEnricher class. Once it created call

• fit to search relevant datasets & features
• than transform to enrich your dataset with features from search result

Let's try it out!

import pandas as pd
from upgini import FeaturesEnricher, SearchKey

# load labeled training dataset to initiate search
train_df = pd.read_csv("customer_churn_prediction_train.csv")
train_features = train_df.drop(columns="label")
train_target = train_df["label"]

# now we're going to create `FeaturesEnricher` class
# if you still didn't define UPGINI_API_KEY env variable - not a problem, you can do it via `api_key`
enricher = FeaturesEnricher(
    search_keys={"subscription_activation_date": SearchKey.DATE},
    keep_input=True,
    api_key="your_long_string_api_key_goes_here"
)

# everything is ready to fit! For 200к records fitting should take around 10 minutes,
# but don't worry - we'll send email notification. Accuracy metrics of trained model and uplifts
# will be shown automaticly
enricher.fit(train_ids_and_features, train_target)

That's all). We have fitted FeaturesEnricher and any pandas dataframe, with exactly the same data schema, can be enriched with features from search results. Use transform method, and let magic to do the rest 🪄

# load dataset for enrichment
test_df = pd.read_csv("test.csv")
test_ids_and_features = test_df.drop(columns="target")
# enrich it!
enriched_test_features = enricher.transform(test_ids_and_features)
enriched_test_features.head()

You can get more details about FeaturesEnricher in runtime using docstrings, for example, via help(FeaturesEnricher) or help(FeaturesEnricher.fit).

✅ Optional: find datasets and features only give accuracy gain to your existing data in the ML model

If you already have a trained ML model, based on internal features or other external data sources, you can specifically search new datasets & features only give accuracy gain "on top" of them.
Just leave all these existing features in the labeled training dataset and Upgini library automatically use them as a baseline ML model to calculate accuracy metric uplift. And won't return any features that might not give an accuracy gain to the existing ML model feature set.

✅ Optional: check stability of ML accuracy gain from search result datasets & features

You can validate data quality from your search result on out-of-time dataset using eval_set parameter. Let's do that:

# load train dataset
train_df = pd.read_csv("train.csv")
train_ids_and_features = train_df.drop(columns="target")
train_target = train_df["target"]

# load out-of-time validation dataset
eval_df = pd.read_csv("validation.csv")
eval_ids_and_features = eval_df.drop(columns="eval_target")
eval_target = eval_df["eval_target"]
# create FeaturesEnricher
enricher = FeaturesEnricher(
    search_keys={"registration_date": SearchKey.DATE},
    keep_input=True
)

# now we fit WITH eval_set parameter to calculate accuracy metrics on OOT dataset.
# the output will contain quality metrics for both the training data set and
# the eval set (validation OOT data set)
enricher.fit(
  train_ids_and_features,
  train_target,
  eval_set = [(eval_ids_and_features, eval_target)]
)

⚠️ Requirements for out-of-time dataset

• Same data schema as for search initialization dataset
• Pandas dataframe representation

🧹 Search dataset validation

We validate and clean search initialization dataset under the hood:
✂️ Check you search keys columns format
✂️ Check dataset for full row duplicates. If we find any, we remove duplicated rows and make a note on share of row duplicates
✂️ Check inconsistent labels - rows with the same record keys (not search keys!) but different labels, we remove them and make a note on share of row duplicates

🆙 Accuracy and uplift metrics calculations

We calculate all the accuracy metrics and uplifts for non-linear machine learning algorithms, like gradient boosting or neural networks. If your external data consumer is a linear ML algorithm (like log regression), you might notice different accuracy metrics after data enrichment.

💸 Why it's a paid service? Can I use it for free?

The short answer is Yes! We do have two options for that 🤓
Let us explain. This is a part-time project for our small team, but as you might know, search is a very infrastructure-intensive service. We pay infrastructure cost for every search request generated on the platform, as we mostly use serverless components under the hood. Both storage and compute.
To cover these run costs we introduce paid plans with a certain amount of search requests, which we hope will be affordable for most of the data scientists & developers in the community.

First option. Participate in beta testing

Now service is still in a beta stage, so registered beta testers will get an 80USD credits for 6 months. Feel free to start with the registration form 👉 here Please note that number of slots for beta testing is limited and we wont' be able to handle all the requests.

Second option. Share license-free data with community

If you have ANY data which you might consider as royalty and license-free (Open Data) and potentially valuable for supervised ML applications, we'll be happy to give free individual access in exchange for sharing this data with community.
Just upload your data sample right from Jupyter. We will check your data sharing proposal and get back to you ASAP:

import pandas as pd
from upgini import SearchKey
from upgini.ads import upload_user_ads
import os
os.environ["UPGINI_API_KEY"] = "your_long_string_api_key_goes_here"
#you can define custom search key which might not be supported yet, just use SearchKey.CUSTOM_KEY type
sample_df = pd.read_csv("path_to_data_sample_file")
upload_user_ads("test", sample_df, {
    "city": SearchKey.CUSTOM_KEY, "stats_date": SearchKey.DATE
})

🛠 Getting Help & Community

Requests and support, in preferred order

Please try to create bug reports that are:

• Reproducible. Include steps to reproduce the problem.
• Specific. Include as much detail as possible: which Python version, what environment, etc.
• Unique. Do not duplicate existing opened issues.
• Scoped to a Single Bug. One bug per report.

🧩 Contributing

We are a very small team and this is a part-time project for us, thus most probably we won't be able:

• implement ALL the data delivery and integration interfaces for most common ML stacks and frameworks
• implement ALL data verification and normalization capabilities for different types of search keys (we just started with current 4)

And we might need some help from community) So, we'll be happy about every pull request you open and issue you find to make this library more awesome. Please note that it might sometimes take us a while to get back to you. For major changes, please open an issue first to discuss what you would like to change

Developing

Some convenient ways to start contributing are:
⚙️ Visual Studio Code You can remotely open this repo in VS Code without cloning or automatically clone and open it inside a docker container.
⚙️ Gitpod You can use Gitpod to launch a fully functional development environment right in your browser.

🔗 Useful links

• Quick start guide
• Kaggle example notebook
• Project on PyPI
• Get API Key

^{😔 Found mistype or a bug in code snippet? Our bad! Please report it here.}