Skip to main content

upliftlab. Same budget, models race. Final standings of the Criteo example: Duality R-Learner 786, R-Learner 783, X-Learner 759, T-Learner 757, DML 699, Random 459 incremental visits. Photo finish: ranks 1 to 5 are within the 95% CI of each other, and all beat Random.

Find the uplift model that buys the most incremental outcome for the same budget, before it goes live.

Live demo PyPI Tests Built on EconML Python 3.10+ MIT License

Run it in the browser →  ·  Install  ·  한국어

The standings above are a real run: the Criteo ad experiment (199,837 users), a 40% budget, 8 cycles, EconML models. The site replays it cycle by cycle.

Why

An offline metric such as Qini scores one ranking on one holdout. In production a model picks who to treat, the effect is measured, the model retrains on what it saw, and the budget runs out at some point. upliftlab runs that loop for every model on the same users and tells you which one bought the most incremental outcome, and whether the gap is real.

In a synthetic coupon market where cost differs by user, the Qini winner bought on average 17% less incremental outcome than the loop winner for the same budget, in all 10 seeds; AUCC, which accounts for cost, picked the loop winner every time. On Criteo the winner changed from seed to seed, and the only clear gap was to Random. The experiment and how to rerun it (Korean).

  • The production loop, cycle by cycle. Each model selects its targets under the budget, the targets are split into treatment and control to measure the effect, and every model retrains before the next cycle.
  • Same conditions for every model. Same users, same budget, same random Explore group. The only difference between models is who they chose.
  • Honest about ties. The gap to the leader comes with a 95% CI. When ranks can't be told apart, the result says it is a photo finish.
  • Your data stays with you. The web version runs in your browser (Pyodide), the local version in your Python. Shared reports hold aggregates only.

Quick start

In the browser, nothing to install. Open upliftlab.dev and press Start. The Criteo data, six built-in models and a 40% budget are preselected.

Locally, with EconML. The same screen, served from your own Python. Use large files, the libraries you already have, and your GPU.

pip install upliftlab
upliftlab                # opens http://127.0.0.1:4180

With uv, uvx upliftlab runs it without installing. The first run downloads the Criteo sample (2.3 MB) once.

Without the screen. Compare models on your own CSV and get the standings in the terminal:

upliftlab run --data my_experiment.csv --algorithm my_model.py

Reading the result

The result screen: a photo finish, but far ahead of the baseline. +71% incremental visits over Random, the gap to the leader with 95% CIs, and the standings.

  • The headline is the conclusion. If the runner-up is within the 95% CI of the leader it reads 결승선은 박빙 (photo finish), and it says whether the gap to the baseline (Random) is real.
  • Over Random: how much more incremental outcome the leader bought than random targeting with the same budget.
  • Gap to the leader, 95% CI: grey when the interval crosses zero, blue when the model is clearly behind.
  • Below: the ranking table, what the leader's targets look like, per-cycle metrics, a row-level log, and the Cost and Qini curves. Offline AUCC, Qini (with a 95% CI) and uplift@k score each served model on that cycle's Explore group, a random holdout it never trained on.

Built-in models

Model EconML What it does
T-Learner metalearners.TLearner Difference between treatment and control outcome models
X-Learner metalearners.XLearner Imputed effects weighted by the propensity
R-Learner dml.NonParamDML Non-parametric CATE from residuals
DML dml.LinearDML Linear CATE from residuals
Duality R-Learner dml.NonParamDML × 2 CATE of outcome and of cost together, with λ picked on validation
Random NumPy The baseline

Each one is a single class Model file you can open and edit on the screen.

Bring your own model

One Python file that doesn't import upliftlab. Use scikit-learn, LightGBM or PyTorch as usual.

class Model:
    def fit(self, train, validation, context):  # train.X, train.t, train.y, train.cost
        ...
    def predict(self, X):                        # one score per user; higher scores are targeted first
        ...
Data format · a randomized experiment log with at least 1,000 rows

Column names are free; the screen guesses the roles and you can change them.

Role Example Values
Treatment treatment 0 = control, 1 = treated. Several treatments: 1, 2, 3…
Outcome retained What you want more of: conversion, purchase, retention
Cost (optional) coupon_cost What the treatment cost. Without it every treatment costs 1
Covariates visit_freq, … User features measured before treatment

The default data is the Criteo uplift dataset: a 100k treated + 100k control sample (199,837 users, outcome = site visit) of the original 13.98M.

MIT. The Criteo sample is CC BY-NC-SA 4.0, non-commercial use only; it is not inside the pip package and is downloaded from this repository on first use. Fonts: Pretendard, Archivo and Geist Mono under the SIL Open Font License (web/fonts/LICENSE); editor bundle: web/vendor/LICENSES.txt.

Metadata

Release files for upliftlab 0.26.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for upliftlab 0.26.6
File Size Uploaded Provenance Actions
upliftlab-0.26.6.tar.gz 3.6 MB With provenance Details for upliftlab-0.26.6.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for upliftlab 0.26.6
File Interpreter ABI Platform Actions
upliftlab-0.26.6-py3-none-any.whl Python 3 none any Details for upliftlab-0.26.6-py3-none-any.whl

Total release size: 7.3 MB

Release files / upliftlab-0.26.6.tar.gz

Download URL upliftlab-0.26.6.tar.gz
Size 3.6 MB
Tags Source
SHA-256 checksum
How to use checksums
09d05d151de98c8292c83ece504b66585e2851c455b011bfdf0a2abe1d318cd4
BLAKE2b-256 checksum
How to use checksums
d38b3f15f669979fe0bedd37067024eb6a32a22a51165ee68af93e55e4d5c9ba
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Release files / upliftlab-0.26.6-py3-none-any.whl

Download URL upliftlab-0.26.6-py3-none-any.whl
Size 3.6 MB
Tags Python 3
SHA-256 checksum
How to use checksums
7a26b1b0405169c15ba15ed7737ff6820ba83fe996189b0c279bfb897b2aaee5
BLAKE2b-256 checksum
How to use checksums
903f0a5dfc42ba398f1aedf92e6d9897dae9405fe301faa14c3b9c5c9718a4b4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

Security for upliftlab 0.26.6

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for upliftlab-0.26.6.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page