increment

Analyze experiments directly from your warehouse or a dataframe.

Estimate treatment effects, quantify uncertainty, and inform decisions using data that already exists. Define warehouse analyses in version-controlled YAML, or start directly from a dataframe. Increment does not assign traffic, ship feature flags, or maintain a separate event store.

Quickstart · Warehouse analysis · Documentation · Statistical limitations

What you can do

• Analyze experiments — estimate effects, reduce variance with CUPED, apply multiple-comparison corrections, and check sample-ratio mismatch.
• Bayesian inference — combine prior beliefs with experiment data to estimate effects and probabilities of benefit.
• Plan and monitor — solve for sample size, power, and minimum detectable effect; use sequential methods for repeated looks under their stated assumptions.
• Estimate causal effects — use IPTW, DML, or AIPW for observational treatments from dataframes, or analyze encouragement designs and switchback contrasts.
• Explore and report — inspect breakouts, trends, and heterogeneous treatment effects; produce readout tables, dashboards, and sitewide-impact estimates.

Not every method works with every metric or design. The method guide helps you choose; the compatibility matrix shows supported combinations.

Quickstart

Requires Python 3.12–3.14. Install the pre-release package and Polars for this example:

pip install --pre increment polars

Start with one row per randomization unit. This example generates a small control/treatment dataset and estimates the relative change in mean revenue:

from random import Random

import polars as pl
from increment import Analysis

rng = Random(0)
df = pl.DataFrame(
    {
        "user_id": [f"u{i}" for i in range(200)],
        "variant": ["control"] * 100 + ["treatment"] * 100,
        "revenue": [rng.gauss(20 + (i >= 100), 5) for i in range(200)],
    }
)

results = Analysis.from_unit_summary(
    df,
    unit="user_id",
    group="variant",
    control="control",
    metrics={"revenue": "mean"},
).run()

for result in results:
    print(f"{result.metric}: lift={result.lift.value:+.1%}")

revenue: lift=+7.5%

Replace the generated data with your own unit-level dataframe. No YAML, warehouse connection, or DuckDB is required. Other Narwhals-supported eager dataframes work as well. For unit-by-day data, use Analysis.from_unit_panel.

The dataframe quickstart covers metric declarations, missing values, analysis plans, uncertainty intervals, and monitoring.

Connect your warehouse

Declare sources, exposures, metrics, and experiments in YAML. Bind the definitions to an Ibis connection to inspect the generated SQL and run an analysis:

import ibis
from increment import Analysis

con = ibis.snowflake.connect(...)
analysis = Analysis.from_definitions("checkout_redesign", "definitions/", con)

print(analysis.summary_sql()["revenue"])
results = analysis.run()

Install your warehouse's Ibis backend separately—for example, pip install 'ibis-framework[snowflake]'. The core package bundles no database driver. DuckDB, PostgreSQL, Snowflake, and BigQuery have execution test suites; coverage is not a promise that every method works on every backend.

See the runnable warehouse example and data-model guide for YAML setup, query inspection, portable unit-day artifacts, and moments exports.

Optional extras

Explore the guides

The examples/ directory contains runnable marimo notebooks for warehouse and dataframe analyses, power, CUPED, causal inference, and reporting. See the examples guide to choose one.

Scope and statistical assumptions

Increment analyzes randomized experiments from dataframes or SQL warehouses; observational treatments are supported from dataframes. It estimates effects—it does not make an observational comparison causal without identification assumptions.

Exact and asymptotic methods have different guarantees. Repeated looks require a compatible sequential procedure to claim frequentist error control; Bayesian posterior probabilities do not automatically provide that guarantee. Unsupported combinations surface coded errors or warnings; partial results may be returned when only some requested cells are unsupported.

Read the statistical limitations, compatibility matrix, and backend verification notes before choosing an analysis for production use.

Contributing and license

See CONTRIBUTING.md for development setup and contribution requirements. Contributions require a signed Contributor License Agreement.

Apache License 2.0 — see LICENSE and NOTICE. Copyright 2026 Kyle Caron.