sparpartner

A deterministic stratified sampler for train/test prep. It doesn't split your data randomly, it deliberately finds the rows that most resemble a benchmark case, so you can hold those out as a genuine test set and train on everything else.

Why this exists

A random train/test split assumes the test set should look statistically like the train set. That's the wrong question if what you actually want to know is: does the model generalize past one specific profile, or did it just memorize the neighborhood around it?

sparpartner answers that by ranking every row in your data by how closely it resembles a benchmark ("bench_marks") you define, then sorting closest-to-benchmark first. You then slice the sorted frame yourself:

• Lookalikes as test, the rest as train: the rows most like the benchmark are always at the top of the sorted result, so head() gives you the test set and tail() (or everything past your cut) gives you the train set. This is the harder, more honest check: it tells you whether the model actually learned something general, or only performs well near cases it's already seen a lot of.
• Just the lookalikes, nothing else: pass sparring_n and skip the manual slice entirely. sample() hands you back only the top sparring_n rows, already ranked, plus a report on exactly that set (see the sparring report section below).

sparpartner only produces the ranking (and, optionally, the top-N slice). The actual train/test cut beyond that is a plain slice on your side (see the usage examples below).

Where the idea comes from

A fighter in camp doesn't spar with whoever's free in the gym, they specifically look for a sparring partner who moves, reaches, and hits like the opponent they're about to face. Training against a random partner tells you nothing about how you'll actually do; training against someone who resembles the real threat does. sparpartner applies that same logic to a model: instead of a random holdout, it finds the rows that resemble the toughest, most relevant "opponent" profile and holds those back as the real test, so what's left to train on is everything unlike that opponent, and the test genuinely checks whether the model can handle the match it's actually walking into.

How the scoring works

You give it:

• df: your data. Any column names are fine, including ones starting with spar; sample() never touches or overwrites your own columns (see Validation).
• bench_marks: a dict of {column_name: benchmark_value}, one entry per signal you care about
• custom_weights: a dict of {column_name: weight}, saying which columns matter and how much. Insertion order is preserved and drives both the show_progress readout order and, combined with descending weight, the tie-break cascade order (see below).

For each weighted column, sparpartner auto-detects the column's type and scores every row's distance to the benchmark on a 0–1 scale (1.0 = exact match, 0.0 = as far as possible):

Date detection is automatic. A column is only treated as a date if its values look date-shaped (contain a separator like -, /, . or a recognizable month name) and parse successfully at least 98% of the time. Bare numeric-looking strings (e.g. "12345") never even reach the date-parsing attempt, and object-dtype columns of digit strings fall through to the exact-match string path instead of being mistaken for numbers. Only a real numeric dtype gets the numeric path.

Each column's 0–1 score is multiplied by its weight and summed into one raw score per row. That raw sum is divided by the total weight to get each row's overall match score, but only if the total weight is > 0. In that normal case the match score always lands in the 0–1 range, however many signals or weights you used. If the weights sum to <= 0, normalization is skipped entirely and the raw, unnormalized weighted sum is used instead (not guaranteed to fall in 0–1).

All of this (the per-signal 0–1 scores and the per-row overall match score) is working state sample() uses internally to sort, tie-break, and slice. None of it is added as columns to the df you get back. The df you receive always contains only your original columns, reordered (and sliced, if sparring_n is set). Score info comes back separately, as aggregates in sparring_report , see The sparring report.

The result is always sorted by match score descending, the row closest to the benchmark is always first, unless you pass best_first=True, see Row order (best_first) below.

Tie-breaking

Rows that land on the exact same match score aren't left to random or arbitrary order. Ties are broken by the per-signal score of the highest-weight signal first (higher wins), then the next-highest, cascading down the weight-sorted signal list until the tie resolves. Signals that share the same weight are compared in the order they appear in custom_weights (dict insertion order). Only if every signal is exhausted and rows are still tied does it fall back to pandas' stable sort (original row order).

Row order (best_first)

By default (best_first=False), the returned df is sorted with the closest match to the benchmark first, same as always. Pass best_first=True and, as the very last step before returning, sample() flips that same set of rows so the worst-of-selection is first and the best-of-selection is last.

This only changes presentation order. It never changes which rows get selected (e.g. via sparring_n), never touches scoring or tie-breaking, and sparring_report is identical either way, since it's an order-independent aggregate.

It exists to replace a manual post-hoc flip like:

result = result.sort_index(ascending=False).reset_index(drop=True)

which is fragile against how sample()'s own indexing/reset works. Use best_first=True instead:

result, report = sample(df, bench_marks, custom_weights, best_first=True)

The sparring report

sample() doesn't just return the ranked frame, it returns a (df, sparring_report) tuple. sparring_report is a flat dict summarizing match quality, with one spar_<name> key per signal plus spar_min_score / spar_max_score / spar_mean_score, e.g.:

{
    "spar_age": 55.0, "spar_income": 57.5,   # one key per signal, avg score
    "spar_min_score": 0.0,
    "spar_max_score": 91.93,
    "spar_mean_score": 56.45,
}

Every value is on a 0–100 scale (100 = perfect match to benchmark) and rounded to 2 decimal places. This is the only place score information comes back to you: individual per-row scores are never returned, only these aggregates.

A signal with weight=0 is excluded from ranking/sorting entirely (it can never move the match score or break a tie), but it still gets its own spar_<name> entry in sparring_report, so you can track how close rows are on a signal without letting that signal influence which rows are considered "closest".

By default (sparring_n=None) both the returned df and the report cover all rows. Pass an int and sample() slices the sorted result down to just the top sparring_n rows (the ones closest to the benchmark) before anything else happens. That slice is what you get back as df, and it's also exactly what sparring_report and the score distribution are computed on. There's no separate "full set" kept around once sparring_n is set; if you need the rest of the rows too (e.g. to build the train set), take them from your original df yourself, or call sample() again with sparring_n=None.

Progress readout (show_progress)

When show_progress=True, the sparring report section of the printed readout marks each signal's average score, and the min / max / mean of the score distribution, with a traffic-light emoji:

• 🔴 avg score in the bottom third (0–33.3)
• 🟡 avg score in the middle third (33.3–66.7)
• 🟢 avg score in the top third (66.7–100)

    age                  avg score= 64.33  🟡
    signup_date          avg score= 38.00  🟡
    country              avg score= 50.00  🟡

  SCORE DISTRIBUTION
  ------------------
    spar_min_score       =   6.67  🔴
    spar_max_score       =  82.00  🟢
    spar_mean_score      =  50.78  🟡

This is purely a print-time visual, it doesn't change anything about sparring_report's actual values.

Usage

Sample usage

import pandas as pd
from sparpartner import sample

df = pd.DataFrame({
    "id": [1, 2, 3, 4, 5],
    "age": [25, 30, 47, 52, 33],
    "signup_date": ["2023-01-15", "2023-03-02", "2022-11-20", "2023-01-10", "2023-06-01"],
    "country": ["US", "US", "CA", "US", "MX"],
})

bench_marks = {
    "age": 30,
    "signup_date": "2023-01-01",
    "country": "US",
}

custom_weights = {
    "age": 2,
    "signup_date": 1,
    "country": 1,
}

result, sparring_report = sample(
    df,
    bench_marks,
    custom_weights,
    sparring_n=None,      # None = every row scored, sorted, and returned
    drop_nan=True,
    show_progress=True,   # prints the full scoring breakdown
    best_first=False,     # False (default) = best match first; True = worst-of-selection first
)

print(result)
print(sparring_report)

age=30, signup_date="2023-01-01", and country="US" closely match row id=1 (age 25, close date, US), so that row lands at or near the top of the sorted output (or the bottom, if best_first=True).

# --- post-sample: turn the ranking into an actual train/test split ---
# Result is always sorted closest-to-benchmark first (default best_first=False).
ranked, _ = sample(df, bench_marks, custom_weights)

# Slice however large you want the test set to be, e.g. the top 30%:
cut = int(len(ranked) * 0.3)
test = ranked.iloc[:cut]    # lookalikes, the harder, honest test set
train = ranked.iloc[cut:]   # everything unlike the benchmark

# Or skip the manual slice and get the test set directly:
test_only, report = sample(df, bench_marks, custom_weights, sparring_n=cut)

Parameters

Returns

sample() returns a (df, sparring_report) tuple, not just a DataFrame. The df always contains only your original columns (reordered/sliced), no score columns are ever attached to it. See The sparring report for how score information comes back to you instead.

Validation

Input validation runs upfront, before any scoring starts, in two passes: general parameter checks, then custom_weights checks.

Raises TypeError if:

• df isn't a pandas DataFrame
• bench_marks isn't a dict
• custom_weights isn't a dict
• drop_nan, show_progress, or best_first isn't a bool
• sparring_n isn't an int or None (bools are rejected too)

Raises ValueError if:

• df has no rows
• custom_weights is an empty dict
• sparring_n isn't a positive integer
• a custom_weights key isn't a string, or doesn't match a column in df
• a custom_weights key has no matching entry in bench_marks
• a custom_weights key is literally named "score". sample() keeps its own overall-total working score internally, and a signal named "score" would generate the exact same internal name, corrupting that total instead of just shadowing a per-signal value. Rename that column in df (and its entries in bench_marks/custom_weights) before calling sample(). Names that merely contain "score", like test_score or score_pct, are unaffected, only an exact match on "score" collides
• a weight isn't numeric (bools are rejected too, a bool is technically an int in Python but was never meant as a weight)
• a weight is NaN, inf, or -inf

Note on duplicate signal names: since custom_weights is now a dict, keys are inherently unique, so a repeated column name can no longer be passed in the first place, Python itself resolves a repeated key in a dict literal (keeping only the last value) before sample() ever sees it. There's nothing left for validation to catch here.

A couple of things worth knowing

• Your own column names are unrestricted: sample() computes its working scores under internally-generated names that can't collide with anything you'd realistically name a column, and those working columns are always dropped before the df is returned. You can freely have your own columns named spar_score, spar_age, or anything else, sample() won't touch, rename, or overwrite them.
• A signal literally named "score": still rejected, the one case where a signal name itself (not a pre-existing df column) would collide with sample()'s own internal total. See Validation above.
• Object-dtype numeric strings: a column of strings like "100", "200" (object dtype, no separator) is scored as an exact-match string column, not auto-converted to numeric. Only genuine numeric dtypes (int, float) get the numeric distance path.
• Positional calls: if you call sample() positionally beyond custom_weights, the parameter order is sparring_n, drop_nan, show_progress, best_first. Use keyword arguments for anything past the first three positional params to avoid ambiguity.