verosynthea-validator 0.2.0

Fairness testing for ML models using Australian demographic data

verosynthea-validator

Quick Start: Test a US-trained Model on Australian Data

This is a real-world fairness scenario. We take a standard classifier trained on US Census data and test how it performs on Australian demographics.

pip install verosynthea-validator

from verosynthea_validator import FairnessReport
from verosynthea_validator.demos import (
    load_us_adult_baseline,
    load_ausynth_test_set,
)

# Load a US-trained income classifier and Australian test data
model = load_us_adult_baseline()
au_data = load_ausynth_test_set()

# Run fairness audit
report = FairnessReport(
    model=model,
    target_column="income_above_threshold",
    protected_attributes=["SEXP", "BPLP", "AGE5P"],
)
report.run(test_data=au_data)
report.show()

What you'll see

• Country-of-birth bias gap (~30%): The US-trained model handles US-typical birth countries well, others poorly
• Income threshold miscalibration: $50K USD doesn't map cleanly to Australian income distributions
• Occupation bias (~18%): Australian occupation categories don't align with UCI Adult codes

This is the standard fairness-testing scenario for Australian deployments: models trained on US data need to be validated against Australian populations before production use.

What just happened?

The UCI Adult Income dataset is the canonical fairness benchmark in ML, but it's US Census data from 1994. When you run a model trained on it against Australian population data, the validator surfaces the bias gaps that come from the distribution mismatch.

For your own models, replace load_us_adult_baseline() with your model and load_ausynth_test_set() with your test data or an AUSynth subset.

---

CI/CD gate

from verosynthea_validator import assert_fair

# Fails the build if any group accuracy gap > 5%
assert_fair(test_data, "label", "prediction", max_accuracy_gap=0.05)

In pytest:

def test_model_fairness():
    predictions = model.predict(test_data)
    test_data["y_pred"] = predictions
    assert_fair(
        test_data, "y_true", "y_pred",
        protected_columns=["SEXP", "BPLP", "profile_name"],
        max_accuracy_gap=0.05,
        max_demographic_parity_gap=0.10,
    )

What it measures

For each protected column (e.g. sex, birthplace, demographic profile), the validator computes:

Metric	What it checks
Accuracy gap	Max accuracy difference between any two groups
Demographic parity gap	Max difference in selection rate (P(y_pred=1))
Equalised odds gap	Max difference in true positive rate or false positive rate

Groups smaller than 30 observations are excluded (configurable via min_group_size).

Why this instead of fairlearn or aif360?

Those are general-purpose fairness frameworks. This package is purpose-built for Australian demographics:

• Pre-loaded demographic data. The free tier includes 5,000 synthetic individuals from AUSynth with 25 Census-calibrated variables. No need to source your own protected attributes.
• 8 demographic profiles. AUSynth clusters every person into one of 8 profiles (High-earning professionals, Young singles, Retired, etc.) — a richer protected attribute than just age or sex.
• Australia-specific calibration. Variables match ABS Census 2021 categories exactly. Income brackets, occupation codes, education levels, birthplace regions — all in Australian standard classifications.
• One-line CI gate. assert_fair() drops into pytest with zero configuration.

Data tiers

Tier	Data	Cost
Free	5,000-row Paddington 4064 sample from Hugging Face	$0
Paid	Full national dataset (32M individuals, 15,352 suburbs) via API	verosynthea.com

from verosynthea_validator import load_ausynth_sample

# Free tier (downloads from HF on first call)
df = load_ausynth_sample()

# Paid tier
df = load_ausynth_sample(api_key="vero_...", geography="bondi-2026-nsw")

The 8 demographic profiles

ID	Name	Typical characteristics
0	Labourers and operators	Blue-collar, lower income
1	Young singles and non-workers	Under 25, students, NILF
2	Children	Under 15
3	Non-earning dependants	Adults not in workforce
4	Trades and technical workers	Certificate-qualified, mid income
5	Established partnered households	Married, mid-career
6	Retired and semi-retired	Over 60, pension income
7	High-earning professionals	Degree-qualified, professional occupations

Installation

pip install verosynthea-validator          # core (pandas + numpy)
pip install verosynthea-validator[hf]     # + Hugging Face datasets loader
pip install verosynthea-validator[paid]   # + httpx for API access
pip install verosynthea-validator[dev]    # + pytest + sklearn for development

Links

• Dataset: vero-synthea/ausynth-sample on Hugging Face
• Full product: verosynthea.com
• Methodology: verosynthea.com/about

Citation

Verosynthea AUSynth (2026). Synthetic Australian Census Data.
https://verosynthea.com

License

MIT