certlim

What any Certify / Reject rule can achieve from the evidence you actually have.

certlim answers a question that accuracy metrics cannot: given two registered classes of systems — those that are safe under a proposed change and those that are unsafe — and the measurement you can afford to take, how often can any procedure return a reliable verdict while controlling both kinds of error?

And when the answer is "not often enough", it tells you which of the two remedies applies:

• buy more of the same evidence — the obstruction is temporary and closes at a computable block length; or
• change the measurement — the obstruction is permanent, and no amount of the same channel, nor any re-encoding of it, will ever help.

Distinguishing those two cases is the point. They look identical to a single-sample detector and they cost very different amounts of money.

pip install certlim

---

Sixty seconds

import certlim as cl

# Two safe populations and one unsafe one, over a two-symbol evidence alphabet.
exp = cl.Experiment(safe=[[0.25, 0.75], [0.75, 0.25]],
                    unsafe=[[0.50, 0.50]])

d = exp.diagnose()
print(d.region)            # 'delayed'
print(d.n_star)            # 2
print(d.explain())

Delayed separation: the one-unit hulls intersect but no exact pair collides, so repetition alone opens the gate at n = 2. Buying more of the same evidence is the correct move here.

Note what just happened. Every safe/unsafe pair in this example is well separated — the minimum pairwise total variation is 0.25 — yet one unit of evidence certifies nothing at all:

exp.pairwise_min_tv()      # 0.25   "looks separable"
exp.delta(1)               # 0.0    actually is not
exp.decisiveness(1)        # 0.10 = alpha + beta, i.e. no better than a coin
exp.delta(2)               # 0.125  two units break the tie

A criterion built on pairwise distances would have declared success here. The composite geometry is what governs, and it is what certlim computes.

---

The frontier

f = cl.frontier(exp, n_grid=[1, 2, 3, 5, 10], alpha=0.05, beta=0.05)
print(f)

   n   states     Delta         D     bound     slack  gate
   1        2  0.000000  0.100000  0.100000  0.000000     1
   2        3  0.125000  0.112500  0.225000  0.112500     0
   3        4  0.187500  0.116667  0.287500  0.170833     0
   5        6  0.273438  0.138889  0.373438  0.234549     0
  10       11  0.409561  0.196970  0.509561  0.312591     0

Delta is the composite separation, D the best worst-case decisiveness any valid rule attains, bound the ceiling min(1, alpha + beta + Delta), and gate flags block lengths where the two hulls still meet.

f.units_for(0.9) returns the smallest gridded block reaching 90 % decisiveness, or None if the grid does not get there.

---

The deployable rule

rule = cl.build_rule(exp, n=3, alpha=0.05, beta=0.05)
rule.decisiveness          # what it actually attains
rule.alert_rate_floor()    # the fraction of blocks that must come back Alert

decision = rule.decide([2, 1], rng=np.random.default_rng(7))
decision.outcome           # 'certify' | 'alert' | 'reject'

rule.error_profile(exp)    # realised worst-case side errors, self-check

The optimal rule is genuinely randomised, so pass a seeded generator whenever the verdict has to be reproducible — in a certification workflow it usually does.

When the hulls still intersect, the optimum is a rule that ignores the data and answers at the allowed error rates. certlim says so plainly rather than dressing it up. That is the signal to change the measurement.

---

From raw measurements

Start from per-unit symbol counts, one row per physical unit:

exp = cl.Experiment.from_counts(safe_counts, unsafe_counts, smoothing=0.5)

Building the evidence menu is where most mistakes happen, so the quantiser enforces the one property that makes menus comparable:

q = cl.NestedQuantizer.from_quantiles(features, bins=2, drop_order=(2, 1))
q.chain()                  # [8, 4, 2] — strictly nested, by construction
counts = q.counts(features, level=0)

Quantising each menu independently produces partitions that are not nested, and the resulting comparison can show a coarse menu beating its own refinement. That is a pipeline artefact. cl.check_data_processing(fine, coarse) will tell you when you have hit it.

---

Finite-sample calibration

A plug-in Delta from observed counts is an estimate:

cl.permutation_test(unit_counts, labels, blocks=chemistry, n_permutations=999)
cl.bootstrap_delta(unit_counts, labels, n_resamples=2000)

Both resample whole physical units. Resampling rows, cycles or time points instead will understate the interval and inflate significance by orders of magnitude, because repeated measurements of one unit are not independent evidence about the population of units.

Neither is a lower confidence bound on the population separation. A significant permutation p-value rejects label exchangeability; it does not certify anything with coverage. certlim keeps that distinction in the result objects.

---

Deployment and procurement

report = cl.transport_margin(source_exp, deployment_exp)
report.certified           # separation survives the registered shift?

The margin is Delta_source - drift_safe - drift_unsafe, measured as class geometry rather than point-prediction error. A non-positive margin means the source evidence does not certify transport — not that the deployment domain collides.

cl.screen([
    cl.Plan("passive",   n=1, decisiveness=0.63, cost={"seconds": 780, "c_rate": 0.0}),
    cl.Plan("active",    n=1, decisiveness=0.53, cost={"seconds": 940, "c_rate": 0.5}),
    cl.Plan("nameplate", n=6, decisiveness=0.10, cost={"seconds": 0,   "c_rate": 0.0}),
], target=0.4)

Costs stay a vector and are never collapsed into one currency. A plan is dropped only when another is no worse on every recorded axis. Plans measured on different ledgers raise rather than being ranked — an unmeasured cost is not a free cost.

---

The report

One command turns the whole analysis into a filing-ready page:

certlim report units.csv -o certification.html --subject "Lot 2026-08"

cl.write_report("certification.html", exp,
                unit_counts=counts, labels=labels, blocks=site,
                subject="Lot 2026-08",
                notes=("Grades come from the bench protocol, not from this channel.",))

The page carries the structural verdict and the action it implies, the frontier as a table and an inline chart, the finite-sample calibration, the decision table and the standing scope caveats. It is a single file with no external stylesheet, script, font or image, so it survives being archived or opened offline, and it is byte-deterministic unless you pass a timestamp — two runs of the same analysis produce the same file, and a diff means something actually changed.

Command line

certlim diagnose  units.csv                          # permanent, delayed, or open?
certlim frontier  units.csv --n 1 2 3 5 10 --target 0.9
certlim calibrate units.csv --block-col chemistry
certlim rule      units.csv --n 3 --json rule.json
certlim report    units.csv -o certification.html

Input is either a CSV of per-unit counts with a grade column holding safe / unsafe, or a JSON file of probability laws:

{"safe": [[0.25, 0.75], [0.75, 0.25]], "unsafe": [[0.5, 0.5]]}

Add --json out.json to any subcommand for a machine-readable report.

---

What certlim does not do

It computes population geometry exactly. It does not know whether:

• your two classes were registered before you looked at the data;
• your grades were identified by a reference experiment rather than by the channel you are evaluating — grading units with the channel under test makes every number here meaningless;
• your sampling unit is the physical unit it should be;
• your evidence is conditionally i.i.d. within a block;
• your registered models are cohorts rather than individual units. If each model is one physical unit, two units can produce identical histograms by chance, which is an exact collision by definition. certlim flags that case rather than letting it read as a structural obstruction, but it cannot decide for you how the classes should have been registered.

Those are design decisions and the package will not make them for you. Every result is relative to the registered classes: enlarging them can only shrink the separation, never grow it.

---

Method notes

Block laws are enumerated over count vectors, the sufficient statistic for i.i.d. draws from a finite alphabet, so the problem is C(n+K-1, K-1) rather than K**n. The composite distance and the optimal decision rule are solved as two independent linear programs, so their agreement with the certification bound is a real check rather than a same-code identity. There is no sampling, no fitting and no tuning parameter anywhere in the core.

delta() is solved through the dual separation game, which is 10x to 450x faster than the primal hull program on large state spaces. The dual loses accuracy as the space grows, and a value near zero is exactly where the gate verdict turns, so anything below 1e-5 is re-solved with the exact program; delta(n, exact=True) forces it everywhere.

Exact enumeration has a ceiling (max_states, default 20 000). It is a time budget, not a memory one — measured cost is about 3 s at 6 000 states, 40-100 s at 20 000, and hours beyond, with run-to-run variation of a factor of two at the same size. Exceeding it raises with the offending (K, n) and what to do about it, rather than silently approximating or starting a job that never finishes.

For a finite registered class with r distinct one-unit laws and no exact collision, the gate must be open by r - 1; Experiment.stopping_bound() returns it, and n_star() searches exhaustively up to it. The bound is tight for small classes and loose for large ones, and it bounds the structural search only — never the number of physical units needed for a confidence statement.

Licence

MIT.