Numeric Rule Finder

An exact, dependency-light library that discovers the conservation laws latent in structured movement/transaction data, instead of making you declare one.

You declare nothing. It recovers the complete lattice of independent conservation laws the data actually obeys — including laws nobody wrote down — finds where they break, types each break (a re-attributable slip vs. a genuine hole), and honest-stops when there is no structure to exploit.

📖 There's a book. Read the ebook → — what it is, how it works, runnable worked examples (with real output), three machine-learning head-to-heads, a raw-XML showcase, a tour across number systems, and a mathematics appendix.

Two front doors

Just want answers? (no maths).

• See it in action — the headline walkthrough, Closing the month at Northwind Retail, in BUSINESS_GUIDE.md: six plain-English checks, each ending in an action (run it: examples/northwind_close/).
• On your own data — python -m numeric_rule_finder.cli (or from numeric_rule_finder import Reconciler); hand it a CSV and two column names.
• What you get — what balances, where it breaks and probably why, and any hidden separate sub-systems — found exactly in one pass (independent_groups, or the groups command).
• It's intelligent — when ordinary balancing finds nothing, it silently escalates to the deeper maths (e.g. modular/parity structure) and reports it in plain words.
• Across domains — accounting, energy, supply chain, ETL, clinical trials, elections, …: python examples/gamut/gamut_demo.py.

Want the mathematics? — read the ebook Appendix — The mathematics. The engine lives in the numeric_rule_finder/ package.

How it works (in brief)

Numeric Rule Finder treats your data as signed movements — a group, a bucket, an amount — and finds the weighted combinations of buckets that every movement leaves unchanged (the conservation laws), then measures exactly where they break. It climbs only as far as it needs:

• balance & structure — which groups don't net out; which buckets form hidden separate books (the exact independent groups, in one linear pass);
• modular laws — patterns invisible to ordinary arithmetic ("moves only in cases of 12", parity), via Smith Normal Form;
• typed residuals — is a break re-attributable (a coboundary) or a genuine hole (an obstruction that names the violated law)?
• multi-source consistency — whether independent reconciliations can all be true at once (an H¹ obstruction);
• substrate generality — the same engine over ℤ, ℚ, 𝔽ₚ, and ℚ[t] (parametric, rate-dependent laws).

Everything is exact (integer/rational arithmetic, never floating point) — and it scales: a modular-rank fast path certifies the honest stop (no conservation structure) in machine-integer arithmetic, skipping the exact rational solve where there is nothing to find. The actual mathematics — definitions, theorems, the coker(S) / H¹ residual typing, Smith Normal Form, and a map of the code — is in the ebook Appendix — The mathematics.

Real data carries more than one law

Point it at a dataset and it reports how many independent conservation laws hold. Real data routinely has several — separate books, conserved moieties, per-SKU stock — not just the one obvious balance:

With and against machine learning

Where the signal is an exact law this beats statistical anomaly detection; where the job is partly fuzzy, it makes the model's job easier; and where the answer is an exact structure, it replaces a slow unsupervised job outright. Three reproducible head-to-heads (all walked through in Chapter 7 of the ebook):

• it beats an Isolation Forest outright on skim fraud — examples/fraud_vs_ml/;
• it feeds a Random Forest as an exact pre-filter + feature, lifting its score — examples/ml_assist/;
• it finds the exact groups in one pass — Reconciler.independent_groups — where a clustering k-sweep is thousands of times slower and wrong — examples/grouping_vs_ml/.

Measurements, not just ledgers — uncertainty-aware testing

A ledger balance either closes to the penny or it does not, and a single scalar tolerance is the right test for it. Measurements are different, and until 0.2.0 this library had no way to say so:

• a residual of 5 is a catastrophe if sigma is 0.1 and noise if sigma is 50;
• a law with coefficients (17, -17) should not face the same absolute cut as one with (1, -1) — both its residual and its sigma scale by 17;
• and "residual consistent with zero" hides two different situations that must not be conflated: the law was tested and holds, or the data could never have tested it.

check_conservation_noisy judges each law against its own propagated sigma:

from numeric_rule_finder import discover_invariants, check_conservation_noisy

d = discover_invariants(rows, entity_key="account", event_key="txn", qty_key="delta")

report = check_conservation_noisy(
    measurements, d.laws,
    entity_key="account", event_key="txn", qty_key="value",
    sigma_key="value_err",      # per-record uncertainty
    z_break=3.0,                # |z| above which a law is judged broken
    resolution=0.5,             # the residual size that would have mattered
)
print(report.summary())
print(report.chi2_per_constraint)

Three verdicts, not two. holds, BREAKS, and — the one that matters — untestable: the law exists, but sigma is larger than the residual size you declared would matter, so no clean bill of health is issued. report.statistically_mute says "the structure is there and this data cannot speak to it", which is a different finding from Discovery.honest_stop ("there is no structure") and worth saying out loud rather than hiding behind a passing tolerance check.

A basis-independent joint statistic. chi2 = r.T C^-1 r with C = Y Sigma Y.T. A naive sum of per-law z^2 depends on which basis of the conservation space happened to be returned; this does not. chi2_per_constraint is then the natural "how badly does the structure fail, per independent constraint" number.

Correlated uncertainties are supported via cov=, because correlation is the normal case whenever a shared quantity was estimated from the same data the laws are tested on.

Discovery stays exact. Which laws exist depends on the incidence pattern, not on measurement precision, so discover_invariants keeps its exact Fraction/integer arithmetic and gains no noise model. The statistical layer sits strictly on top. Supply no uncertainties and you get the previous exact behaviour back unchanged.

Worked example: loop closure across overlapping instruments

Conservation laws are not only ledger balances. Given several vantages that measure the same quantity with unknown per-vantage offsets, and overlapping coverage, the offsets must sum to zero around every cycle in the overlap graph — a loop-closure condition, testable with no reference truth. Map it by taking entities = overlaps and events = vantages, and the discovered laws are the independent loops:

recs = []
for (a, b), (delta, sigma) in pairwise_measurements.items():
    recs.append(dict(edge=f"{a}|{b}", node=a, inc=1))
    recs.append(dict(edge=f"{a}|{b}", node=b, inc=-1))

d = discover_invariants(recs, entity_key="edge", event_key="node", qty_key="inc")
# d.n_laws == the number of independent loops == the closure degrees of freedom

Feeding the measured pairwise differences and their sigmas to check_conservation_noisy then reproduces the weighted-least-squares closure chi-squared exactly, and additionally names which loops break. Validated against eight overlapping astronomical instruments (JWST NIRISS/NIRCam/NIRSpec, HST/STIS+WFC3, VLT/FORS2, Spitzer/IRAC): 5 independent laws recovered, joint chi-squared agreeing with a hand-rolled solve to 1e-9, and the two breaking loops both localised to the same triangle of instruments.

Run

numeric_rule_finder has no third-party dependencies; petra_adapter uses petra-nn only to read Petri nets, and the ML examples use scikit-learn.

pip install -e .                                      # optional (pure-stdlib core)
python -m numeric_rule_finder.cli check  data.csv --group txn_id --amount amount
python -m numeric_rule_finder.cli groups data.csv --group txn_id --account account  # exact independent groups
python examples/northwind_close/close_the_books.py    # a worked example
python examples/gamut/gamut_demo.py                   # the same engine across domains
python -m pytest tests examples -q                    # the suite

License

MIT-with-attribution — see LICENSE.