enum2x2

Every 2×2 table a published summary statistic still allows.

A study comparing two diagnostic criteria on one population computes its numbers from a 2×2 table, then publishes the numbers and not the table. The quantity a reader wants — how many patients the two criteria classify differently, and in which direction — is gone. It can often be recovered, because the same reports print the sample size, both marginal totals and a statistic such as κ, and together those leave a few integer tables, often just one.

The problem, concretely

A pooled series of 768 patients, two readings of the DSM-5 delirium criteria, published as κ = 0.29. That reads like two definitions that disagree a fair amount.

Here is the table underneath it:

The strict class is nested inside the relaxed one. No patient is strict-positive and relaxed-negative; the 308 they differ on all fall the same way, relaxed-positive and strict-negative. One reading is simply three times wider than the other. Observed agreement is the highest these two marginals permit: κ = 0.29 is the highest κ these marginals allow.

None of that is in "κ = 0.29", and κ = 0.29 is all the paper printed.

Install

pip install enum2x2

For development:

git clone https://github.com/elliottower/enum2x2 && cd enum2x2 && pip install -e ".[test]"

Standard library only. Python 3.10+. make compare and make crosscheck need R and skip cleanly without it.

Use

import enum2x2

r = enum2x2.recover(768, n_a=158, n_b=466, kappa="0.29")
r                     # <Recovery unique on N=768: Table(158, 0, 308, 302)>
r.status              # 'unique'
r.table.discordant    # (0, 308)   -- 308 patients one way, none the other
r.table.asymmetry     # 1.0

r = enum2x2.recover(20306, n_a=866, n_b=1603, kappa="0.22")
r                     # <Recovery 11 tables on N=20306: 320-330/536-546/...>
r.cell_ranges["n10"]  # (536, 546)

r = enum2x2.recover(370, p_a="44.6", p_b="43.2", marginals_as_percent=True,
                    kappa="0.48", agreement="73", agreement_as_percent=True)
r.status              # 'infeasible'
r.reason              # 'the marginals and kappa admit 1 table(s); adding the
                      #  published agreement admits none'

What can close the table

A 2×2 with N fixed has three degrees of freedom and the two marginals use two, so one further quantity closes it. Each of these does, passed as the string the source printed:

enum2x2.recover(113, n_a=39, n_b=83, kappa="0.32")        # agreement coefficient
enum2x2.recover(113, n_a=39, n_b=83, agreement="0.61")    # raw observed agreement
enum2x2.recover(113, n_a=39, n_b=83, mcnemar="<0.001")    # a printed McNemar result
enum2x2.recover(113, n_a=39, n_b=83, discordant=44)       # the discordant total itself
enum2x2.recover(113, n_a=39, n_b=83, positive_agreement="0.64")   # specific agreement

Everything accepted, including Byrt's two indices, which are taken as checks but close nothing on their own:

McNemar is the one worth spelling out. It looks only at the two discordant cells, and the marginals already fix their difference, so the statistic fixes their sum and the table follows. A paper printing a McNemar result printed the table without saying so. The exact (binomial) variant is a ratio of integers and is decided exactly; the chi-square variants compute a tail probability that is not rational, and say so. The exact test recomputes its binomial tail for each candidate table, so with McNemar as the only closing figure it slows above a few thousand cases; alongside kappa it is fast.

κ is passed as the string the source printed, not as a float: "0.10" and "0.1" imply different intervals and a float cannot tell them apart. Passing a float is refused with that reason.

Four statuses, and they are kept apart: unique, set, infeasible (the published figures admit no common table), insufficient (a required figure was never published). An impossible call — a marginal above N, a κ outside [-1, 1], two marginals given for one criterion — raises InvalidInput. An impossible source is reported, never raised.

recover_many cannot raise, so a row that would have raised InvalidInput comes back with a fifth status, impossible, and its reason. Counting those as insufficient would inflate how many sources under-reported.

enum2x2 comparisons.csv       # a file of comparisons, one per row
make test                     # the test suite
make compare                  # against metafor::conv.2x2, under rounding
make crosscheck               # this kappa against every R implementation installed

Why rounding is the whole problem

With exact inputs the algebra is one line. For positive marginals p_A and p_B on N observations,

p_e  = p_A·p_B + (1 − p_A)(1 − p_B)
p_o  = κ(1 − p_e) + p_e
p_11 = (p_o + p_A + p_B − 1) / 2

But nobody prints exact inputs. A marginal printed 4.26% and a κ printed 0.22 are intervals, so the inputs identify a set of integer tables. This walks every non-negative integer table summing to N whose statistics round back to the printed strings, and reports the set. Where a source also prints its raw agreement, that enters as a further constraint and narrows the set.

Rounding conventions

Which interval a printed figure stands for depends on how the source got from the exact value to the digits. The default reading is round-half-up, so 14.7 stands for [14.65, 14.75]. A source that truncates prints 14.7 for everything in [14.7, 14.8), which shares only its lower end with the first.

u is one unit in the last printed place, read off the literal string, so "3.3" and "3.30" differ here as they do everywhere else.

enum2x2.recover(353, n_a=84, n_b=46, kappa="0.648", convention="any")

Goyal et al. (2025) print a kappa and a disagreement percentage for each of the 55 pairwise comparisons of eleven gestational-diabetes criteria on 353 women. Held against the table its own kappa identifies, each printed disagreement is consistent with either convention (33 of them), with truncation alone (15), or with rounding alone (7). Read as rounded throughout, 15 of the 55 comparisons return no compatible table; under any all 55 are uniquely identified.

Four outcomes, deliberately kept apart:

The last two are different findings and collapsing them would report a gap in the record as a defect in the source.

Against metafor::conv.2x2

conv.2x2 converts a summary into a single table. This enumerates every table the summary admits. The names are meant to sit next to each other.

It takes an odds ratio, a phi coefficient or a chi-square — not Cohen's κ, which is what agreement studies report — and on rounding its documentation is candid:

The calculations underlying the function are exact only for unrounded values … The present function uses optimization methods to reconstruct the table counts so that the discrepancy between the reported measures and the reconstructed ones are minimized. This is not guaranteed to reconstruct the actual table exactly, but should usually yield a close match.

make compare measures what "close" costs, on 1,200 known tables at N = 2,000, 9,170 and 20,306 with one statistic printed to two decimals:

conv.2x2 is not broken; it does what it says. But it hands you one table and no indication of how far off it is. Where a finding rests on how many patients were reclassified in each direction, an unstated error of up to 20 patients can change it.

Where this sits among neighboring packages

mlscorecheck is the closest. It reads printed scores as rounding intervals, as this package does, but takes the exact counts on one side and no second marginal, so where it runs it usually leaves many tables where both marginals leave one; on every table this package recovers, the two agree. We know of no other package that recovers a paired-agreement table from both marginals and reports every table the figures admit.

What it will not do

• It does not tell you which criterion is correct. Neither does the table.
• It does not separate disagreement between criteria from inconsistent application of either. That needs the same criterion applied twice to the same patients, which no aggregate report contains.
• An empty candidate set says the published figures admit no common table under the definitions and rounding rules used here. It does not say an error was made — a denominator, an analysis set, or a version of a variable may simply have gone unstated.
• Three or more categories are out of scope; the identity above is the binary case.

Validation

1. Exhaustively, over the whole space. The enumerator must agree, as a set, with a brute-force sweep over every non-degenerate integer table at N = 18 and N = 22 — 1,258 and 2,212 of them, being every table for which κ is defined. Not a sample.
2. Against a second implementation. Written from the definitions in exact rational arithmetic, sharing no code with the enumerator, over every table at N = 16, 20 and 24. Agreement between two implementations that share helpers proves only that the helpers are consistent. Those sweeps run at small N, where the granularity of κ and of the marginals is coarser than the interval arithmetic being checked, so the rounding boundaries are stated as concrete cases instead: a κ of exactly 0.625 sitting on the endpoint of "0.62", and a marginal printed "0.512" at N = 240 whose only valid count is lost to truncation.
3. Against the established R implementations of κ itself. The enumeration is only as good as the statistic it inverts, so make crosscheck computes κ for 600 integer tables with this module and with every R implementation present on the machine, and reports the largest disagreement. Against irr::kappa2, psych::cohen.kappa, vcd::Kappa and DescTools::CohenKappa the worst disagreement is 6 × 10⁻¹⁵ — floating-point representation, not arithmetic. epiR::epi.kappa cannot be installed here (its dependency chain wants GDAL) and is checked against its source instead, which is exact: its (pO.p - pE.p)/(1 - pE.p) with pE.p <- sum(r.totals * c.totals)/n^2 is the same formula. Packages that are absent are named rather than silently skipped.

The study this was built for

The delirium table above, and eight other published comparisons, are analyzed in a manuscript on what a published agreement statistic conceals. That work lives in its own repository, with the corpus, the preregistration frozen before the enumeration was run, the results, and the sentence each published figure was read from. This package carries only the method.

Citation

See CITATION.cff.

License

MIT.