dictcollision

Calibrate dictionary hit rates. Given a list of short strings and a reference dictionary, separate real matches from chance collisions.

The general problem

You have a stream of short strings and a big reference dictionary. Some fraction match. How many are real matches vs. the dictionary being large enough that anything would match?

If the input is short (2–4 chars) and the dictionary is large (10K+), naive hit-rate metrics are badly inflated by chance. This package fixes that.

(¹) When the candidate plaintext was produced by a stochastic search over a key space (SA, hill-climbing), the cipher symbols themselves are also a relevant input — see When your decode came from a search.

Install

pip install dictcollision
pip install "dictcollision[viz]"   # with matplotlib

Or with uv:

uv add dictcollision                 # into a uv project
uv pip install dictcollision         # into the active venv
uv tool install dictcollision        # install the CLI globally

Quick start

from dictcollision import noise_floor, classify, recommend

# 1. One-line prediction: how much of my 43% hit rate is chance?
predicted = noise_floor(decoded_tokens, dictionary)
print(f"Chance alone predicts {predicted:.1%}; observed 43.0%; "
      f"excess {0.43 - predicted:.1%}")

# 2. Full four-category analysis.
result = classify(decoded_tokens, dictionary)
print(result.summary())

# 3. Rank candidate dictionaries when you don't know the language.
ranked = recommend(decoded_tokens,
                   {"latin_10k": latin_words, "german_50k": german_words})
print(ranked[0].name, ranked[0].excess)

result.summary() prints:

ClassifyResult (n=5000 tokens)
  apparent hit rate :  99.6%
  net signal        :  70.1%   <- calibrated metric
  correction        :  29.5%   <- amount subtracted

  signal       70.1%  ████████████████░░░░░░░  real matches
  shared_hit   19.4%  ████░░░░░░░░░░░░░░░░░░░  chance collisions
  anti_signal   0.6%  ░░░░░░░░░░░░░░░░░░░░░░░  phantom matches
  shared_miss   9.9%  ██░░░░░░░░░░░░░░░░░░░░░  non-dict tokens

  Interpretation: strong signal — dictionary is a good fit

Command line

No Python required:

python -m dictcollision --tokens decoded.txt --dict latin_50k.txt
python -m dictcollision --tokens decoded.txt --dict latin.txt --baselines
python -m dictcollision --tokens decoded.txt --dict latin.txt --json > report.json

Supported dictionary formats: one word per line, word count (hermitdave FrequencyWords), or CSV. See python -m dictcollision --help.

Input and output contract

Input:

decoded_tokens : list[str]          # e.g. ["the", "cat", "ab", "cd", ...]
                                    # any Unicode; no preprocessing assumed
dictionary     : set[str] | list[str]   # reference words, same encoding

Output (classify) → ClassifyResult dataclass:

Interpreting net_signal

The core equation

The predicted noise floor for dictionary $D$ against decoded text with character distribution $p$ is:

$$\hat{r} \;=\; \sum_{w \in D}\; \prod_{i=1}^{|w|} p(w_i)$$

For every word in the dictionary, multiply together the character frequencies of your decoded output. Sum. That number is how many of your tokens would match by accident.

Four-category framework

Net signal = Signal − Anti-signal is the calibrated metric.

Null corpora are generated from the decoded text's character bigram distribution (configurable: unigram / bigram / trigram), preserving character-pair frequencies and token lengths while destroying word identity. On wrong-language evaluations the four-category framework is the only method among six tested that correctly reports signal as ≤ 0.

When your decode came from a search

If your decoded tokens are the output of a stochastic search over a key space (simulated annealing on a substitution alphabet, hill-climbing, AZdecrypt, etc.), net_signal alone can mislead. The search itself can manufacture apparent signal: a quadgram-optimised key on a short cipher will find local optima that resolve into a handful of high-frequency dictionary words even when the cipher has no underlying linguistic structure. The Dorabella case (Ruckman 2026) documents this failure mode at +0.55 net_signal.

The fix is to give the same search procedure the same matched-budget opportunity on shuffles of the cipher — multiset-preserving permutations that destroy positional content but keep the character budget constant. If the search finds materially more signal on the real cipher than on its shuffles, that excess is the calibrated signal.

from dictcollision import search_calibrated_signal

result = search_calibrated_signal(
    cipher_symbols=cipher,        # the cipher itself, not decoded tokens
    search_fn=my_sa_search,       # cipher -> decoded tokens
    dictionary=word_set,
    n_shuffles=30,
)
print(result.summary())
# z_score >= 3 → search finds real signal
# −1 ≤ z < 1   → indistinguishable from a shuffle baseline

search_calibrated_signal and null_distribution solve different problems:

Both can be informative; reach for search_calibrated_signal whenever the decoded tokens were chosen by a key-space optimiser.

Full API

from dictcollision import (
    noise_floor,                  # analytical collision prediction
    classify,                     # four-category classification
    classify_by_length,           # per-length-bucket breakdown
    recommend,                    # rank candidate dictionaries
    null_distribution,            # Monte Carlo null distribution
    bootstrap_ci,                 # bootstrap CI on net_signal
    search_calibrated_signal,     # matched-budget shuffle calibration
    load_dictionary, load_tokens, # file loaders
)

from dictcollision.baselines import (
    apparent_hit_rate,            # no correction
    subtract_null,                # naive baseline
    permutation_test,             # per-word Poisson test
    bh_fdr,                       # Benjamini-Hochberg
    blast_evalue,                 # BLAST-style E-value
    all_methods,                  # all six in one dict (Table 2 style)
)

from dictcollision.viz import (
    plot_decomposition,           # paper Figure 1
    plot_size_sweep,              # paper Figure 2
    plot_method_comparison,       # paper Figure 5
    plot_length_stratified,       # paper Figure 13
)

Examples

Self-contained scripts in examples/:

• 01_vigenere.py — evaluate a Vigenere candidate key
• 02_paper_table2.py — reproduce the six-method comparison
• 03_dictionary_recommender.py — pick the right dictionary without knowing the language
• 04_search_calibrated.py — calibrate a stochastic search against shuffle baseline

Paper

Methodology, experiments, validation:

Ruckman, M. (2026). The Dictionary Collision Effect in Computational Decipherment. Source, figures, and reproduction code: https://github.com/mruckman1/signal-isolation-paper

Citation

@article{ruckman2026dictcollision,
  title={The Dictionary Collision Effect in Computational Decipherment},
  author={Ruckman, Matthew},
  year={2026},
  url={https://github.com/mruckman1/signal-isolation-paper}
}

License

MIT