attribution-graph

Calibrated entity resolution for attribution investigations.

Given observations about domains, accounts, seller IDs and registry records, decide which of them denote the same real-world entity — with a probability you can defend rather than an edge in a graph you can't.

No network I/O. Bring your own collectors.

pip install attribution-graph

Why this exists

OSINT tooling is well served on collection — SpiderFoot, recon-ng, Amass and theHarvester all acquire data competently — and on storage, where OpenCTI and Maltego hold and visualize the result. The gap is in between: deciding what the collected observations actually support.

Graph traversal is the usual answer and it fails on entity attribution for a structural reason. Infrastructure identifiers are primary keys; entity identifiers are evidence. Treating the second like the first produces three predictable errors:

• Unbounded transitivity. A shares an IP with B, B shares a registrar with C, C shares an analytics ID with D. Every edge is true and the component grows until it's useless.
• Correlated evidence counted as independent. 400 commits from one repo look like 400 observations. They're one.
• Uniform edge weight. A shared GA4 measurement ID and a shared Cloudflare IP are the same edge in a graph database, and differ by six orders of magnitude in what they imply.

This library fixes all three. Full derivation in METHOD.md.

The model

llr = reliability × weight × decay(age) × log(1 / selectivity)

Selectivity is measured, not asserted. It's the fraction of the corpus carrying an identifier value, so a GA4 ID held by one site is worth ~13.8 nats and a Cloudflare IP is worth ~0 — without anyone telling the system what Cloudflare is.

Correlated observations can't stack. Every claim declares a correlation_group; within a group, evidence aggregates as max + log(1+n) rather than summing.

Corroboration is structural. The per-group cap (8.0 nats) is deliberately smaller than the prior (−11.5 nats), so no single inferential source can carry a merge on its own — enforced by arithmetic, not analyst discipline.

Registry assertions are exempt. GLEIF saying an LEI belongs to a legal name is definitional, not inferential. Without that carve-out the system could never resolve a company to its own LEI.

Quickstart

import asyncio
from attribution_graph import (
    CaseScope, Engine, Claim, Identifier, IdKind, Predicate, Reliability,
    SourceClass, assess, write_all,
)

class MyCollector:
    name = "my_registry"
    accepts = (IdKind.ORG_NAME,)
    source_class = SourceClass.PUBLIC_REGISTRY   # checked at load; denied classes raise
    priority = 2

    async def collect(self, ident):
        return [Claim(
            subject=ident,
            predicate=Predicate.SAME_AS,
            object=Identifier(IdKind.LEI, "5493001KJTIIGC8Y1R12"),
            collector=self.name,
            source_url="https://api.gleif.org/...",
            reliability=Reliability.AUTHORITATIVE,
            correlation_group="gleif|5493001KJTIIGC8Y1R12",   # get this right
        )]

scope = CaseScope.load("case.yaml")     # refuses to load without an authorization ref
engine = Engine(scope, collectors=[MyCollector()])
result = asyncio.run(engine.run())
write_all(engine.graph, result, scope, Path("./out"))

Or score a set of claims directly, with no engine:

from attribution_graph import assess

a = assess(claims, holder_lookup=lambda ident: corpus.count(ident))
print(a.probability, a.band.value, a.estimative, a.independent_groups)
# 0.989 ATTRIBUTED 'almost certainly' 2

What it produces

• investigation_graph.json — claims with full provenance
• entities.ftm.json — FollowTheMoney, loads into yente and Aleph
• graph.cypher — Neo4j
• attribution_report.md — ICD 203 estimative language, blocked merges, evidence groups

The one thing to get right

correlation_group. It defines what counts as one observation, and it's the difference between a calibrated score and a confident wrong answer. All SANs on one certificate are one group. All profiles found by enumerating one username are one group — the observation is "this person reuses a handle", made once.

Selectivity needs a corpus

InMemoryIndex (the default) counts only what the current case has seen, so it systematically overestimates uniqueness. Assessments produced with it are upper bounds on confidence. Implement SelectivityIndex against a persistent observation store before treating output as evidential:

class CorpusIndex:
    def holders(self, ident) -> int: ...
    def universe(self) -> int: ...

engine = Engine(scope, collectors, index=CompositeIndex(CorpusIndex(), InMemoryIndex(graph)))

Scope is executable, not documentary

Attribution machinery generalizes across targets by construction: what identifies a scraper operator identifies anyone. So the constraints are code, not a runbook.

• CaseScope.load() refuses to start without an authorization reference
• The engine raises on any collector declaring a denied source class — data brokers, breach corpora, authenticated scraping, biometrics, location brokers
• pivot_radius hard-caps hops from an authorized seed
• A case scoped to Company won't instantiate Person entities
• Identifiers are salted-hashed at rest under minimize: true
• Every collector call hits an append-only audit log

The deny list lives in the loader rather than the documentation because a pipeline that can reach a source eventually will.

Verification trail

What a reviewing investigator needs to check a report now: the reasoning in the order it happened, an exact timestamp on every item, and numbered citations tying each assertion to its source.

trail = Trail(case_ref, authorization)
trail.seed("domain:scraper-site.example")
n = trail.cite("https://scraper-site.example/ads.txt", body_sha256=h, status=200)
trail.extract("seller ID", "pubmatic.com/156423", n)
trail.empty("https://api.ch/search?q=Acme", "UK Companies House", "companies_house_uk")
trail.infer("domains share an operator", basis="shared AdSense account",
            citations=[n])

Renders as an ordered, per-step-timestamped list with a numbered source appendix. Steps are typed, so (inference) lines are visibly distinct from observations — a reviewer can see at a glance which lines are things seen and which are conclusions drawn. Queries that returned nothing are recorded as steps, because a reviewer must be able to tell "checked and empty" from "never checked".

Name variants across scripts

When two names match through transliteration, the report states the exact transform chain:

>>> name_match("Lakshmi", "Laxmi").describe()
"matched at 'lakshmi': 'Laxmi' via indic_ksh (1 transform)"
>>> name_match("Мосэнерго", "Mosenergo").describe()
"matched at 'mosenergo': 'Мосэнерго' via cyrillic_bgn_pcgn (1 transform)"
>>> name_match("Zhang Wei", "Chang Wei").describe()
"matched at 'chang wei': 'Zhang Wei' via pinyin_wade_giles (1 transform)"

23 named rules across Latin, Indic, Cyrillic (BGN/PCGN, ISO 9, ALA-LC), Arabic and CJK (Pinyin/Wade-Giles, Hepburn/Kunrei, RR/McCune-Reischauer). Rule-based rather than learned, deliberately: a model that cannot say why two names matched is unusable in a report.

Over-generation is the real risk, so matches that collapse both names to a stub are rejected — without that guard two shortening rules meet in the middle and report a match between unrelated people.

Negative evidence

Three things get conflated under "found nothing", and they carry different weight: not looked for (no information), looked for and absent (weak evidence against, scaled by source completeness), and expected and absent (the hypothesis predicted it — the strongest negative signal available, and the one nothing else models).

Coverage is declared per source rather than assumed. A UK company absent from Companies House is meaningful; absent from Wayback means almost nothing.

Evidence packages

For work that may reach counsel, EvidenceLog preserves what a report can only describe:

log = EvidenceLog(scope, Path("./out/evidence"))
log.record(url, status, body, collector="gleif")
log.record_negative(url, "companies_house_uk", "no matching entity on 2026-08-17")
write_evidence_package(log, findings_summary)

Produces content-addressed capture storage, a hash chain (removing or reordering any capture breaks every subsequent entry), evidence_manifest.json, RFC 3161 timestamping instructions, a declaration skeleton, and a standalone dependency-free verify.py a third party can run without installing anything.

The framing that matters: live web findings are not reproducible. Re-running queries next year will not return this year's answers, and any report implying otherwise is misleading. The preserved bodies are the evidentiary artifact; the steps explain the reasoning. The manifest says this in as many words.

It also keeps observation separate from opinion. Captures are facts about what a source contained at a time. Confidence scores are analysis — and per the calibration gap below, analysis with no measured error rate. Don't let a report blur the two.

Companion packages

• adtx-attribution — collectors for ads.txt/sellers.json, GLEIF, EDGAR, Companies House, RDAP, CT logs, plus a reverse seller-ID index that doubles as a selectivity corpus.
• handle-correlation — same-actor scoring for usernames observed across forums, code hosts and messaging platforms.

Reports

Confidence scores are optional (show_scores=False) — sometimes the finding and its sources are the deliverable and a probability just invites false precision. When scores are shown, every report carries a disclaimer stating they are analytical judgment rather than measured fact, and that the model is not yet calibrated against ground truth. Every report ends with a per-source attribution and terms table covering only the sources actually used.

Status

0.2.0. The API is not yet stable. Calibration has not been validated against a labelled ground-truth corpus — bands are principled but not empirically fitted, so read probabilities as ordinal for now. That validation is the top open item; see METHOD.md §7 and issue #1.

License

Apache-2.0. See CITATION.cff if you use this in published work.