The detection core

maisecrets/detection/ is the detector: it finds secrets and personal data in text. It holds no plugin, no home folder and no reference form of its own. The plugin gives those to it; a library caller can give its own.

Use it as a library

pyproject.toml builds the core as a wheel. It holds maisecrets/__init__.py and maisecrets/detection/ with its rule data, and no module of the plugin. It needs no other package and runs on Python 3.9 and later. It is not on PyPI:

python3 -m pip wheel --no-deps -w dist .        # dist/maisecrets-<version>-py3-none-any.whl
python3 -m pip install dist/maisecrets-*.whl

The version is the plugin's version (.claude-plugin/plugin.json); a wheel from main or a tag has the released one. importlib.metadata.version("maisecrets") gives it. Without the plugin modules, each argument left None takes no config, no sources and no reference form: a detector from the wheel never reads the home folder.

from maisecrets.detection import Active, Detector, Host, NO_REFERENCES, NO_SOURCES

d = Detector(
    regions=Active(regions=("generic", "de"), languages=("en", "de"), source="mine"),
    sources=NO_SOURCES,          # rules of your own, allow list, term lists, imported values: a Sources object
    references=NO_REFERENCES,    # how the caller writes a reference into text: a References object
    host=Host(user="me", year=2026),
)
for m in d.findings(text, channel="tool", path="notes.txt"):
    print(m.kind, m.type, m.start, m.end)

• Each argument left None takes the plugin's default when the plugin is there: the config of this computer, the readers of the home folder (maisecrets/sources.py), the placeholder form (maisecrets/placeholder.py), and this computer's user name and year.
• A detector with all four given reads nothing of the host and imports no module of the plugin (tests/test_detection_package.py shows this in a fresh process).
• Two detectors in one process keep their own rule lists, allow lists and caches, also in two threads and one inside a scan of the other (tests/test_detector.py).
• A function given as regions is asked when the rules load: the first scan and each reset(). A changed config takes effect at reset(). regions() names the regions that the rule list was loaded for.
• with d.activate(): makes every function of the module in this thread use d, for a caller (such as a hook) that calls maisecrets.detect.scan directly.

The API

maisecrets.detection.__all__:

The rule tables and the helpers of core.py are no API: they change with the rules.

core.py keeps six parts in modules of their own, so that each file stays readable: value_checks.py (the checks of a value), own_rules.py (OWN_RULES, FALLBACK_RULES), text_index.py (the index of a text and the test-code check), encoded_mail.py (an address with its @ as %40), keyword_tail.py (the vendored keyword rules in linear time) and deadline.py (the deadline of a scan and ScanIncomplete). core.py imports each of their names, so core.<name> is the same object. A patch of such a name on the core does not reach the functions of the part: patch the module that defines the name.

Field names: fields (since 0.8)

A caller that has the structure of its data (a JSON response walked with a field path, a key-value record) can give the field name of a value with the text, instead of the core guessing it back from the text:

d.findings("0171 4829173", channel="tool", fields=[(0, 12, ("customer", "telefon"))])   # one PHONE, 0-12
d.findings("0171 4829173", channel="tool")                                            # no hit

• The shape: a list or tuple of (start, end, name). start and end are the span of the value in code points of the text, 0 <= start < end <= len(text); name is the field path, outermost first, a tuple (or list) of str and int parts. A wrong shape raises TypeError, an offset outside the text ValueError; the message names the position (fields[1]) and never a part of the text or of a name. Fields may overlap.
• What a name does: it is evidence as the same label before the value in text is, and only that. The last part of the path that is no list index is put before a copy of the value (telefon: <value>) and that line is scanned with the rules of the call. A hit inside the value is added when the bare value has none there and the scan of the whole text has none there either (maisecrets/detection/fields.py). Two parts of a name are read for what they name (since 0.8.26, ai-gateway #1764): a last part that only wraps the value (value, number, nummer, wert) takes the name above it (("phone", 0, "value"), ("contact", "phone", "number"), ("steuer_id", "value")), and a phone word with qualifiers only (phone_home, telefon_privat, mobileBusiness, phone_cell, phoneNumbers) is the phone word. A name with another word (phone_type, phone_country_code, mobile_app_version) stays as it is.
• What it never does: take a hit away (the hits of the text stay as they are), add a hit outside its span, bring back a hit that the text around the value took away, or parse anything: the core reads JSON and the other formats of the text itself, as without fields. A field that names what the core read itself adds no second hit.
• Names that add nothing: a name the label rules give no evidence for (id, value, data, title, an unknown word), key alone (in API data it names a ticket key, an object key or a translation key as often as a credential; api_key, secret_key are labels as in text, and so is token alone), the cursor of a paged list (nextPageToken, syncToken, continuationToken, purchaseToken; not PrevAuthToken), and a name with a line break, :, = or a quote, or longer than 64 characters. build_number stays text.
• Without fields (or with an empty list) a scan is the one of before: scripts/detection_fingerprint.py gives the same bytes. Each field costs two scans of its value, one with the label and one bare. In findings() with no path, a field over the whole text costs one, since the scan of the text is its bare scan; scan(..., fields=) scans twice.
• tests/test_detection_fields.py holds these points; tests/conformance/cases.json has cases with context.fields in the gateway's shapes (a leaf with its field path, and a JSON text with fields of its own keys).

The API: the decision of 2026-10-20

The wheel (pyproject.toml) makes the core usable without the plugin, so its names carry a promise. Decided by the maintainer on 2026-10-20 (merge request !160):

Changes to a stable name go through a deprecation, also before 1.0: the old form keeps working with a DeprecationWarning for at least one minor version, the changelog says so under "API" (a commit footer API: …, scripts/release.py), and only then it goes. A 1.0 of the wheel would freeze the stable names. region and channel for user_rule come only when they are meant to be carried as API.

Checks that hold it: tests/test_detection_package.py pins __all__, the parameters of user_rule, the stable Match fields, a rule of your own found through a Detector, the ValueError of a wrong argument, that LazyRegex is gone and the TypeError of a text that is not a str. tests/test_code_point_offsets.py pins the offsets on texts with emoji and combining characters. scripts/wheel_smoke.py makes a rule of its own through user_rule in a fresh venv.

The type vocabulary

maisecrets/detection/types.json lists every type that a Match.type of a built-in rule or pass can carry, and TYPES is the same list as a read-only mapping:

from maisecrets.detection import TYPES
TYPES["DE_TAX_ID"]       # {"family": "pii", "class": "pii.tax_id"}
TYPES["SECRET"]          # {"family": "secret", "class": "secret"}

• family is secret or pii. class is one of the classes of the benchmark (bench/cases.py), such as pii.phone, pii.national_id or pii.bank_account, or the family alone when the type does not say which kind it is (SECRET). The benchmark reads its classes from this file (bench/sources.py). The classes today are the classes of the benchmark, so several types under pii.national_id (postcodes, vehicle plates, tax and company numbers) are candidates for a finer class later.
• Stable: a type that a release lists is not renamed or removed without the deprecation path above. When its last rule goes, the entry stays with a deprecated field for at least one minor version. The family of a listed type never changes.
• May be refined in a minor release: the class of a listed type. The changelog names each type whose class moved.
• May grow in any release: new types (a new rule, a new region, a vendored rule set that brings a new entity), and new optional fields in an entry. A caller handles a type it does not know by its family and class from TYPES of the installed version: that version lists every type it emits.
• Not listed: the type of a rule of your own (user_rule) and of a term list or an imported value of the plugin; the caller or the person chose it. The terms and imports of the plugin take listed types only.

tests/test_type_vocabulary.py derives the emitted types from the rules of every region and label language and from the passes, and fails when a type is emitted and not listed, or listed and emitted by nothing (unless it is deprecated). A vendored rule-set update that brings a new type is therefore red until the type gets an entry.

The JSON entry point: maisecrets.detect/1

A caller in any language runs the core as a process: one JSON request on stdin, one JSON response on stdout.

python3 -m maisecrets.detection < request.json > response.json

It needs Python 3.9 or later and the wheel installed (or the plugin folder as the working directory), and no other package. It reads no home folder, no config and no network. Each call is one process; it loads the rules of its request and ends (about 0.1 s on an M-series laptop). A Python server uses the library in process instead.

The schema of the request and of the response is maisecrets/detection/detect-1.schema.json (JSON Schema 2020-12, $defs/request and $defs/response); it ships in the wheel next to the code. The contract follows docs/DETECTION-CONTRACT-PROPOSAL.md, with the keys the core fills today.

The request. A JSON object in UTF-8 (a byte order mark is allowed):

The order of regions and of languages does not matter. A key that the table does not name is an error, so a misspelt key never goes unnoticed. One key of the proposal is not in maisecrets.detect/1 yet: exclusions has no form in the core today (a reference form is a References object of six questions, not a list of spans). A caller drops a finding inside its own placeholders itself. A later release may accept it as a new optional key; a request without them keeps its meaning.

{"contract": "maisecrets.detect/1",
 "text": "{\"customer\": {\"telefon\": \"0171 4829173\", \"build_number\": \"20261021\"}}",
 "channel": "tool", "regions": ["generic", "de"], "languages": ["en", "de"]}

The response. One line of JSON with a fixed key order:

{"contract": "maisecrets.detect/1",
 "engine": {"name": "maisecrets", "version": "0.8.0", "ruleset": "sha256:…"},
 "findings": [{"start": 26, "end": 38, "type": "PHONE",
               "provenance": {"rule": "phone-after-phone-word", "origin": "builtin", "layer": "deterministic"}}]}

• start and end count Unicode code points of text, end exclusive. The value is text[start:end]; the response never carries it, so a secret stays out of the output stream. A UTF-16 caller (JavaScript, Java, .NET) counts a character outside the BMP, such as an emoji, as two units and converts.
• type is a type of maisecrets/detection/types.json (section "The type vocabulary"), with its family and class there. The findings are sorted by start, end, type and rule, and no two of them overlap.
• provenance is informative: rule is the rule id (Match.kind), and it may change with the rules; origin is builtin and layer is deterministic in this release. There is no confidence.
• engine.version is the version of the plugin or the wheel. engine.ruleset is a sha256 over the rule list the request loaded (each rule's id, type, pattern, keywords and checks, in scan order), the regions, the label languages and the year. It is computed from the loaded rules, not from the files: a checkout with CRLF line ends and the release tree give the same digest. The same request, version and ruleset give the same bytes.
• The year is the one input from outside the request: a birth date must lie before the current year (UTC). It is part of ruleset, so the digest changes on 1 January.

Errors. An error is a response too, on stdout, with a non-zero exit status. stderr stays empty.

{"contract": "maisecrets.detect/1", "error": {"code": "invalid_request", "path": "regions[1]", "message": "is no known region"}}

Exit status 0 is a response with findings. Any other status (1 from the interpreter itself, for example when the package is not found) means no response was written. The message of an error is a fixed text of the engine: it never holds a value of the request, not even the name of an unknown key, since a key is input as well.

tests/test_detection_entry_point.py runs the conformance cases (tests/conformance/cases.json) through the entry point and compares each response with the Python API, checks every response kind and the two examples above against the schema, runs one request twice in fresh processes and compares the bytes, and sends a secret-shaped value in each wrong field and checks that no output holds it. scripts/wheel_smoke.py calls the entry point from the installed wheel.

A parsed object: findings per string, few scans (since 0.8.10, provisional)

A caller that holds a parsed response (a dict from an API) can scan each string in it apart, after its key. A call has a fixed cost of about 0.6 ms, and a gateway response has 69 strings on average (prod, 7 days, 2026-11-02), so most of the time goes to the calls. maisecrets.detection.leaves.findings_by_leaf scans fewer texts and gives each finding back in the coordinates of its string:

from maisecrets.detection.leaves import findings_by_leaf
for leaf, m in findings_by_leaf(detector, response):     # layout "lines", the default since 0.8.11
    leaf.path, leaf.value[m.start:m.end], m.type          # path: dict keys and list indices, outermost first

• layout "lines" (the default since 0.8.11): one scan per record, each string on a line of its own after its last key, as written. A record is an item of the outermost list on a string's path; the strings in no list are one record of their own. So the scan of a string sees what a scan of that string after its key sees, plus the other strings of its record (last_name next to first_name), and nothing of another record. On the shapes of the gateway's check (a line break, JSON inside a string, a context word in a neighbouring record) it finds what each string alone after its key finds (tests/test_detection_leaves.py, LinesLayoutTests). A context of one field that speaks against a finding (a test marker, a company word, a template) counts only inside that field (tests/test_record_negative_context.py). leaves.RECORD_CONTEXT_PER_FIELD names the level of this guarantee and moves up each time a window that crossed a field is closed: 2 since 0.8.18 (the street after its label). A release without the constant is below 2. A string that its record already holds after the same label (Slack writes each message twice, in text and in blocks) is scanned once, and the copy gets the findings of the first (since 0.8.18). A value in an inline node of its own after a cue node (since 0.8.18): the text nodes of an Atlassian document paragraph are scanned also as one sentence, and the findings are cut into the nodes; a value found in one string of a response is masked in the other strings that hold it as a whole word (Slack's bold or code element after Passwort: in its blocks, found from the message's text field). A name or user name spreads with two words or more, any other value with 8 to 256 characters, a letter and a digit and none of the common shapes (a date or a time, host:port, file:line, __name__, @Annotation, a ticket key, a version): one word (Wolf, Sommer, a word after password:) is also a word of other strings. A string whose key names no secret (a cursor, component_key, a lone key, an id, token_expires_at, a revision) gets no spread.
• layout "json" (the default of 0.8.10): one scan of json.dumps(obj, ensure_ascii=False). In that text a line break is the two characters \n and a quote gets a backslash, so an address can run over a line end and a label inside a string is lost; and a context word of one record can give a finding in the next. The gateway's check of 0.8.10 found 4 such differences in 48 cases. json_text(obj) gives that text with the place of each string.
• The name of a person's role (since 0.8.12): under author, reviewer(s), assignee(s), reporter, creator, organizer, attendee(s), committer, approver, requester, merge_user, merged_by, closed_by, approved_by and requested_by, the key name, displayName or fullName is written as <role>_name (author_name: …), a person label of the core: a full name whose surname is also an English word is found there, which after the weak name read as words. A value with a word of an account, a group or a state (Deleted User, Platform Team, Plattform Mannschaft) stays text after such a key, in a scan of text too.
• The label of a string is its last key, unless that key could change the label line: a key that is empty or longer than 64 characters, holds a line boundary (\r, U+2028 and the others of str.splitlines) or a control character, holds one of : = | " ', or names the cursor of a paged list (nextPageToken, as the gateway). Such a string stands on its line without a label, and starts a scan of its own: after another string, that string's last line (password:) would read as its label. leaves.label(path) gives the label of a path, "" for none (since 0.8.16; _label stays an alias).
• A span is one of the string as the caller holds it; m.value is leaf.value[m.start:m.end]. A finding in a key, a label or the structure belongs to no string (leaf is None); one that runs past the end of its string is cut there; a span of length zero stays one, also at the end of a string. A string that is empty or white space only is not scanned and does not cut its record (it holds nothing to find). Placing the findings is one pass over the strings and the findings. A value JSON cannot hold raises TypeError, a cycle ValueError.
• tests/test_detection_leaves.py holds these points. So a password and a first and last name reach such an element; a single surname alone in it stays as it is.

A deadline: stop a scan in time (since 0.8.24)

Detector.findings(text, ..., deadline=) and leaves.findings_by_leaf(detector, obj, ..., deadline=) take a time.monotonic() value, as findings_with() does for the remote step. Without it nothing changes.

Past the deadline the scan stops between rules, between the hits of a rule (since 0.8.25), between its steps (the personal data, the cleaned copy, the terms) and between record segments. It raises maisecrets.detection.ScanIncomplete:

• findings: what it found so far, in the call's return shape and offsets (Match for findings(), (leaf, Match) for findings_by_leaf());
• reason: "deadline";
• read (findings_by_leaf() only): the leaves whose findings in findings are complete. Their segment ran to its end. A copy of a string, and the nodes of a joined sentence whose sentence was not read, are not in it. A value found only in a leaf outside read is not spread into the leaves in it. None for findings().

The text after the last finding is not scanned by every rule. A caller treats the rest as unscanned, for example with its own patterns.

One rule's regex on one text cannot be interrupted, so a scan can pass its deadline by its slowest single step. A References you give is asked during the work on each hit: a slow one adds its own time per hit, but past the deadline only the hit in work still asks it (before 0.8.25 every hit of the running rule did). Measured on the reference host (AMD EPYC 9645, 2026-11-05), five runs per budget of 0.05 to 1 s:

A remote detector (optional)

The rules find few names (docs/NER-EXPERIMENT.md). A named-entity service can add them after the rules, through maisecrets/detection/remote.py. It is off unless a caller gives one, and the entry point above never asks one: it reads no network.

from maisecrets.detection import remote

ner = remote.RemoteDetector("http://127.0.0.1:8765/", format="hf", model="d3-onnx-fp32@d421f57d", timeout=0.3)
result = detector.findings_with(text, channel="tool", remote=ner)
result.findings          # the findings of the rules as they are, and the spans of the service (Match.score set)
result.remote.state      # "ok", "unavailable" (result.remote.reason: timeout, refused, network, policy, http, answer), "too_long"
response = contract.result_response(detector, result)    # the same as JSON (maisecrets.detect/1)

• Offsets of the answer are code points of the text, as a Python service counts them (str indices). A service that counts UTF-16 units or bytes masks the wrong place after a character outside the BMP; the rules' findings stay, the name does not get masked.
• Wire formats. hf: Hugging Face token classification, {"inputs": text} in, [{"entity_group" (or entity), "start", "end", "score"}] out. presidio: Presidio Analyzer, POST /analyze with {"text", "language"}, [{"entity_type", "start", "end", "score"}] out. A label B-PER or I-PER counts as PER.
• Classes. classes maps a class of the service to a type of types.json. The default is PER and PERSON to NAME, the one class the NER experiment measured; an operator adds more ({"LOC": "ADDRESS"}). min_score sets the least score per type; there is none by default, and no threshold here was chosen from a benchmark.
• A product name is no person. A name span that remote.is_product_name(span) accepts (a word of PRODUCT_WORDS alone, or one followed only by words of PRODUCT_WORDS and PRODUCT_TAILS, as "Claude Code") is dropped, and so is a word of GREETING_WORDS alone. A caller that applies the spans itself calls the same function (since 0.8.15).
• It only adds. A span is kept only where no finding of the rules and no span kept before it lies. A finding of the rules is never removed, made smaller, moved or retyped. A span that ends inside a word is widened to the whole word.
• The service never sees a finding of the rules. Each one goes out as filler (_) of the same length, so the offsets of the answer fit the text. mask=False sends them as they are: the filler costs the model of the NER experiment 21 of 2,482 names on the V3 samples (docs/NER-EXPERIMENT.md, "The remote path"). The plugin allows it only for a service on loopback or a Unix socket.
• A failing service changes nothing. Slow (over timeout, or past the deadline of findings_with, one budget for several texts), refused, an HTTP error, or an answer with any item it cannot read: the findings of the rules alone, and result.remote says unavailable and why. A text longer than max_chars is not sent (too_long). Nothing raises.
• No network unless asked. The URL must be a loopback address (127.0.0.0/8, ::1, localhost) or a Unix socket (unix:/path), else RemoteDetector raises ValueError; allow_network=True allows any http or https host. A host name is looked up at each request: without allow_network every address must be loopback, and the connected peer is checked again before a byte is sent, so a localhost that a hosts file sends elsewhere is refused.
• The deadline holds for the whole request. Name lookup, connect, send and the answer run in a thread that the caller leaves at the deadline, so a service that sends its answer a byte at a time cannot hold the caller. A wrong setting (classes, min_score, timeout, max_chars) raises ValueError when the detector is made.
• The model version is the operator's word. Neither format names its model in the answer, so model is what the operator states, and engine.remote.model of a response is only as true as that setting. The reference service (examples/ner_service.py) reports its own at GET /info, so an operator can compare. Its model (D3) is AFL-3.0 and its base model Apache-2.0: an image that contains them keeps their notices (docs/NER-EXPERIMENT.md, "Licence of D3").
• More classes than names. {"inputs": text} asks the reference service for persons only. With "parameters": {"labels": [...]} it answers the spans of those classes, each with its class in entity_group; a class that GET /info does not list in labels is a 400. RemoteDetector sends the labels only with ask_labels=True (the Hugging Face API has no such parameter) and when classes maps a class beyond PER/PERSON, so any other service gets the request it always got. GET /health answers {"status": "ok"}. The image mcpgate/ner runs this service, and in the plugin /maisecrets:ner sets it up (README, "Names from a service").
• In a response (contract.result_response), a finding of the service has score and provenance.origin and layer model; engine.remote holds format, model, state and, when unavailable, reason. Without a remote detector the response is the one of the entry point. The schema has both fields as optional.

These names are new and provisional: RemoteDetector, RemoteStatus, Result, Detector.findings_with, contract.result_response and Match.score (None for a finding of the rules). They are not in __all__; a change to them goes into the changelog under "API" without a deprecation until a minor release makes them stable. tests/test_remote_detector.py holds the points above against a fake service of each format.

The old names

maisecrets.detect and maisecrets.checksums, lazy_regex, normalise, regions, registry, rulespec, rule_filter and personal stay. Each is the same module object as its module in maisecrets/detection/, so an import or a patch through an old name reaches the core itself. The plugin and its tests use them. detect.scan(), detect.findings() and the other functions use the default detector, which use_regions, use_sources and use_references configure.

The boundary tests

scripts/detection_fingerprint.py records the hits of 27,337 inputs. A change that moves the code and not the rules leaves the fingerprint byte-identical.

A language-neutral spec

docs/DETECTION-SPEC-DRAFT.md measures how far the core is from a spec that a second implementation could follow: what is data today, what is Python, about 31 semantic primitives, and a case format that tells normative cases from snapshot records. It is an analysis; nothing in the runtime depends on it.