You describe the entity types you want, in plain English, with any rules about what counts and what doesn't. The model finds and labels the spans (extract mode), or labels spans you already have (label mode). To change its behaviour, you change the rulebook, not the weights; nothing is retrained.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PERSON
[DESCRIPTION] A person's name.

[TYPE] ACCOUNT_ID
[DESCRIPTION] An account or customer number.

[TYPE] EMAIL
[DESCRIPTION] An email address.

[TYPE] PHONE
[DESCRIPTION] A phone number.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PRODUCT
[DESCRIPTION] A named software product or app.

[TYPE] VERSION
[DESCRIPTION] A version number.

[TYPE] DEVICE
[DESCRIPTION] A named hardware model.

[TYPE] ERROR_CODE
[DESCRIPTION] An error code.

At a glance

Quickstart

pip install redefiner

Python 3.10 or newer. Runs on CPU, or on an NVIDIA GPU if your PyTorch build has CUDA (PyTorch's install selector gives the command for your system). On Linux with a GPU, install "redefiner[gpu]" instead: it adds flash-linear-attention's kernels, without which the GPU path is much slower. On Linux without a GPU, add --extra-index-url https://download.pytorch.org/whl/cpu to skip PyTorch's CUDA build, a download of several GB.

from redefiner import Tagger

tagger = Tagger.from_pretrained("jungroo-ai/redefiner-0.8b")   # GPU if available, else CPU; device="cpu" forces CPU
print(tagger.device)   # cpu or cuda:0; a warning says so when an NVIDIA GPU is present but PyTorch can't use it

rulebook = """[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PERSON
[DESCRIPTION] A person's name.
PE1 Label the full name; leave out titles such as "Dr" or "Prof".

[TYPE] ORGANIZATION
[DESCRIPTION] A company, university or other institution.
"""

tagger.extract("Prof Ada Lovelace corresponded with the Royal Society.", rulebook)
# [{'start': 5, 'end': 17, 'text': 'Ada Lovelace', 'label': 'PERSON', 'confidence': 0.977},
#  {'start': 40, 'end': 53, 'text': 'Royal Society', 'label': 'ORGANIZATION', 'confidence': 0.489}]

# Without rule PE1, the same call keeps the title:
without_pe1 = rulebook.replace('PE1 Label the full name; leave out titles such as "Dr" or "Prof".\n', "")
tagger.extract("Prof Ada Lovelace corresponded with the Royal Society.", without_pe1)
# [{'start': 0, 'end': 17, 'text': 'Prof Ada Lovelace', 'label': 'PERSON', 'confidence': 0.606},
#  {'start': 40, 'end': 53, 'text': 'Royal Society', 'label': 'ORGANIZATION', 'confidence': 0.424}]

Each span's confidence is its span probability times its type probability. Two settings decide which spans come back: threshold (default 0.5) is the minimum span probability, and none_bias (default 2) is subtracted from the score of "none of these types" before the type is picked, so borderline spans get a type instead of being dropped. The confidence is computed without that bias, so a returned span can score below 0.5. Above, "Royal Society" is almost certainly a span (0.997), but the model weighs ORGANIZATION (0.491) and none of the types (0.508) about equally: 0.997 × 0.491 = 0.489. Filter on confidence if you only want the spans the model is sure of.

Label mode, when you already have the spans:

text = "Ada Lovelace corresponded with the Royal Society."
tagger.label(text, [(0, 12), (35, 48)], rulebook)
# [{'start': 0, 'end': 12, 'text': 'Ada Lovelace', 'label': 'PERSON', 'confidence': 1.0,
#    'alternatives': [('ORGANIZATION', 0.0)]},
#  {'start': 35, 'end': 48, 'text': 'Royal Society', 'label': 'ORGANIZATION', 'confidence': 0.997,
#    'alternatives': [('PERSON', 0.003)]}]

Here confidence is the probability of the chosen type among the rulebook's types. Both calls also take a list of texts (and, for label, one list of spans per text) and return one result list per text.

Writing your own rulebook: start from a template (a blank layout; PII, news, product reviews and scientific text; and a skeleton for types that depend on context), and see How to write rules that work.

See a rule at work

Every card on this page is real output: the released int8 model on a laptop CPU, with nothing retrained between runs. Only the rulebook changes. Green lines were added to the rulebook; spans outlined in red disappear, spans outlined in green are new or changed. Try your own text and rules in the demo.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] BRAND
[DESCRIPTION] A brand name.

[TYPE] PRODUCT
[DESCRIPTION] A product line or model name.

[TYPE] COLOR
[DESCRIPTION] A colour.

[TYPE] SIZE
[DESCRIPTION] A size.

[TYPE] PRICE
[DESCRIPTION] A price or a price limit.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] BRAND
[DESCRIPTION] A brand name.

[TYPE] PRODUCT
[DESCRIPTION] A product line or model name.

[TYPE] COLOR
[DESCRIPTION] A colour.

[TYPE] SIZE
[DESCRIPTION] A size.

[TYPE] PRICE
[DESCRIPTION] A price or a price limit.
PR1 A price limit keeps its limit word in the span: "under $50", "below 30 dollars", "less than $200".

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] COMPANY
[DESCRIPTION] A company name.

[TYPE] PRODUCT
[DESCRIPTION] A named product or device.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] COMPANY
[DESCRIPTION] A company name.

[TYPE] PRODUCT
[DESCRIPTION] A named product or device.

[TYPE] NOT_AN_ENTITY
NE1 Our own company and products: "Acme", "Acme Home". Never label them.

The next card uses an example rulebook for a travel desk, with 11 types, written the way the model reads best: general rules, precedence rules that settle conflicts, numbered type rules with "query" -> span examples, and what never to extract. "Before" is the same rulebook with only its types and descriptions.

On 21 test queries the full rulebook got 19 exactly right, against 11 with types and descriptions only and 9 with type names only. Both misses came from one rule: a single code with the wrong name ("my ticket number is ZK48TQ") keeps the type of its name; the format rule works when both kinds of code appear together. The layout is explained in Structure for hard rules, and docs/templates/hard-rules.txt is a blank copy to fill in.

[RULES] Travel desk rulebook. Label each entity in the traveller's message with one of the types below.
G1 Label all entities in the message, not only the first one.
G2 Keep each span as short as the entity itself ("a flight to Oslo" -> "Oslo").
G3 When two types fit, the precedence rules below decide.
[PRECEDENCE]
P1 An airport name or a three-letter airport code is AIRPORT, never ORIGIN or DESTINATION, even after "from" or "to".
P2 Direction decides between ORIGIN and DESTINATION: a place after "from", "out of" or "leaving" is ORIGIN; after "to", "into", "arriving in" or "back to" is DESTINATION.
P3 The person who will fly is TRAVELLER; the person who arranges or pays is BOOKER.
P4 A date after "leave", "depart" or "out on" is DEPARTURE_DATE; after "return", "back on" or "fly home" is RETURN_DATE.
P5 An airline name followed by a programme word ("Miles", "Club", "Rewards") is LOYALTY_PROGRAM; any other airline name is AIRLINE.
P6 The reference format decides, whatever the traveller calls it: a code that mixes letters and digits is BOOKING_REF, even when called a ticket number; a long code of digits only is TICKET_NUMBER, even when called a booking reference.
[TYPE] TRAVELLER
[DESCRIPTION] A person who will take the trip.
TR1 "Book Wanda on the morning flight" -> "Wanda"; "seats for Ilse and Gus" -> "Ilse", "Gus".
[TYPE] BOOKER
[DESCRIPTION] A person who arranges or pays for someone else's trip.
BK1 "Petra is booking for her son Leon" -> "Petra" is BOOKER, "Leon" is TRAVELLER.
BK2 "charge it to Viktor's card" -> "Viktor".
[TYPE] ORIGIN
[DESCRIPTION] The city or country the trip starts from.
OR1 "from Porto" -> "Porto"; "leaving Seoul" -> "Seoul".
[TYPE] DESTINATION
[DESCRIPTION] The city or country the trip goes to.
DS1 "to Nairobi" -> "Nairobi"; "back to Quito" -> "Quito".
[TYPE] AIRPORT
[DESCRIPTION] A named airport or a three-letter airport code.
AI1 "from YYZ" -> "YYZ"; "into Changi" -> "Changi".
[TYPE] DEPARTURE_DATE
[DESCRIPTION] The date the outbound trip starts.
DD1 "leave on 4 June" -> "4 June"; "I went out on 2 May" -> "2 May".
[TYPE] RETURN_DATE
[DESCRIPTION] The date of the return trip.
RD1 "back on 11 June" -> "11 June".
[TYPE] AIRLINE
[DESCRIPTION] An airline the traveller flies with.
AL1 "on Skyvane" -> "Skyvane".
[TYPE] LOYALTY_PROGRAM
[DESCRIPTION] An airline's points or membership programme.
LY1 The span keeps the programme word ("my Skyvane Club number" -> "Skyvane Club").
[TYPE] BOOKING_REF
[DESCRIPTION] A booking reference: a short code of letters and digits.
BR1 "reference QX7PLM" -> "QX7PLM"; "my ticket number is HB2RWA" -> "HB2RWA" is BOOKING_REF.
[TYPE] TICKET_NUMBER
[DESCRIPTION] An e-ticket number: a long code of digits only.
TN1 "ticket 2201234567890" -> "2201234567890"; "booking ref 1259876543210" -> "1259876543210" is TICKET_NUMBER.
[NOT_AN_ENTITY]
NE1 Places the traveller wants to avoid ("not via Doha", "no stop in Frankfurt").
NE2 Request phrases ("can you", "please") and generic words ("flight", "seat", "trip").

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] ORDER_ID
[DESCRIPTION] An order number.

[TYPE] INVOICE_ID
[DESCRIPTION] An invoice number.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] ORDER_ID
[DESCRIPTION] An order number.

[TYPE] INVOICE_ID
[DESCRIPTION] An invoice number.
[RULES]
The format decides, whatever the customer calls it: a reference starting with # is ORDER_ID ("#B-11872"), even when the customer calls it an invoice; a reference starting with INV- is INVOICE_ID ("INV-2025-0099"), even when the customer calls it an order. This overrides everything above.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PERSON
[DESCRIPTION] A person's name.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] PERSON
[DESCRIPTION] A person's name.
PE1 Also label the pronouns that refer to a person: "she", "he", "they", "her", "him", "them".

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] SKILL
[DESCRIPTION] A technical skill: a programming language, framework, platform or tool.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] SKILL
[DESCRIPTION] A technical skill: a programming language, framework, platform or tool.

[TYPE] LANGUAGE
[DESCRIPTION] A programming language, such as "Java" or "Rust".
LA1 Programming languages are LANGUAGE, never SKILL.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] COMPANY
[DESCRIPTION] A company name.

[TYPE] PERSON
[DESCRIPTION] A person's name.

[TYPE] PRODUCT
[DESCRIPTION] A product name.

[TYPE] DATE
[DESCRIPTION] A date or day.

[RULES]
Extract every entity of the supplied types. Return exact spans from the text.

[TYPE] COMPANY
[DESCRIPTION] A company name.

[TYPE] PERSON
[DESCRIPTION] A person's name.

[TYPE] PRODUCT
[DESCRIPTION] A product name.

[TYPE] DATE
[DESCRIPTION] A date or day.

[TYPE] TICKER
[DESCRIPTION] A stock ticker symbol, such as "TSLA" or "MSFT".

The quoted examples inside the rules ("under $50", "Java", "TSLA", "#B-11872") are not in the texts: the model applies the rule, it doesn't copy it. Hard rules work when they are concrete, as here; a rule that asks the model to judge what didn't happen ("only cities the writer actually visited, not ones they plan to") is not followed yet. See Limitations. To regenerate the cards, run scripts/build_rule_cards.py; it stops if any output differs from what the cards show.

Benchmarks

Entity-level F1 (exact boundary and type) unless marked; S1MB is accuracy. Full tables, protocols and caveats are on the benchmark page, with every rulebook it reports in benchmarks/rulebooks/.

All numbers use one fixed setting (int8, span threshold 0.5, NONE bias 2, 256-token query cap). Baseline numbers are published results unless marked as our run.

The 13 GoLLIE zero-shot NER sets, with GoLLIE's own guidelines

GoLLIE's guideline text and five examples per type, converted into a rulebook with nothing added; scored with GoLLIE's code. ReDefiNER scores are the mean over 3 orders of the types in the rulebook; GoLLIE scores are from its paper.

ReDefiNER leads GoLLIE 34B on 9 of the 13 sets with about 1/40 of its parameters. The largest loss is WikiEvents, which labels pronouns and common nouns under the ACE05 scheme GoLLIE was trained on.

Open NER: the same 7 CrossNER and MIT test sets

Mean F1 over 6,470 sentences, with each type given as a one-line description or by its name only. For our runs of other models, the first number uses the model's default threshold and the second a threshold chosen on the dev sets; ReDefiNER uses its one fixed release setting.

ReDefiNER leads when types are described, also against the tuned baselines. With names only it trails GLiNER large, NuNER Zero-span and GNER. Descriptions hurt both NuNER models, which were trained on short type names; their labels are lower-cased as NuNER's card requires. NuNER Zero's own reported 68.0 on these sets is token-level F1, which gives credit for partly right spans; scored that way, our run gives 67.9. The Policy-Linter takes free-text rules at inference, like ReDefiNER, but scores each word on its own, so multi-word names often break apart.

S1MB span labeling: Few-NERD and Gretel PII

Accuracy on the benchmark's own cases. Rows not marked "our run" are from the S1MB leaderboard. The d1 decision models got S1MB's records as they are, through their system_one interface. Settings, prompts and every other caveat are on the benchmark page.

Rulebooks

A rulebook has an optional [RULES] section with rules for every type, then one [TYPE] section per type: a one-line [DESCRIPTION] and any number of rule lines (a type with two sections is an error). An optional [TYPE] NOT_AN_ENTITY section lists what is never tagged. The same rulebook can be passed as a dict:

rulebook = {
    "general": ["Extract every entity of the supplied types."],
    "classes": [
        {"name": "PERSON", "description": "A person's name.",
         "rules": ['PE1 Label the full name; leave out titles such as "Dr" or "Prof".']},
        {"name": "ORGANIZATION", "description": "A company, university or other institution."},
    ],
    "never": ["Pronouns."],
}

Type names use letters, digits and underscores (DATE_OF_BIRTH, not DATE-OF-BIRTH), on a line of their own: [TYPE] NAME. A line that starts with [TYPE] but is written any other way is an error, so a type is never dropped silently. Names are case-sensitive and come back exactly as written, so Person and PERSON are two types. To see what was read, parse(to_text(rulebook)) (both from redefiner.rulebook) returns the layout and the type names, e.g. ("uniform", ["PERSON", "ORGANIZATION"]).

Rule lines.

• One rule per line. Don't wrap a long rule onto continuation lines: on CrossNER AI, wrapping the long rules of two rulebooks lowered F1 by 0.8 and 0.5.
• Start each rule with a short ID: G1 in [RULES], the type's letters under a type (PE1), NE1 under NOT_AN_ENTITY. IDs let rules refer to each other ("the precedence rules below decide"). They are optional: removing them from the same two rulebooks changed F1 by −0.02 and +0.6. Every rulebook in this repo uses them. In the dict form, write the ID in the string; to_text doesn't number rules.
• A rule belongs to the section it sits in, not to its ID. Everything under [TYPE] X up to the next [TYPE] or [RULES] line is X's. Put [DESCRIPTION] on the line right after [TYPE], and keep it to one line.
• Quoted strings under a type are read as examples of that type, even inside a "never" sentence, so put exclusions in [RULES] or NOT_AN_ENTITY (why).
• Rules that apply after all the types, such as which rule wins when two disagree, go under a second [RULES] line at the end. In the dict form that is the "closing" list.

Two layouts. The layout above has [RULES] on a line of its own and [TYPE] NOT_AN_ENTITY for what is never tagged. The other, used by hard-rules.txt and the travel-desk example, has text on the [RULES] line, [PRECEDENCE] rules before the types, and a final [NOT_AN_ENTITY] section. Use one or the other; a rulebook that mixes them (a [RULES] line of its own and a [NOT_AN_ENTITY] section) is an error. Windows line endings are fine.

Templates to copy are in docs/templates/.

What works, from the benchmarks (see How to write rules that work):

• Descriptions matter. With a one-line description per type the model scores 7 points higher than with type names alone on our open-NER sets; always write them.
• Boundary, exclusion and convention rules work. "The span includes the @ of a Twitter handle" took BroadTwitter from 45.5 to 70.6 F1. "Never label Houston, Texas or Harvey" cut those false positives from 389 to 5 on HarveyNER.
• Rules that fight a strong prior don't. Asking it to label a person's name as something other than a person, or to label common nouns as entities, mostly has no effect. Pronouns are the exception: listed under the type, they are labelled (the "Count pronouns as people" card above). See Limitations.

Label mode

label(text, spans, rulebook, allow_none=False, snap=False)

• Spans are character offsets into the original text, start inclusive, end exclusive, as (start, end) or {"start": .., "end": ..}.
• Each span gets the best type and its confidence, plus the next two alternatives.
• allow_none=False (default) forces a choice among the rulebook's types. allow_none=True lets the model answer "NONE" for a span that isn't any of them.
• A span the model can't read comes back with "unsupported": reason instead of a label, never dropped and never changed silently:
  • it doesn't start and end on token boundaries ("would be snapped"), e.g. part of a word. Pass snap=True to label it widened to whole tokens; the result then carries the widened offsets and "snapped": True;
  • it is longer than 16 tokens;
  • its offsets are invalid.

Long texts

The model reads up to 256 tokens of text per pass. Longer texts are split automatically into overlapping windows, cut at sentence ends where possible:

• in extract mode, where two windows disagree about a span, the one further from a window edge is kept;
• in label mode, each window is placed around the spans it labels.

Each window is read on its own, so the same name can get different labels in different parts of a long document. If consistency matters, label each distinct string once with label and apply the result everywhere.

Speed

CPU, int8 weights, on CrossNER AI test sentences (about 32 tokens each). A rulebook is encoded with the first text and then cached (the last 8 rulebooks are kept), so each later text only pays for itself.

Apple M2 (8 cores):

Linux cloud server (Ubuntu 24.04, Intel Xeon 2.3 GHz, 4 cores, no GPU, Python 3.13), measured by a tester:

Peak memory is reached with 16 texts in one call; on the Linux server the process used 1.1 GB right after loading and 1.4 GB after the first text. Sending texts in one call is faster than one at a time on the M2 and on a GPU, but saves little on the 4-core server. On CPU the package replaces one slow PyTorch operation (a depthwise convolution with no MKLDNN fast path) with an exact equivalent; that alone is a 7–14× speedup on Macs.

On one NVIDIA A10 (int8 weights, fp32 compute, flash-linear-attention 0.5.2 installed, as redefiner[gpu] does), with a 7,700-character rulebook of 16 types (about 2,000 tokens), measured through our benchmark server, HTTP included:

GPU memory use was not measured.

HTTP server

redefiner serve --model jungroo-ai/redefiner-0.8b --port 8000

It listens on 127.0.0.1 (--host to change) and handles one request at a time. Both endpoints take a list of texts and return one result list per text. The rulebook is text or, as here, the dict form of to_text:

RULEBOOK='{"general": ["Extract every entity of the supplied types. Return exact spans from the text."],
           "classes": [{"name": "PERSON", "description": "The name of a person."},
                       {"name": "ORGANIZATION", "description": "A company, university or other institution."}]}'

curl -s localhost:8000/extract -d "{\"rulebook\": $RULEBOOK, \"texts\": [\"Ada Lovelace corresponded with the Royal Society.\"]}"
# {"results": [[{"start": 0, "end": 12, "text": "Ada Lovelace", "label": "PERSON", "confidence": 0.989},
#               {"start": 35, "end": 48, "text": "Royal Society", "label": "ORGANIZATION", "confidence": 0.431}]], "ms": 2101}

curl -s localhost:8000/label -d "{\"rulebook\": $RULEBOOK, \"texts\": [\"Ada Lovelace corresponded with the Royal Society.\"], \"spans\": [[[0, 12], [35, 48]]]}"
# {"results": [[{"start": 0, "end": 12, "text": "Ada Lovelace", "label": "PERSON", "confidence": 0.999, "alternatives": [["ORGANIZATION", 0.001]]},
#               {"start": 35, "end": 48, "text": "Royal Society", "label": "ORGANIZATION", "confidence": 0.994, "alternatives": [["PERSON", 0.006]]}]], "ms": 302}

A span that can't be labelled comes back with an unsupported reason in place of a label (see Label mode). A malformed request gets HTTP 400 and an error message.

The server has no authentication and no limit on request size. Keep it on 127.0.0.1, or put it behind your own proxy if it has to be reachable from other machines.

Limitations

• Rules against a strong prior are mostly ignored. Relabelling a person's name as something other than a person, asking for common nouns as entities, or "only revenue amounts, not every money amount" mostly has no effect. Pronouns do work when they are quoted in a short list under the type.
• Types that depend on context need structure. Precedence rules and numbered rules with worked examples help (see Structure for hard rules); rules that need counting or judging what didn't happen are still mostly ignored.
• Names-only rulebooks are weaker than descriptive ones; always write a one-line description per type.
• Abstaining is weak. When a type's condition isn't clearly met, the model tends to label anyway.
• Spans up to 16 tokens; text is read 256 tokens at a time, and longer texts are split into overlapping windows.
• English only; tested on English benchmarks only.

Intended use

ReDefiNER is meant for extracting entities from English text when you need control over what counts: your own entity types, document fields, search-query slots, and research on rule-following extraction.

• Check outputs before acting on them in legal, medical, financial or other high-stakes work. The model can miss or mislabel spans, and its confidences are model scores, not guarantees.
• Don't rely on it as the only safeguard when finding personal data for redaction or compliance; it will miss some.
• Don't use it to profile or track people without a legal basis and their consent.

Repository

Development

git clone https://github.com/jungroo-ai/redefiner && cd redefiner
pip install -e ".[dev]"
hf download jungroo-ai/redefiner-0.8b --local-dir weights   # optional: enables the end-to-end tests
pytest
ruff check .

To check a published number on your hardware: python benchmarks/reproduce/reproduce.py --model weights (CrossNER AI for extract mode, S1MB Few-NERD for label mode, each compared with the A10 run).

Licence

Apache-2.0, for the code and the weights. The weights are a modified version of Qwen3.5-0.8B (Apache-2.0); NOTICE in the model repo lists the modifications. The GoLLIE guideline text in benchmarks/ comes from GoLLIE (Apache-2.0). The benchmark datasets keep their own licences: test sentences and quoted examples from them (in benchmarks/ and the demo) are covered by those licences, not by Apache-2.0; see Datasets and licences.

Citation

@misc{redefiner2026,
  title  = {ReDefiNER: Rule-Defined Named Entity Recognition},
  author = {{Jungroo AI Labs}},
  year   = {2026},
  url    = {https://huggingface.co/jungroo-ai/redefiner-0.8b},
  note   = {Release 1.0}
}

Credits

Created by Jungroo AI Labs.