Wikidata NER Classifier 0.7.0

Classification into one shared retrieval-oriented hierarchy through three complementary paths:

1. WikidataNERClassifier classifies Wikidata items deterministically from P31/P279 token clues and optional descriptions.
2. OpenRouterNERClassifier infers the type of one target mention from free text, a structured record, or tabular context using an LLM.
3. HierarchicalNERPredictor routes unlinked free-text mentions and table columns/cells through deterministic candidate generation, with an optional contrastive LLM resolver over at most ten fine types.

All paths return classes from the packaged hierarchy:

coarse_type -> fine_type -> subtype -> specific_type

Source-first hierarchical prediction

HierarchicalNERPredictor is the pre-retrieval API. It never links an entity or returns a QID. Coarse and fine vocabularies, subtype parents, facets, and derived retrieval types all come from the packaged rule files through HierarchyIndex.

Free text:

from wikidata_ner import HierarchicalNERPredictor

predictor = HierarchicalNERPredictor()  # deterministic; no LLM is required
prediction = predictor.predict_free_text(
    "Chrysler Cirrus",
    "The Chrysler Cirrus is a mid-size four-door sedan model.",
)

assert prediction.retrieval_path == (
    "PRODUCT",
    "VEHICLE_WEAPON_OR_EQUIPMENT_MODEL",
    "CAR_MODEL",
)
assert prediction.retrieval_key == (
    "PRODUCT/VEHICLE_WEAPON_OR_EQUIPMENT_MODEL/CAR_MODEL"
)

Table column and cell:

stations = [
    "Roma Termini",
    "Milano Centrale",
    "Napoli Centrale",
    "Bologna Centrale",
]

column_prediction = predictor.predict_table_column(
    "Departure station",
    stations,
    neighboring_headers=["Arrival station", "Duration"],
    table_title="Italian high-speed train connections",
)
assert column_prediction.retrieval_path == (
    "FACILITY",
    "TRANSPORT_STATION",
    "RAILWAY_STATION",
)

cell_prediction = predictor.predict_table_cell(
    "Roma Termini",
    column_header="Departure station",
    row_context={
        "Departure station": "Roma Termini",
        "Arrival station": "Milano Centrale",
        "Duration": "3h 10m",
    },
    same_column_values=stations,
    neighboring_headers=["Arrival station", "Duration"],
    table_title="Italian high-speed train connections",
    column_prediction=column_prediction,
)

Column samples and row fields are bounded deterministically. Column predictions are cached and act as soft cell priors; strong cell evidence can override them. The generated retrieval plan always retains item_category: ENTITY:

print(prediction.retrieval_plan.to_dict())
# {
#   "mode": "fine_type_filter_specific_type_boost",
#   "filters": {
#     "item_category": "ENTITY",
#     "coarse_type": "PRODUCT",
#     "fine_type": "VEHICLE_WEAPON_OR_EQUIPMENT_MODEL",
#   },
#   "boosts": {"ner_specific_types": ["CAR_MODEL"]},
#   ...
# }

To add contrastive resolution, inject LLMTypeResolver(OpenRouterClient(...)). The resolver receives the target context and a general zero-shot rule. The bounded controlled labels appear only as response-schema enums; type cards, definitions, examples, and hierarchy paths are not placed in the prompt. Application code validates the semantic decision and constructs every retrieval field locally. The main configuration dataclasses are InputContextConfig, CandidateScoringConfig, HierarchicalPredictorConfig, RetrievalPolicyConfig, HierarchyCompatibilityConfig, and CandidateRankingConfig.

The dependency-free similarity score is deliberately lightweight. Applications can inject a semantic-similarity callback or replace the in-memory hierarchy index with a vector-backed implementation without changing the predictor API.

Fast input prediction with Cerebras through OpenRouter

For free text and tabular inputs, the Cerebras-hosted LLM makes the prediction from the target and its context with a general zero-shot semantic rule. The packaged hierarchy is deliberately not serialized into the prompt. Returned coarse types, fine types, subtypes, and facets are validated locally, and specific types plus retrieval paths are constructed by the application.

export OPENROUTER_API_KEY='...'

from wikidata_ner import OpenRouterNERClassifier

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
)
prediction = classifier.predict_text(
    "Rome Against Rome is a 1964 sword-and-sandal film.",
    mention="Rome Against Rome",
)

assert prediction.fine_type == "FILM"
print(prediction.specific_type)  # SWORD_AND_SANDAL_FILM
print(prediction.usage)

The general rule is deliberately ontology-agnostic:

Classify the target by what it denotes in context. Prefer an explicit target-bound type or description, then target-bound relations and structure, and treat surface form or general knowledge as weak evidence. Choose the most specific schema-admitted type clearly supported; otherwise choose a broader admitted type or abstain.

Inference remains two-stage to keep Cerebras schemas small: first select a broad coarse type, then a fine type and optional refinements within that branch. The system prompts contain no taxonomy index, rule definitions, examples, clue lists, or candidate cards. Controlled IDs live in the strict response schema, and Python validates parentage and builds ner_retrieval_key, ner_retrieval_path, ner_retrieval_tags, and specificity fields. Existing ner_*, prior, popularity, URL, QID, and previous coarse/fine fields are stripped from structured-record prompt input; fields such as label, labels, aliases, types, and description remain available as semantic evidence.

Multi-mention inference

predict_many() batches independent mention/context pairs into shared model requests. Each request reuses the same general zero-shot rule; no hierarchy is serialized into the batch prompt. Controlled coarse/fine labels remain in the strict output schema and are checked again locally. Use MentionTask when contextual text needs an explicit target:

from wikidata_ner import (
    MAX_MENTIONS_PER_BATCH,
    MentionTask,
    OpenRouterNERClassifier,
)

assert MAX_MENTIONS_PER_BATCH == 8

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
    max_mentions_per_batch=8,
    max_batch_characters=80_000,
)

predictions = classifier.predict_many(
    [
        MentionTask(
            data="Rome Against Rome is a 1964 sword-and-sandal film.",
            mention="Rome Against Rome",
        ),
        MentionTask(
            data="Ada Lovelace was an English mathematician and writer.",
            mention="Ada Lovelace",
        ),
        MentionTask(
            data={"label": "Dune", "description": "1965 science-fiction novel"},
        ),
    ],
)

assert [prediction.fine_type for prediction in predictions] == [
    "FILM",
    "HUMAN",
    "BOOK_OR_WRITTEN_WORK",
]

coverage = classifier.hierarchy_coverage_report()
assert coverage["complete"] is True
assert coverage["prompt_embeds_hierarchy"] is False
assert coverage["strategy"] == "zero_shot_semantic_routing_local_validation"
assert coverage["source_file"] == "B_full_rule_spec.json"
assert coverage["coarse_type_count"] == 21
assert coverage["fine_type_count"] == 187

For each chunk, the classifier makes one shared coarse request and then one shared fine/refinement request for each coarse branch present. If eight mentions resolve to two coarse branches, this is three requests instead of the 16 requests made by individual two-stage prediction. Results retain input order; numbered task IDs keep identical surface forms with different contexts separate.

The absolute MAX_MENTIONS_PER_BATCH is 8. This keeps the controlled coarse and fine enums inside Cerebras's expanded strict-schema budget even without a hierarchy catalog in the prompt. The constructor can set a lower instance ceiling with max_mentions_per_batch; omitting batch_size then uses that ceiling. Longer iterables are chunked automatically, and the configured prepared-context budget may split a chunk earlier. A method call cannot exceed the instance ceiling or the hard library ceiling. A single large task is still sent alone. Set batch_size=1 to use the individual-request path.

Batch outputs use a fixed object with one required key per task and a shared $defs result schema. This is intentional: Cerebras strict output supports schema references but not minItems/maxItems, so array bounds cannot reliably require one result for every mention. The output does not repeat target_mention; the required result key binds each object back to the locally stored target. This avoids asking the model for a field forbidden by the strict batch schema. Cerebras expands the referenced result for every task when counting property and enum strings. Batch schemas therefore constrain the exact object shape and retain the primary coarse/fine enum. Every returned coarse type, fine type, subtype, and facet is validated against the packaged taxonomy locally.

Evidence strength is also enforced locally. A positive result cannot carry NONE; such a result abstains instead. Model confidence is capped at 0.90 for CONTEXTUAL evidence and 0.65 for SURFACE_ONLY evidence. Lowercase evidence labels are normalized before validation.

Fine-stage batch output retains up to two locally validated secondary fine types when the evidence is genuinely ambiguous. Unknown IDs, types from another coarse branch, and the selected primary type are removed locally.

Every prediction exposes lossless NER tags at several granularities:

print(prediction.ner_tag)       # most precise primary tag
print(prediction.ner_tags)      # flat coarse/fine/specific/facet tags
print(prediction.ner_tag_sets)

# {
#   "primary": ("SWORD_AND_SANDAL_FILM",),
#   "coarse": ("CREATIVE_WORK",),
#   "fine": ("FILM",),
#   "subtype": (),
#   "specific": ("SWORD_AND_SANDAL_FILM",),
#   "facets": ("GENRE:SWORD_AND_SANDAL",),
#   "flat": (...),
#   "hierarchical": (
#       "COARSE:CREATIVE_WORK",
#       "FINE:FILM",
#       "SPECIFIC:SWORD_AND_SANDAL_FILM",
#       "FACET:GENRE:SWORD_AND_SANDAL",
#   ),
# }

If secondary fine types are returned, they appear after the primary under ner_tag_sets["fine"] and as FINE_ALTERNATIVE:<ID> in the hierarchical tag set. These tags are derived locally and require no additional model request.

Token and timing values under prediction.usage describe the shared request, so do not sum them across predictions from the same batch when calculating cost. The usage metadata includes shared_batch, batch_size, and batch_task_id for this reason. It also records the request ID, routed model, and exact system-prompt, user-prompt, and strict-schema character counts. For Cerebras requests it also records cerebras_expanded_string_budget, the provider-relevant budget after shared definitions are conservatively expanded.

Tabular data has a dedicated batch helper. Each TableCellTask applies the same bounded preprocessing as predict_table_cell() before entering the shared LLM requests:

from wikidata_ner import TableCellTask

table_tasks = [
    TableCellTask(
        cell="Rome Against Rome",
        column_header="title",
        row_context={
            "work_type": "film",
            "director": "Giuseppe Vari",
        },
        same_column_values=["Dune", "Solaris", "Arrival"],
        table_name="works",
    ),
    TableCellTask(
        cell="Dune",
        column_header="title",
        row_context={
            "work_type": "novel",
            "author": "Frank Herbert",
        },
        same_column_values=["Solaris", "Neuromancer", "Foundation"],
        table_name="works",
    ),
]

predictions = classifier.predict_table_cells(table_tasks)

Every task can provide its own header, row context, column samples, table name, description, max_row_fields, and max_column_samples. Empty row values, duplicate samples, and the target itself are removed. Each returned prediction retains its own table_preview and context_report, including omitted-context counts. TableCellTask is also accepted directly by predict_many() when text, record, and table tasks need to share one input iterable.

Inspect the exact zero-shot batch request without spending an API call:

preview = classifier.preview_batch_prompts(
    table_tasks[:MAX_MENTIONS_PER_BATCH],
    assumed_coarse_type="PRODUCT",
)

print(preview["coarse_request_characters"])
print(preview["fine_request_characters"])
assert preview["hard_max_mentions_per_batch"] == 8

What changed in 0.7.0

• Replaced hierarchy-heavy LLM prompts with a general zero-shot semantic rule; controlled labels remain constrained by response schemas and validated locally.
• Added source-first hierarchical prediction for free text, table columns, and table cells before Wikidata candidate retrieval.
• Added canonical retrieval paths, confidence-aware retrieval plans, hierarchy compatibility scoring, and candidate ranking safeguards.
• Added shared multi-mention OpenRouter requests, bounded table-cell contexts, prompt/schema preflight reporting, and schema-safe batching of up to 8 targets.
• Added deterministic NER tags and locally constructed retrieval keys without allowing the model to generate QIDs or retrieval metadata.

What changed in 0.6.0

• Added 23 controlled occupation types for real people, including POLITICIAN, ACTOR, MUSICIAN, WRITER, ATHLETE, and SCIENTIST.
• Human occupations are multi-valued: one person can expose several compatible specific_types without forcing an arbitrary single occupation.
• Enabled the same human-specific taxonomy for deterministic Wikidata input and locally validated OpenRouter mention inference.
• Generic humans still resolve to HUMAN when no occupation is supported.

What changed in 0.5.1

• Added order-preserving native predict_batch() for mappings and generators.
• Added an instance-local bounded LRU that reuses coarse/fine branch decisions, including abstentions, across batch calls.
• Kept description and context refinement independent for every item.
• Indexed subtype, facet, and composite retrieval rules by selected fine branch.
• Added cache statistics and explicit cache clearing.

See CHANGELOG.md for release history.

What changed in 0.5.0

• Added mention-focused type inference for free text and structured/tabular data.
• Added a dependency-free OpenRouter client with strict JSON-schema output.
• Added complete coarse-to-fine LLM inference over the packaged hierarchy, with branch-local subtype and facet selection in the second stage.
• Added OpenRouter provider pinning for fast Cerebras inference with fallbacks disabled when deterministic latency is required.
• Added exact mention-span marking, target-focus validation, and safe abstention.
• Added an auditable, bounded context report and table preview for cell inference.
• Retained the deterministic Wikidata token/clue classifier unchanged.

The QID is retained as an identifier and is never used as a lookup key.

Two classification paths

Wikidata items: deterministic token clues

Use this path when P31/P279 labels are already available:

from wikidata_ner import WikidataNERClassifier

classifier = WikidataNERClassifier()
prediction = classifier.predict(
    qid="Q3441181",
    types=[{"id": "Q11424", "name": "film"}],
    description="1964 sword-and-sandal film directed by Giuseppe Vari",
)

The primary branch is selected with the library's deterministic token/clue rules. Descriptions can refine that branch but cannot replace its P31/P279 anchor.

Input mentions: LLM inference through OpenRouter

Use this path when the input is a mention and its type must be inferred from context:

export OPENROUTER_API_KEY='...'

from wikidata_ner import OpenRouterNERClassifier

classifier = OpenRouterNERClassifier(
    model="openai/gpt-oss-120b",
    provider="cerebras",
    allow_fallbacks=False,
    reasoning_effort="low",
)

prediction = classifier.predict_text(
    "Rome Against Rome is a 1964 sword-and-sandal film directed by "
    "Giuseppe Vari; the story is set partly in Rome.",
    mention="Rome Against Rome",
)

print(prediction.fine_type)       # FILM
print(prediction.specific_type)   # SWORD_AND_SANDAL_FILM

Only Rome Against Rome is classified. The later Rome is contextual evidence about a different mention and cannot become the prediction target.

For a structured record:

prediction = classifier.predict_record(
    {
        "label": "Rome Against Rome",
        "types": [{"name": "film"}],
        "description": "1964 sword-and-sandal film",
    }
)

For a table cell:

prediction = classifier.predict_table_cell(
    "acetylsalicylic acid",
    column_header="active ingredient",
    row_context={
        "drug": "Aspirin",
        "molecular_formula": "C9H8O4",
    },
    same_column_values=["ibuprofen", "paracetamol", "naproxen"],
)

print(prediction.table_preview)
print(prediction.context_report["context_usage"])

The cell is always the target. Headers, row attributes, and same-column samples are evidence about the cell, never alternative targets. The returned context_report renders the exact selected table information, explains how each context component was interpreted, and reports whether fields or samples were omitted. The LLM receives that information as structured JSON; the Markdown preview is human-readable and is not duplicated in the prompt.

By default, table context is bounded to 12 non-empty same-row fields and 8 distinct same-column samples. Empty values, duplicate samples, and the target itself are removed from the sample set. Adjust the limits only when the table requires it:

prediction = classifier.predict_table_cell(
    cell,
    column_header="title",
    row_context=relevant_row_fields,
    same_column_values=column_examples,
    max_row_fields=8,
    max_column_samples=5,
)

For best accuracy, pass fields that describe or relate directly to the target cell—such as a type/category, description, unit, identifier, creator, location, or parent relation. Avoid unrelated display metadata and entire unfiltered rows.

For contextual free text, mention= is required. You can disambiguate repeated surface forms with an exact character span:

prediction = classifier.predict_text(
    text,
    mention="Rome",
    mention_span=(start, end),
)

Use preview_prompts(...) to inspect the normalized mention, exact prompts, and JSON schemas without making an API call:

preview = classifier.preview_prompts(
    text,
    mention="Rome Against Rome",
    assumed_coarse_type="CREATIVE_WORK",
    assumed_fine_type="FILM",
)

The default predictor uses two schema-constrained zero-shot calls:

1. Select exactly one controlled coarse branch or abstain.
2. Select one fine type and only its legal subtypes and facets inside that branch.

Passing a model slug does not select a hosting provider on OpenRouter. Use provider="cerebras" with allow_fallbacks=False when Cerebras latency is required. Each stage reports the routed provider and wall-clock duration under prediction.usage.

OpenRouter is called at https://openrouter.ai/api/v1/chat/completions with strict JSON-schema output, provider.require_parameters=true, temperature zero, and optional response healing. The package continues to have no runtime dependencies. You may inject a custom client= for testing or infrastructure integration.

MentionPrediction supports the same retrieval conveniences as deterministic predictions:

payload = prediction.to_dict()
fields = prediction.to_retrieval_fields(prefix="ner")
query_filter = prediction.elasticsearch_filter()

The selected OpenRouter model must support structured outputs. Pin a model slug in production and store the returned model, prompt version, taxonomy version, usage, evidence, and confidence with each result.

Deterministic evidence policy

The default evidence policy is now:

1. types[].name selects coarse_type and fine_type.
2. ancestor_types[].name, when supplied, provides lower-weight class ancestry.
3. Direct type labels and description refine only the selected branch.
4. context_string is ignored by the classifier by default because it often contains related people, organizations, countries, genres, and formats.
5. Description evidence cannot change a FILM branch into a location, company, person, or another unrelated branch.
6. Unsupported specificity is not invented.

The packaged configuration contains:

• 187 fine-type rules;
• 295 structural subtype rules;
• 56 controlled facet rules;
• 7 branch-local composite-type templates.

Installation

python -m pip install wikidata-ner-classifier

Python 3.10 or newer is required. The library has no runtime dependencies.

Deterministic basic use

from wikidata_ner import WikidataNERClassifier

classifier = WikidataNERClassifier()

prediction = classifier.predict(
    qid="Q3441181",
    types=[{"id": "Q11424", "name": "film"}],
    description="1964 sword-and-sandal film directed by Giuseppe Vari",
)

print(prediction.to_dict())

Relevant output:

{
  "coarse_type": "CREATIVE_WORK",
  "fine_type": "FILM",
  "subtype": null,
  "specific_type": "SWORD_AND_SANDAL_FILM",
  "specific_types": [
    "SWORD_AND_SANDAL_FILM"
  ],
  "facets": {
    "genre": [
      "SWORD_AND_SANDAL"
    ]
  },
  "refinement_sources": [
    "description"
  ]
}

The description adds specificity only inside the already established FILM branch.

Native batch prediction

Use predict_batch() when classifying many items. It accepts any iterable of item mappings, returns normal Prediction objects in input order, and retains each QID:

items = [
    {
        "qid": "Q3441181",
        "types": [{"id": "Q11424", "name": "film"}],
        "ancestor_types": [],
        "label": "Rome Against Rome",
        "description": "1964 sword-and-sandal film",
        "context_string": None,
    },
]

predictions = classifier.predict_batch(items, cache_size=100_000)

Batch prediction normalizes the direct and ancestor type labels, groups identical ordered signatures, and performs the full coarse/fine rule scan once per missing signature. The bounded cache is a true least-recently-used cache and is local to the classifier instance, so custom rules and configuration never share entries with another classifier. Successful decisions and abstentions are both cached.

Only the primary coarse/fine decision is reused. Subtypes, facets, composite specific types, evidence, and refinement sources are calculated independently for every item, so descriptions and context strings remain item-specific. Results are exactly equivalent to calling predict() on every item.

By default, the cache key is the ordered normalized direct and ancestor label signature. If use_description=True, the normalized description is also part of the key. If use_entity_label=True, the normalized entity label is also part of the key. Description/context settings used only for branch-local refinement do not widen the primary key.

Inspect or reset the cache with:

info = classifier.branch_cache_info()
print(info.hits, info.misses, info.maxsize, info.currsize)

classifier.clear_branch_cache()

Passing cache_size=0 disables reuse across calls while still deduplicating repeated signatures inside the current batch. Changing cache_size on a later call immediately evicts the least recently used entries until the cache fits. The implementation is synchronous, dependency-free, and deterministic; callers can place independent classifier instances in an external process pool.

Alpaca or Elasticsearch entities

Both source objects and complete Elasticsearch hits are accepted:

prediction = classifier.predict_entity(hit_or_source)

{
  "qid": "Q3441181",
  "types": [{"name": "film"}],
  "description": "1964 sword-and-sandal film directed by Giuseppe Vari"
}

{
  "_id": "Q3441181",
  "_source": {
    "qid": "Q3441181",
    "types": [{"name": "film"}],
    "description": "1964 sword-and-sandal film directed by Giuseppe Vari"
  }
}

A complete Elasticsearch response can be processed with:

predictions = classifier.predict_elasticsearch_response(response)

Why both subtype and specific type exist

A subtype describes a structural kind. A facet describes an independent characteristic. A specific type is a retrieval-oriented composition.

prediction = classifier.predict(
    "Q1",
    [
        {"name": "film"},
        {"name": "feature film"},
        {"name": "comedy film"},
    ],
)

This can produce:

{
  "fine_type": "FILM",
  "subtype": "FEATURE_FILM",
  "specific_type": "FEATURE_FILM",
  "specific_types": [
    "FEATURE_FILM",
    "COMEDY_FILM"
  ],
  "facets": {
    "genre": [
      "COMEDY"
    ]
  }
}

FEATURE_FILM and COMEDY_FILM are compatible. They may be combined by the retriever rather than forced into a single mutually exclusive label.

Human occupations use the same compatibility model:

person = classifier.predict(
    "Q7259",
    [{"name": "human"}],
    description="British politician and writer",
)

assert person.fine_type == "HUMAN"
assert person.specific_type == "POLITICIAN"
assert person.specific_types == ("POLITICIAN", "WRITER")
assert person.facets["occupation"] == ("POLITICIAN", "WRITER")

The controlled human occupation types are ACADEMIC, ACTIVIST, ACTOR, ARCHITECT, ARTIST, ATHLETE, BUSINESSPERSON, EDUCATOR, ENGINEER, EXPLORER, FILMMAKER, INVENTOR, JOURNALIST, LEGAL_PROFESSIONAL, MEDICAL_PROFESSIONAL, MILITARY_PERSONNEL, MUSICIAN, POLITICIAN, PUBLIC_OFFICIAL, RELIGIOUS_FIGURE, ROYALTY, SCIENTIST, and WRITER.

Example refinements from the Alpaca query

Type labels	Description	Fine type	Subtype	Most specific retrieval type
film	1951 film directed by Luigi Zampa	FILM	none	FILM
film	1964 sword-and-sandal film ...	FILM	none	SWORD_AND_SANDAL_FILM
album	album by Holger Czukay	MUSICAL_WORK_SONG_OR_ALBUM	MUSIC_ALBUM	MUSIC_ALBUM
literary work	Alternative history, military science fiction story	BOOK_OR_WRITTEN_WORK	FICTION_STORY	MILITARY_SCIENCE_FICTION_LITERARY_WORK
pencil drawing	1953 work of art ...	VISUAL_ARTWORK_PHOTOGRAPH_OR_COMIC	PENCIL_DRAWING	PENCIL_DRAWING
human	British politician and writer	HUMAN	none	POLITICIAN, WRITER

A generic description cannot justify an invented subtype. A generic film remains FILM when neither its type labels nor description contain a safe refinement.

Retrieval indexing helpers

Store the prediction alongside each entity using stable keyword fields:

fields = prediction.to_retrieval_fields(prefix="ner")

Example fields:

{
  "ner_coarse_type": "CREATIVE_WORK",
  "ner_fine_type": "FILM",
  "ner_subtype": null,
  "ner_specific_type": "SWORD_AND_SANDAL_FILM",
  "ner_specific_types": [
    "SWORD_AND_SANDAL_FILM"
  ],
  "ner_facets": {
    "genre": [
      "SWORD_AND_SANDAL"
    ]
  }
}

A deterministic Elasticsearch filter can be generated with:

query_filter = prediction.elasticsearch_filter(
    field="ner_specific_types",
    require_all=True,
)

For several compatible specific types, require_all=True emits one term filter per type. Use require_all=False to emit a terms disjunction.

Index-time and query-time predictions should use the same library and rule-file version.

Description and context controls

Description refinement is enabled by default:

classifier = WikidataNERClassifier(
    use_description_for_refinement=True,
    use_context_string_for_refinement=False,
)

Disable it when only class labels should be considered:

classifier = WikidataNERClassifier(
    use_description_for_refinement=False,
)

Noisy context refinement is available only as an explicit opt-in:

classifier = WikidataNERClassifier(
    use_context_string_for_refinement=True,
)

The separate use_description=True option allows description text to add low-weight support to the primary coarse/fine scorer. It is disabled by default. Descriptions therefore do not rescue a missing or unknown type anchor unless the caller explicitly changes that policy.

Live Alpaca notebook

Open examples/alpaca_live_test.ipynb.

The notebook:

1. issues the supplied Alpaca Elasticsearch request;
2. extracts QID, type labels, and description;
3. ignores context_string during classification;
4. displays predicted_subtype, predicted_specific_type, all compatible specific_types, facets, and confidence values;
5. demonstrates a retrieval filter generated from the prediction.

Set the bearer token before starting Jupyter:

export ALPACA_TOKEN='your-token'

The notebook also supports a hidden token prompt when the environment variable is not set.

CLI

wikidata-ner response.json > predictions.json
cat response.json | wikidata-ner

Relevant flags:

--no-description-refinement
--context-refinement
--description-for-primary
--entity-label

Validation

The source package includes unit tests for:

• direct type-label classification;
• description-only branch refinement;
• context exclusion by default;
• explicit context opt-in;
• subtype and facet compatibility;
• generic fallback behavior;
• QID independence;
• Elasticsearch hit input;
• retrieval-field generation;
• generated Elasticsearch filters.

Mention-focused OpenRouter notebook

examples/openrouter_mention_focused_ner.ipynb classifies one explicit target mention at a time from free text, structured records, or table cells. Context is used only as evidence for that target. Contextual free text requires mention= or an exact span; table helpers make the selected cell the target. The installable library now exposes the same workflow through OpenRouterNERClassifier; the notebook remains useful as an expanded prompt inspection and evaluation example.