glinerel-fast

Fast, lightweight inference for glinerel entity, PII and relation extraction models.

• Same predictions as the models' reference implementation in float32 (checked on 1,288 texts, model sizes 72m, 128m and 314m, CPU and Apple GPU).
• Faster: up to 4x lower single-text latency on CPU, up to 2x the reference's speed on the GPU, and 4-7x faster loading (see Performance).
• Small: ~1,650 lines of Python on PyTorch, NumPy, tokenizers and safetensors. No transformers, no model conversion: the checkpoint's model.safetensors is read directly.
• Runs where PyTorch runs: CPU (Linux, Windows, macOS), NVIDIA / AMD GPUs, Apple silicon GPUs. The best available device is picked automatically.

Install

pip install glinerel-fast

For a specific PyTorch build (CPU-only, a given CUDA version, ROCm), install PyTorch first from pytorch.org. Requires Python 3.10+ and PyTorch 2.5+.

The models are downloaded from the Hugging Face Hub on first use. If a repository is private, log in first (hf auth login) or set HF_TOKEN.

Quick start

from glinerel_fast import GLiNERel

model = GLiNERel.from_pretrained("path/to/glinerel-model")   # local folder or Hugging Face repository id

model.extract(
    "Hi, I'm Laura Bianchi. Reach me at +39 347 555 0192 or laura.bianchi@example.com.",
    entities=["first name", "last name", "phone number", "email"],
)
# {'entities': [{'label': 'first name', 'text': 'Laura', 'start': 8, 'end': 13, 'score': 0.9986},
#               {'label': 'last name', 'text': 'Bianchi', 'start': 14, 'end': 21, 'score': 0.9991},
#               {'label': 'phone number', 'text': '+39 347 555 0192', 'start': 35, 'end': 51, 'score': 0.9986},
#               {'label': 'email', 'text': 'laura.bianchi@example.com', 'start': 55, 'end': 80, 'score': 0.9972}],
#  'relations': []}

Entities and relations

result = model.extract(
    "Marie Curie was born in Warsaw in 1867. She married Pierre Curie.",
    entities=["person", "location", "date"],
    relations=["born in", "spouse"],
)
for r in result["relations"]:
    print(r["head"]["text"], f"--[{r['label']}]-->", r["tail"]["text"], r["score"])
# Marie Curie --[born in]--> Warsaw 0.9686
# Marie Curie --[spouse]--> Pierre Curie 0.9479

extract(texts, entities=None, relations=None, *, entity_threshold=0.5, relation_threshold=0.25, batch_size=8, trim_punctuation=True, merge_mentions=False)

• texts: one string (returns one result) or a list of strings (returns a list in the same order). Pass many texts per call: they are sorted by length and batched internally.
• entities / relations: a list of labels, or {label: threshold} for per-label thresholds.
• Each result is {"entities": [{"label", "text", "start", "end", "score"}], "relations": [{"label", "head": {"text", "start", "end"}, "tail": {...}, "score"}]}, with character offsets (end exclusive). Entities are sorted by position, relations by score.
• trim_punctuation=False keeps the model's word-level spans ("Warsaw."); merge_mentions=True keeps one relation per pair of entity surfaces.

PII redaction

PII = ["first name", "last name", "date of birth", "phone number", "email", "city", "passport number"]
text = "Hi, I'm Laura Bianchi, born on 3 May 1988. Call me at +39 347 555 0192, I live in Milano."

model.predict_entities(text, PII)   # GLiNER-style list of entities
model.redact(text, PII)
# "Hi, I'm [FIRST_NAME] [LAST_NAME], born on [DATE_OF_BIRTH]. Call me at [PHONE_NUMBER], I live in [CITY]."
model.redact(text, PII, placeholder="<{label}>")

Devices and precision

GLiNERel.from_pretrained(model, device=None, precision="fp32", threads=None)

• device: "cpu", "cuda", "cuda:1", "mps"; None picks CUDA, then MPS, then the CPU.
• precision: "fp32" (the reference's predictions), "bf16" or "fp16" (half-precision matrix products, for GPUs; 99.6-100% agreement with fp32).
• threads: CPU threads. On Linux, prefer setting OMP_NUM_THREADS before starting Python.

Tips: pass many texts per call and, on GPUs, use batch_size=32 or more. Every label adds a few tokens to every text, so ask only for the labels you need. Long texts are not truncated; split very long documents by paragraph.

Command line

glinerel-fast extract --model path/to/glinerel-model --text "Contact Laura at laura@example.com" \
    --entities person email
glinerel-fast extract --model path/to/glinerel-model --input texts.jsonl --output out.jsonl \
    --entities person organization --relations "works for"
glinerel-fast bench --model path/to/glinerel-model      # time per text on this machine
glinerel-fast info                   # versions, GPUs, default device

Performance

Apple M3 Pro, PyTorch 2.14, 64 PII texts with 20 PII labels, time per text. The reference is the models' inference.py on the same PyTorch and device.

On a Linux CPU (OpenBLAS) the two implementations are bound by the same matrix products and run at about the same speed (0.9-1.4x), with the faster loading. NVIDIA GPUs and Windows have not been benchmarked yet.

Accuracy against the reference (agreement F1, the reference's predictions as gold; 1,000 PII texts in 5 languages and the 288 CoNLL04 test sentences):

License

Apache-2.0 for this library. The model weights are subject to their own license on the Hugging Face Hub.