annotations4all

Languages: Deutsch · English

annotations4all is a schema-first Python library for LLM-assisted span annotation (commonly used for named-entity recognition). It generates prompts from a user-defined tag schema, parses annotated LLM responses in <<TAG>>…</TAG>> format, and returns matches as offset-based spans.

Status: 0.1.1 is intended as an alpha release. The stable v0.1 surface consists of prompt taggers, parsers, and client helpers. Experimental legacy clients are marked as such.

Documentation

The tutorial documents the v0.1 API surface and shows examples against local, OpenAI-compatible installations (e.g. a llama.cpp server or Ollama's OpenAI-compatible endpoint).

Installation

After the PyPI release:

python -m pip install annotations4all

For development, from the repository:

python -m venv .venv
. .venv/bin/activate
python -m pip install -e .
python -m pytest

Schema-first quick start

The following example shows the preferred v0.1 entry point: define the tag schema first, then generate prompt messages and map the model response back to spans.

from annotations4all import ConfigurableTagger

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)

messages = tagger.get_prompt(text)
for message in messages:
    print(message["role"])
    print(message["content"])

# Response of an LLM, e.g. from an OpenAI-compatible endpoint:
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."
spans = tagger.parse_response(response, text)
print(spans)

tags describes the tag schema of the concrete workflow. context holds material- or task-specific annotation hints, not arbitrary runtime data. Common NER tags such as PER, LOC, and ORG are well suited for quick starts, but the library does not enforce a fixed ontology.

Minimal example without an LLM call

The following example shows the smallest stable core: an already annotated response is mapped back to character positions in the original text.

from annotations4all import parse_region_response, parse_region_response_detailed

text = "Max Mustermann lives in Berlin."
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."

spans = parse_region_response(response, text)
detailed = parse_region_response_detailed(response, text)
print(spans)
print(detailed.warnings)
# [{'label': 'PER', 'start': 0, 'end': 14}, {'label': 'LOC', 'start': 24, 'end': 30}]

Writing custom taggers

For advanced use cases, custom taggers can be implemented. The base class ChatTagger remains importable for this purpose but is not part of the highlighted package-root API of v0.1.

from annotations4all.taggers.base import ChatTagger, Message
from annotations4all.utils.response_parser import parse_region_response


class MyTagger(ChatTagger):
    def name(self) -> str:
        return "my-tagger"

    def get_prompt(self, text: str) -> list[Message]:
        return [
            {
                "role": "system",
                "content": "Annotate persons as <<PER>>…</PER>> and places as <<LOC>>…</LOC>>.",
            },
            {"role": "user", "content": text},
        ]

    def parse_response(self, response: str, text: str, logfile=None):
        return parse_region_response(response, text, logfile=logfile)

The model response must reproduce the original text as exactly as possible and mark spans with opening and closing tags:

<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>.

Tag schemas

The library does not enforce a fixed ontology. The tag schema belongs to the respective research or annotation workflow. Common quick-start tags include:

• PER: persons
• LOC: places
• ORG: organizations
• project-specific tags such as DOM, DAT, KG, etc.

Tags can optionally carry metadata, e.g. <<LOC:city>>Berlin</LOC>>. The parser returns this metadata as meta when present. The returned objects are spans with at least label, start, and end.

Backends and clients

For v0.1, only a narrow, explicit OpenAI-compatible chat-completions interface is officially supported. The target server must offer an endpoint such as /v1/chat/completions and be compatible with the request/response shape of the README examples.

from annotations4all import ConfigurableTagger, OpenAICompatClient

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)
client = OpenAICompatClient(
    api_url="https://example.invalid/v1",
    api_key="<token>",
)

chunks = client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    stream=True,
    temperature=0,
)
response = "".join(chunk.answer for chunk in chunks)
spans = tagger.parse_response(response, text)
print(spans)

OpenAICompatClient reads API keys either from api_key= or from an environment variable. The default is OPENAI_API_KEY; api_key_env= selects a different name. Local servers that do not require authentication automatically receive a dummy key, because the underlying OpenAI SDK still expects a value.

Provider-specific request fields (e.g. reasoning options) are passed through generically via extra_body= — the library does not interpret the payload, the endpoint defines the schema:

client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    extra_body={"reasoning": {"enabled": False}},
)

The v0.1 compatibility promise is deliberately narrow: standard content responses and the streaming form used in the tests/examples are the target. The semantics of provider-specific fields are explicitly not a stability promise — the generic extra_body passthrough itself is.

Tests

python -m pytest

The tests check prompt invariants, parser golden cases, a small fuzz baseline, and client helper structures.

Citation

If you use this software in academic work, please cite it as follows:

Dresselhaus, Nicole. (2026). annotations4all (Version 0.1.1) [Software]. Humboldt-Universität zu Berlin. https://scm.cms.hu-berlin.de/annotations4all/annotations4all

DOI: 10.5281/zenodo.22011371

Machine-readable metadata is available in CITATION.cff.

License

This software is licensed under the MIT License. See LICENSE for details.