llm-dialects

Streaming parsers for the tool-call and reasoning formats of open-weight LLMs. Raw text in, structured deltas out. Zero dependencies.

from llm_dialects import parser

p = parser("qwen3", tools=tools)          # or parser.detect(chat_template=..., model_id=...)
for delta in text_deltas:                 # raw model text, in chunks of any size
    for ev in p.feed(delta): ...          # ReasoningDelta, ContentDelta, ToolCallStart, ToolCallArgsDelta, ToolCallEnd
for ev in p.close(): ...

Fed with what Qwen3 actually generates, nine characters at a time —

<think>\nThe user wants the weather in Paris.\n</think>\n\nLet me check.\n<tool_call>\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n</tool_call>

— the loop yields:

ReasoningDelta(text="T")
ReasoningDelta(text="he user w")
ReasoningDelta(text="ants the")
ReasoningDelta(text=" weather i")
ReasoningDelta(text="n Paris.")
ContentDelta(text="Let me c")
ContentDelta(text="heck.")
ToolCallStart(index=0, id="call_0", name="get_weather", known=True)
ToolCallArgsDelta(index=0, text="{")
ToolCallArgsDelta(index=0, text='"city": "Paris"}')
ToolCallEnd(index=0, arguments='{"city": "Paris"}', status="ok")

No markup leaks, no half-emitted <tool_c, and the argument fragments concatenate to valid JSON. The same events come out for any chunking of the same text.

pip install git+https://github.com/cb-Sigma/llm-dialects

Python 3.10+. Not on PyPI yet.

Contents

• Why this exists
• Supported dialects
• The chunk-boundary guarantee
• When the model misbehaves
• API in one screen
• Building an OpenAI-compatible server
• The conformance corpus
• Performance
• Limitations and maintenance reality
• Related work
• FAQ
• Roadmap
• Contributing

Why this exists

Open-weight models do not emit tool calls or reasoning as structured data. They emit text, in a format ("dialect") that each model family invented for itself:

<tool_call>{"name": "f", "arguments": {...}}</tool_call>             Qwen, Hermes
<|python_tag|>{"name": "f", "parameters": {...}}                     Llama 3.x
[f(a=1, b="x")]                                                      Llama 4
[TOOL_CALLS]f[ARGS]{...}                                             Mistral
<start_function_call>call:f{a:<escape>x<escape>}<end_function_call>  FunctionGemma
<tool_call>f<arg_key>a</arg_key><arg_value>x</arg_value></tool_call> GLM
<|channel|>commentary to=functions.f<|message|>{...}<|call|>         gpt-oss

Turning that text into reasoning_content / content / tool_calls deltas while it streams is re-implemented inside every inference engine — vLLM, SGLang, llama.cpp, Ollama, mlx-lm — and it is a steady source of bugs. On 2026-11-13 the vLLM issue tracker listed 389 issues labelled tool-calling and 60 with "streaming tool parser" in the title; the recent ones are the same few failures in one parser after another: streaming and non-streaming paths disagreeing, text leaking into or out of arguments, content dropped after a marker, a call lost because a closing tag was missing. A few that were read in full for this project:

• vllm#39056: a tool call emitted before </think> ends up in reasoning, and tool_calls stays empty.
• mlx-lm#1627: the Qwen3-Coder parser ran int("140.0"), raised, and the server silently dropped the call while still reporting finish_reason: "tool_calls"; an agent looped ~390 times.
• mlx-lm#1096, mlx-lm#1885: a new model family (Gemma 4, MiniCPM5) simply had no parser, so calls arrived as plain text.
• llama-cpp-python#1784 (open since 2024) and #2227: Qwen2.5 and Gemma 4 tool calls come back as raw text in content, because the Python binding has no output-side parser.

If you run one of the big engines' own servers, you already have their parsers. But if you write your own OpenAI-compatible endpoint — on MLX, transformers, llama-cpp-python, LitServe, a raw /completions endpoint — or post-process raw completions, there has been nothing small to import. The engines' parsers are wired into their servers and tokenizers. The one pip-installable extraction, llm-router-utils (SGLang's parsers), depends on pinned transformers, openai, fastapi and on xgrammar, which requires torch.

llm-dialects is that small thing: a few thousand lines of typed Python with no dependencies, built around two guarantees that are tested rather than hoped for — it never emits text it would have to take back, and its output does not depend on how the stream was chunked.

Two things upstream matter for where this is going, and they are not hidden here: llama.cpp now derives its parsers from the chat template, and Hugging Face transformers is introducing response templates that models can ship. See Limitations and maintenance reality.

Supported dialects

"Real output" means the corpus contains raw text captured from that model on this project's own runs, and the parser is tested on it (token by token, as generated). "Engine vectors" means the dialect is tested against test vectors adapted from the engines' own test suites and against the layout its chat template renders, but has not been run against a live model here.

Dialect	Model families	Reasoning	Tool calls	Validated on	Sources
hermes (qwen, qwen3, chatml)	Qwen2.5, Qwen3, Hermes 2 Pro / 3, SmolLM3	<think>	✓	real output: Qwen2.5-0.5B, Qwen3-0.6B; engine vectors	vLLM, SGLang, llama.cpp, Ollama, templates
think	DeepSeek-R1 distills, QwQ, any <think> model	<think>	–	real output: DeepSeek-R1-Distill-Qwen-1.5B; engine vectors	vLLM, SGLang, Ollama
llama3-json	Llama 3.1 / 3.2 / 3.3	–	✓	real output: Llama-3.2-1B; engine vectors	vLLM, SGLang, templates
gemma (functiongemma, gemma4)	FunctionGemma, Gemma 4	Gemma 4 thought channel	✓	real output: FunctionGemma-270M; Gemma 4 on engine vectors only	vLLM, SGLang, Ollama, templates
deepseek (deepseek-r1, deepseek-v3.1)	DeepSeek-V3, R1, V3.1	<think>	✓	engine vectors only (tool format)	vLLM, SGLang, templates
pythonic (llama4)	Llama 4, Llama 3.2 pythonic prompt	–	✓	engine vectors only	vLLM, SGLang, template
mistral (magistral, devstral)	Mistral 7B v0.3, Nemo, Small 3.x, Magistral	[THINK]	✓ both generations	engine vectors only	vLLM, SGLang, llama.cpp, Ollama, templates
granite	Granite 3.0–3.3, 4.0	<think> + <response> (3.3)	✓	engine vectors only	vLLM, templates
phi4	Phi-4-mini	–	✓	engine vectors only	vLLM, llama.cpp
glm	GLM-4.5 / 4.6 / 4.7	<think>	✓	engine vectors only	vLLM, SGLang, llama.cpp, Ollama, templates
kimi-k2	Kimi K2	<think>	✓	engine vectors only	vLLM, SGLang, template
minimax-m2	MiniMax-M2	<think>, interleaved	✓	engine vectors only	vLLM, SGLang, template
qwen3-coder	Qwen3-Coder	<think>	✓	engine vectors only	vLLM, SGLang, Ollama, template
harmony (gpt-oss)	gpt-oss-20b / 120b	analysis channel	✓	engine vectors only; text-level, see note	Ollama, SGLang, template

Each dialect page has the grammar, the markers, the exact source files it was derived from, and the cases where it deliberately differs from an engine. Sources are pinned to commits in docs/sources.md.

Harmony: this is a text-level parser checked against vectors from Ollama's Harmony tests. Harmony is defined on tokens; when you have token ids, the official openai-harmony is the reference and the better choice. It is here so that servers with only rendered text can use one API for every model.

Programmatic access to all of this: llm_dialects.dialects() returns the registry with models, sources and validation level.

The chunk-boundary guarantee

For any input and any way of cutting it into chunks:

1. Nothing is emitted that might be taken back. A marker split across chunks (<tool_ + call>) is held, minimally, until it is decided; a comma, an object key, a number are held until what follows makes them certain. Everything else passes through at once — a 200 kB string argument streams in pieces as it arrives.
2. The canonical event sequence is identical — and parse(text) returns exactly that sequence. ("Canonical" = adjacent text deltas merged, llm_dialects.coalesce; how text is cut into deltas necessarily follows the chunks.)
3. ToolCallArgsDelta fragments of a call always concatenate to valid JSON, whatever the model wrote.

This holds by construction — each parser is a generator that reads characters and yields when it needs more, so it cannot see chunk boundaries, and parse() is literally feed() + close() on the same machine (docs/design.md) — and it is tested, because by-construction arguments have gaps (the first property-test run found one):

Test	What it checks
every split point	each of the 222 corpus cases, cut at every single position, gives the same events and the same issues
one character at a time	each corpus case fed character by character
recorded token chunks	each real case replayed with the exact token pieces the model produced
random multi-way splits	seeded sweeps and hypothesis-drawn cut points over the corpus
render → parse round trip	hypothesis generates responses (reasoning, content, calls with nested JSON arguments), a test-side writer renders them in each dialect, and the parser must return the same structure for every chunking; 16 writers cover 13 of the 14 dialects (think is exercised through the reasoning part of the others)
truncation anywhere	the rendered responses cut short at a random point: still valid, still chunk-invariant
marker soup	random concatenations of every marker of every dialect, JSON fragments and junk, in all three reasoning modes: feed() never raises, arguments are valid JSON, events and issues are chunk-invariant
mutated corpus	corpus inputs with spans deleted, duplicated or spliced
plain text	text without markup comes back as content byte for byte
strict policy	under STRICT, a case either parses to the same events or raises the same error at the same position for every chunking
JSON transcoder	valid JSON passes through verbatim; every prefix of a valid document becomes a valid document; arbitrary text never produces invalid output

In numbers: the suite has 1,832 tests and passes on Python 3.10–3.13. The corpus sweep parses 40,034 distinct single-cut chunkings of the 222 cases and replays 4,736 recorded model tokens; 95 property tests draw 9,500 generated inputs per run by default (19,000 in CI, and 190,000 with HYPOTHESIS_PROFILE=thorough, which was run before this release). mypy --strict and ruff are clean.

When the model misbehaves

Small and quantised models produce broken output routinely; large ones do it at max_tokens. What happens then is a policy, explicit and configurable:

from llm_dialects import Policy, STRICT, parser

p = parser("qwen3", tools=tools)  # LENIENT (default): never raises
p = parser("qwen3", tools=tools, policy=STRICT)  # raise on every deviation
p = parser(
    "qwen3", tools=tools, policy=Policy(malformed_arguments="raise", text_after_tool_call="drop")
)

Situation	Policy field	Lenient (default)	Other choices
Arguments are not valid JSON	malformed_arguments	"repair": fix what is unambiguous → status="repaired"; otherwise close what was streamed into valid JSON → status="invalid"	"raise" → ArgumentsError
Stream ends inside a tool call	truncated_tool_call	"repair": close strings and brackets → status="truncated"	"raise" → TruncatedError
Call to an undeclared tool	unknown_tool	"flag": ToolCallStart.known=False	"raise" → UnknownToolError
Text after a tool call	text_after_tool_call	"content": emitted as content	"drop", "raise"
Tool call before reasoning closed	tool_call_in_reasoning	"extract": ends the reasoning, parses the call	"reasoning" (leave it in the reasoning text), "raise"
Stream ends inside reasoning	unclosed_reasoning	"accept"	"raise"

Everything handled leniently is also recorded in parser.issues (code, message, position), and ParseResult.ok is true only if nothing was flagged.

What "repair where unambiguous" covers — each of these appears in the corpus, several in real model output:

{'city': 'Paris', 'days': None,}              single quotes, Python literals, trailing comma
{city: "Paris"}                               unquoted keys
{"text": "line one⏎line two"}                 raw newline inside a string
{"name": "f", "arguments": {"a": 1}</tool_call>    a closing brace short of valid
{{"name": "f", "arguments": {...}}}           doubled braces (Qwen2.5-0.5B does this with the official GGUF template)

Truncation never invents data: {"a": 1, "b": + end of stream becomes {"a": 1}, not {"a": 1, "b": null}.

A parse that is syntactically valid is still not a decision to execute. known=False and status != "ok" are yours to act on; see SECURITY.md.

API in one screen

from llm_dialects import parser, parse, detect, dialects, coalesce, Policy, LENIENT, STRICT

p = parser("mistral", tools=tools, policy=LENIENT, reasoning=None, id_factory=None)
p.feed(text) -> list[Event]        # events that became certain
p.close()    -> list[Event]        # end of stream: flush, close open calls
p.issues     -> list[Issue]        # what was repaired or flagged

result = parse(text_or_chunks, "mistral", tools=tools)   # same machine, same result
result.content, result.reasoning, result.tool_calls, result.events, result.issues, result.ok
call = result.tool_calls[0]; call.name, call.id, call.arguments, call.json(), call.status, call.known

found = detect(chat_template=tokenizer.chat_template, model_id="Qwen/Qwen3-0.6B")
found.dialect, found.reasoning, found.confidence, found.sure, found.reasons
p = parser.detect(chat_template=..., model_id=...)       # raises if unknown, warns if a guess

• Sync core, async-friendly. feed() does no I/O and never blocks; call it inside async for (examples/async_stream.py). One parser per response.
• tools= takes the OpenAI-format tool list of the request. It enables unknown-tool flagging and, for the dialects whose arguments are untyped text (GLM, MiniMax, Qwen3-Coder), gives each value its declared JSON type.
• reasoning= is "auto" (reasoning starts at an opening marker), "prefilled" (the chat template already opened the block, so output starts in reasoning — DeepSeek-R1, MiniMax-M2, GLM-4.7) or "off". Each dialect defaults to what its reference template does; detect() reads it off a template.
• Ids are deterministic (call_0, call_1, …) unless the dialect carries a native id (Kimi K2, Mistral [CALL_ID]). Pass id_factory=llm_dialects.random_call_id for globally unique ones.
• detect() says when it is unsure. A chat template with a known signature is confidence="high". A model id alone is at best "medium" — fine-tunes keep names and change templates — and parser.detect() warns in that case.

Building an OpenAI-compatible server

The adapters in llm_dialects.openai produce plain dicts in the Chat Completions shapes, so a custom server needs a few lines per endpoint (examples/openai_server_fastapi.py is the complete file):

from llm_dialects import parser, random_call_id
from llm_dialects.openai import ChunkStream, completion
from llm_dialects.result import collect


@app.post("/v1/chat/completions")
async def chat_completions(body: dict):
    machine = parser("qwen3", tools=body.get("tools"), id_factory=random_call_id)
    deltas = generate(body["messages"], body.get("tools"))  # your engine: raw text deltas

    if not body.get("stream"):
        events = [ev for delta in deltas for ev in machine.feed(delta)] + machine.close()
        return completion(collect(events, machine.issues), model=body["model"])

    async def sse():
        stream = ChunkStream(model=body["model"])
        for delta in deltas:
            for chunk in stream.chunks(machine.feed(delta)):
                yield f"data: {json.dumps(chunk)}\n\n"
        for chunk in stream.chunks(machine.close()):
            yield f"data: {json.dumps(chunk)}\n\n"
        yield f"data: {json.dumps(stream.finish())}\n\n"  # finish_reason: "tool_calls" or "stop"
        yield "data: [DONE]\n\n"

    return StreamingResponse(sse(), media_type="text/event-stream")

Reasoning goes to reasoning_content (pass reasoning_field="reasoning" for the newer spelling). With llama-cpp-python, use raw completion and let the GGUF's own chat template render the prompt: examples/llama_cpp_python_tools.py.

Two things your server still has to do: make the tokenizer render special tokens (skip_special_tokens=False, special=True) — a parser cannot see markers that were stripped — and pass finish_reason="length" to finish() when generation hit the token limit.

The conformance corpus

corpus/ holds 222 cases as plain JSON, one file per dialect: the raw input, the tools of the request, the expected segments, and where the input came from.

{
  "id": "kimi-k2/truncated",
  "input": "<|tool_calls_section_begin|><|tool_call_begin|>functions.get_weather:0<|tool_call_argument_begin|>{\"city\": \"Bei",
  "expected": [
    {"type": "tool_call", "name": "get_weather", "arguments": {"city": "Bei"}, "id": "functions.get_weather:0", "status": "truncated"}
  ],
  "provenance": {"kind": "engine-test", "source": "vllm:tests/tool_parsers/test_kimi_k2_tool_parser.py test_truncated_tool_call_no_end_marker"},
  "divergences": [{"engine": "vllm", "behaviour": "streamed arguments are '{\"city\": \"Bei' (asserted by the test), which is not JSON", "reason": "arguments are always valid JSON; the call is flagged 'truncated'"}]
}

Provenance	Cases	Meaning
real	48	captured from a real model through its own chat template, with the token pieces as generated
engine-test	110	adapted from the test suites of vLLM, SGLang, llama.cpp and Ollama (attribution in NOTICE)
engine-source	10	an example or documented transition from an engine's parser source
upstream-issue	2	the scenario of a reported engine bug
template	27	the layout the model's chat template renders
synthetic	25	written to pin down an edge case

The format (docs/corpus-format.md) does not depend on this library, so other implementations can test against it; examples/check_other_implementation.py is a 40-line harness. 24 cases record an engine that behaves differently, and why the corpus expects something else; they are tabulated, together with the places where engines disagree with each other, in docs/engine-differences.md.

To contribute a case: capture the raw text (raw completion, special tokens visible — scripts/capture_gguf.py does it for GGUF models), add an object to corpus/<dialect>.json, write expected by hand, state the provenance honestly. All chunk-boundary tests pick it up automatically.

Performance

Parsing is not a bottleneck: a model streams roughly 0.001 MB/s of text per request.

Workload (2 MB of text)	4-character chunks (≈ one token)	64-character chunks	4096-character chunks
plain content	2.2 MB/s	24.8 MB/s	664 MB/s
reasoning + content	2.3 MB/s	25.0 MB/s	820 MB/s
many small JSON tool calls (hermes)	1.8 MB/s	5.5 MB/s	6.9 MB/s
one large JSON tool call (hermes)	2.6 MB/s	14.2 MB/s	21.3 MB/s
one large XML tool call (qwen3-coder)	1.4 MB/s	16.1 MB/s	152 MB/s
pythonic calls	2.2 MB/s	6.9 MB/s	7.8 MB/s
Mistral v11 calls	2.0 MB/s	6.5 MB/s	10.8 MB/s

At one token per feed() that is 350,000–650,000 tokens per second on one core — three orders of magnitude above what a model produces for a single stream.

Measured with benchmarks/throughput.py on CPython 3.12, one core of a shared 4-core development VM (so treat the figures as a floor). With token-sized chunks the cost is dominated by the per-call overhead of feed(); JSON arguments are validated character class by character class, which is why they are slower than plain text.

Limitations and maintenance reality

• New dialects arrive with model releases. A parser library is never finished; the vLLM tree alone has more than fifty tool-parser modules, many added in the last year. This project covers the formats in wide use that could be verified from source, and will always trail. The contribution path for a new dialect is deliberately strict (sources, corpus cases, a writer for the property tests) so that what is here stays right.
• Ten of fourteen dialects have not been run against a live model here. The big-model formats (DeepSeek, Mistral, GLM, Kimi, MiniMax, Qwen3-Coder, gpt-oss, Llama 4, Granite, Phi-4-mini, Gemma 4) are tested against engine test vectors and template layouts only. Real output from small models already changed the hermes and gemma parsers; expect the same when these meet reality. Corpus cases from real runs are the most valuable contribution.
• Engines are evolving their own parsers. llama.cpp replaced most hand-written parsers with an auto-parser that derives the format from the chat template by differential analysis (docs/autoparser.md, since March 2026). vLLM moved to a declarative parser engine (vllm/parser/), has a Rust front-end in progress (RFC #40846, experimental) and reads Hugging Face response_template metadata. If you serve with those engines, use their parsers; this library is for everything built outside them.
• Hugging Face response templates may make hand-written dialects unnecessary. transformers documents tokenizer.parse_response() and a streaming get_response_parser() driven by a response_template that a model ships next to its chat template. Where a model ships one and you already depend on transformers, that is the model author's own description of the format and should win. What remains for this library: environments without transformers (GGUF, MLX, raw completion APIs), models that ship no response template, valid-JSON argument streaming (response templates stream tool-call bodies as raw "dirty" chunks and parse at region close), and explicit robustness policies. Reading a response_template as a dialect definition is on the roadmap.
• Text-level, not token-level. A model that writes about <tool_call> in prose is indistinguishable from one that emits the special token. Engines with token ids can tell; this library cannot. reasoning="off" and the fallback-to-content rules limit the damage.
• The parser cannot see the prompt. Whether a template pre-opened <think> is not in the output. Defaults follow the reference templates; detect() and reasoning= cover the rest; a wrong setting shows up as a stray_reasoning_close issue.
• No schema validation or coercion. Arguments are valid JSON, not valid against your schema.
• Not supported yet: Phi-4-mini's <|tool_call|> form (only the functools[...] form that vLLM and llama.cpp test could be verified), Cohere Command, InternLM, LFM2, xLAM, Hunyuan, Step, Ernie, Olmo, MiniCPM5, MiniMax-M3, DeepSeek V3.2/V4 (DSML), Kimi K3, Jamba, Granite 3.2 prose-marker reasoning, Granite 20B function calling. See the roadmap.
• One edge of the no-buffering rule: if a model writes "arguments" before "name", the arguments are held until the name arrives.

This is an independent project. It is not affiliated with, sponsored by or endorsed by any inference engine or model vendor; model and product names are used only to say which format is which.

Related work

Project	What it is	How this differs
vLLM, SGLang, llama.cpp, Ollama, mlx-lm	The engines' built-in parsers; the ground truth this library was derived from and tested against	Importable without the engine; one state machine for streaming and non-streaming; explicit policies
transformers response templates	Declarative, model-shipped output parsing with a streaming parser	No transformers dependency; works when the model ships no template; arguments stream as valid JSON. Likely the long-term standard — see above
openai-harmony	Official renderer and token-level parser for the Harmony format	Harmony only; use it for gpt-oss when you have token ids
llm-router-utils	SGLang's reasoning and tool-call parsers and chat templates, extracted for routers; ~44 detectors	Far more formats than here. Depends on pinned transformers, openai, fastapi, xgrammar (→ torch)
tool-parser (Rust crate)	Streaming tool-call parsers for about a dozen formats, Apache-2.0	Rust. The PyPI package of the same name is a 0.0.1 placeholder announcing Python bindings
Tooletta	Parse and render tool calls between dialects for training and distillation data	Complete strings, not streams; rendering is its focus, which this library does not do
iyulab/ToolCallParser	.NET library normalising tool calls across provider APIs, plus Hermes/Qwen text markup	.NET; provider JSON rather than raw model text; streaming is limited to complete calls per chunk
partial-json-parser, json-repair	Parse incomplete / repair broken JSON	JSON only, re-parse per call; here the transcoder emits incremental fragments and knows the surrounding dialect

Projects that were looked for and not found under the names they were mentioned by (so nothing is claimed about them): a package or repository called toolcall-dialects, and the shepherd-ai/shepherd repository that the PyPI tool-parser placeholder links to.

FAQ

I use vLLM / SGLang / Ollama / llama-server. Do I need this? No. Their servers parse for you. This is for code that sits where those servers would: your own endpoint, a proxy in front of a raw completion API, batch post-processing, evals.

Which dialect is my model? detect(chat_template=tokenizer.chat_template) — the template contains the markers the model was trained on. llm_dialects.dialects() lists model families. For GGUF files the template is in the metadata (tokenizer.chat_template).

Why are the deltas different when I change the chunk size? The text of deltas follows the chunks; their concatenation does not. Compare coalesce(events) or use parse().

Why did whitespace disappear? Whitespace that touches a marker (</think>\n\n, \n<tool_call>) is treated as markup, uniformly for all dialects. Everything else is verbatim. docs/design.md

My model's reasoning_content is empty and the answer starts with thinking text. The chat template opened <think> in the prompt. Use reasoning="prefilled" (or detect()). The symptom is a stray_reasoning_close issue.

Arguments arrive in one piece instead of streaming. For GLM, MiniMax-M2 and Qwen3-Coder, a value's JSON type is unknown until it ends unless you pass tools=; then string parameters stream.

Can it constrain generation so the output is always valid? No — that needs the sampler (xgrammar, llguidance, GBNF). This library deals with whatever was generated.

Can it render tool calls and tool results back into the prompt? No. That is the chat template's job.

Is the default policy safe for agents? It is safe in the sense that it never crashes and never hides anything: check call.status and call.known before executing. For "refuse anything irregular", use STRICT.

Roadmap

• Real-output corpus cases for the dialects that have only engine vectors (needs larger models than fit the current test setup: Mistral Small, Phi-4-mini, Gemma 4, Granite 3.x, Qwen3-Coder).
• More dialects, in order of demand: Cohere Command, LFM2, DeepSeek V3.2/V4 (DSML), Kimi K3, MiniMax-M3, InternLM, xLAM, Granite 3.2 reasoning.
• Load a Hugging Face response_template as a dialect.
• Optional token-id input, to tell special tokens from look-alike text.
• OpenAI Responses API adapters.
• PyPI release once the API has settled with real users.

Contributing

Issues with raw model output are the best contribution; there is a "New dialect / wrong parse" form for them. For code:

pip install -e ".[dev]"
ruff check . && ruff format --check . && mypy && pytest

Adding a dialect requires cited sources (engine code at a commit and the model's chat template — not memory), corpus cases with honest provenance, and a writer in tests/renderers.py so the round-trip and chunk-boundary properties cover it. CONTRIBUTING.md has the checklist.

License

Apache-2.0. Copyright 2026 The llm-dialects Authors. Test vectors adapted from vLLM, SGLang, llama.cpp and Ollama are attributed in NOTICE.