entail

Keep what a value means intact across LLM inference-stack boundaries.

An inference stack is a chain of parts — checkpoint and config, loader, engine, kernels, quantization, cache. Each part can be correct on its own terms while the meaning of a value is lost between two of them: a declared property the chosen kernel ignores, a setting that arrives under a name nobody reads any more, a cache that silently loses a token. The output is then wrong, fluently and without a warning.

entail declares what a value means where it crosses a boundary, checks the declaration against the real thing, and when they disagree it resolves the mismatch first — routes the value to a consumer that honours it, or converts it to the form the consumer reads — and prints one line saying what it changed. It stops only when no fix exists.

The name is the logical sense of entail: what a checkpoint declares must entail what the engine executes. (ent·AI·L — an AI library.)

Status: research prototype (alpha). Measured on one RTX 4070 Ti with the versions under "Tested with".

Why: a case measured end to end

Restating a model's own rope_scaling at launch — the route model cards give for enabling YaRN — drops rope_theta under transformers 5, and the engine silently falls back to a RoPE base of 10,000.

Llama-3.2-3B-Instruct, greedy, GSM8K (first 500 on vLLM, first 200 on SGLang):

The degraded runs are identical to an explicit rope_theta = 10000. Outputs stay fluent; the answers are wrong. Qwen3 dense models happen to be safe because those model files fill in 1,000,000; Llama, Qwen3-MoE, Gemma and others do not.

Install

pip install "git+https://github.com/wwoosshh/entail"

Install it into the same environment as your engine (vLLM, SGLang or transformers). entail has no dependencies of its own. Once it is on PyPI: pip install entail-ai (the import name stays entail). uv pip install works the same way.

Check what it sees:

entail doctor

Use

Turn it on with one environment variable; nothing else changes.

ENTAIL=load vllm serve meta-llama/Llama-3.2-3B-Instruct --hf-overrides '{"rope_scaling": {...}}'
ENTAIL=load python -m sglang.launch_server --model-path ... --json-model-override-args '{...}'
ENTAIL=load python your_transformers_script.py

When it changes something, it says so:

[entail] resolved LlamaConfig.rope_scaling was given after the config was built; it replaces rope_parameters and
would have dropped rope_theta=500000.0, ...; kept it, as config.json would

From inside a script:

import entail
entail.enable()          # mode="load", policy="resolve"; child processes inherit it

How it reaches engine worker processes

vLLM and SGLang run the model in processes they start themselves. pip install puts one file, entail-autoinstall.pth, into site-packages; Python reads it at every start-up. Its single line checks the environment and does nothing unless ENTAIL is set. entail hook status|install|uninstall shows or manages it (an editable install does not place it — run entail hook install).

What it does

Resolves

Checks (and stops, when nothing can resolve it)

• attention properties against each engine's backends, before any weight is read
• config keys that would be swallowed, and tied-embedding declarations against the checkpoint
• vLLM weights after repacking: layout, stride and a value sample against the declared transform
• weights in memory against the checkpoint file (ENTAIL_SOURCE=1)
• the KV cache contract (a request holds what it needs, nothing shrank) on transformers, vLLM's paged cache and SGLang

entail preflight --model /path/to/model --engine sglang --list runs the start-up checks without starting a server.

Configuration

Tested with

transformers 5.12.1 and 5.17.0, vLLM 0.30.0, SGLang 0.5.20, torch 2.13–2.14, Python 3.12, one RTX 4070 Ti (12 GB). Other versions may work; entail doctor prints what is installed. On transformers 4.x the RoPE resolver has nothing to do and stays out of the way.

The capability table (which backend honours what) records its evidence per entry, and only entries marked measured are used as resolution targets. The measurement scripts and raw results are kept in the author's research workspace and are not in this repository yet.

Development

git clone https://github.com/wwoosshh/entail && cd entail
pip install -e .
entail hook install          # editable installs do not place the start-up hook
PYTHON=python bash tests/run_all.sh

Tests that need a local model look in ENTAIL_TEST_MODELS (default ~/models) and skip when it is absent.

한국어 안내: README.ko.md

License

See LICENSE.