avrik-server

The run-time server for Avrik decision models: the prompt renderer, the letter readout, calibration, the /v1/systemone HTTP API, and the llama.cpp (CPU) and vLLM (GPU) backends.

An Avrik model reads a state and typed questions (yes/no, choice, score, multi-label) and returns a calibrated probability distribution over each question's options, in one forward pass with no generated text.

Most people install the avrik CLI, which depends on this package and runs it from a verified model download:

pip install llama-cpp-python==0.3.35 --no-deps --only-binary=:all: \
  --index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
pip install "avrik[cpu]"
avrik pull Avrik-Lite-4B
avrik serve Avrik-Lite-4B        # http://127.0.0.1:8080

To call the API from Python, use avrik-sdk.

What it does

1. Renders the state and question with a fixed prompt template whose SHA-256 is returned with every answer; options are letters A, B, C, ...
2. Reads the probability of every tokenizer form of each option letter (never top-k) and refuses the answer if the letters hold less than 98% of the probability mass.
3. Calibrates with per-type temperatures fitted for the exact build, bound to its hash; a calibration file for another build is refused.
4. Answers with the distribution, top, margin, confidence, abstain, and eight audit fields: model hash, adapter hash, calibration version, prompt-template SHA-256, allowed-token-set hash, input tokens, backend and precision.

Extras

The GPU backend talks to a vLLM server on loopback started with --logprobs-mode raw_logprobs; see each model card for the exact command.

Privacy

Inference makes no network calls, sends no telemetry and stores nothing. The server listens on 127.0.0.1 unless you configure otherwise.

Models

Built on Gemma 4 by Google (Apache-2.0). Licence: Apache-2.0. Security reports: avriktech.ai@gmail.com.

Website: www.avrik.ai · Models: huggingface.co/avrik-ai · Acceptable use policy