avrik-sdk

Python client for Avrik decision models.

An Avrik model reads a state (text or JSON) and typed questions, and returns a calibrated probability distribution over each question's options. It runs in one forward pass and generates no text. This package is a thin, typed HTTP client for the /v1/systemone endpoint served by avrik serve. Its only dependencies are httpx and pydantic, it sends no telemetry and it contacts only the URL you give it.

pip install avrik-sdk

Python 3.10 or later.

Start a server

avrik-sdk talks to a running server. To run one on your own machine, on a CPU, with the Lite model:

pip install llama-cpp-python==0.3.35 --no-deps --only-binary=:all: \
  --index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
pip install "avrik[cpu]"
avrik pull Avrik-Lite-4B
avrik serve Avrik-Lite-4B          # http://127.0.0.1:8080

Avrik-Pro-12B and Avrik-Max-31B serve the same API from a GPU. Their model cards on Hugging Face give the commands.

Ask a question

from avrik_sdk import Client, choice, noul, score

state = (
    "Customer bought headphones 12 days ago, unopened, receipt attached. "
    "Policy: unopened items within 30 days get a full refund."
)

with Client("http://127.0.0.1:8080") as client:
    d = client.decide(
        state,
        {
            "refund": noul("Should the refund be approved?"),
            "action": choice("Which action?", ["approve", "deny", "escalate"]),
            "risk": score("How risky is this refund?", ["low", "medium", "high"]),
        },
    )

print(d["refund"].noul)            # P(yes), between 0 and 1
print(d["action"].choice)          # the top option, e.g. "approve"
print(d["action"].probabilities)   # {"approve": ..., "deny": ..., "escalate": ...}, sums to 1
print(d["risk"].score, d["risk"].legend)

Every answer carries:

• probabilities: the calibrated distribution over the offered options, summing to 1;
• top and margin: the top probability, and its lead over the second;
• confidence: the top probability, corrected for chance;
• abstain: whether the model declined to answer.

noul gives P(yes), choice the top option, and score the expected level, counted from 0 for the first level.

Questions can also be plain dicts in the wire shape:

{"type": "choice", "instructions": "Which team?",
 "criteria": {"billing": "payments and refunds", "technical": "bugs", "account": "login"}}

Confidence measures which offered option fits best, not whether any of them fits. If "none of these" can be the answer, offer it as an option. To give the model a way to decline, pass abstain_option=True to a question builder. That adds "Insufficient information to decide" as the last option. When that option wins, the answer has abstain=True and reports its mass in unknown_probability.

Several labels at once

from avrik_sdk import multi_label

d = client.decide(ticket, {"tags": multi_label("Which apply?", ["billing", "bug", "urgent"])})
print(d["tags"].scores, d["tags"].selected)   # labels at or above 0.7 are selected

Batches

d = client.decide_batch(
    [{"id": "t1", "text": "Refund for an unopened item, day 12."},
     {"id": "t2", "text": "Refund for a used item, day 45."}],
    {"refund": noul("Should the refund be approved?")},
)
for r in d.results:
    print(r.id, r["refund"].noul if r.error is None else r.error.code)

Results come back in input order. If the server refuses one input, its error appears in that input's place and the other inputs are still answered.

Async

from avrik_sdk import AsyncClient

async with AsyncClient("http://127.0.0.1:8080") as client:
    d = await client.decide(state, {"refund": noul("Should the refund be approved?")})

Errors

from avrik_sdk import AvrikError, RateLimited

try:
    d = client.decide(state, questions)
except RateLimited as e:          # 429 at capacity or over a rate limit; 503 backend down
    print("retry after", e.retry_after, "s")
except AvrikError as e:           # any other refusal, with a stable code
    print(e.status, e.code, e.message, e.field)

Error codes are stable strings, for example:

A server never switches to a different model when it is busy. It refuses. The client doesn't retry unless you ask: Client(..., max_retries=3) waits for each Retry-After (at most 60 seconds) before retrying.

Audit

Each decision records what produced it:

d.audit.model_hash            # weights identity
d.audit.calibration_version
d.audit.prompt_template_sha256
d.audit.backend, d.audit.precision
d.usage.input_tokens

On the same backend and precision, the same hashes give the same answers. Log these fields with each decision to replay it later.

Discovery

client.health().ok                 # the backend is up
[m.name for m in client.models()]  # e.g. ["Avrik-Lite-4B"]
client.models()[0].avrik           # hashes, backend, max_options, max_context_tokens

Authentication and other servers

Client(url, api_key="...") sends Authorization: Bearer ... when the server requires a key. The client works with any endpoint that speaks the /v1/systemone shape. Fields added by newer servers are kept on the response objects as extra attributes.

Licence

Apache-2.0. © Avrik AI.

Website: www.avrik.ai · Models: huggingface.co/avrik-ai · Acceptable use policy