Kayak

Typed AI decisions for Python. Classify text, route requests, rank candidates, and ask structured questions using a local model or a service you operate.

Quickstart · Documentation · Examples · Contributing

Give Kayak text state, named questions, and candidate descriptions. It returns validated answers, the underlying distributions, and model identity. Your application decides how to use the result.

The same operations are available on a local Model, Client, and AsyncClient. Optional Laya and Jev adapters use the same named questions while preserving each provider's result semantics.

Quickstart

Kayak 0.4.0 requires Python 3.11+. Install it in your project with uv:

uv add 'kayak==0.4.0'

The base package provides typed values, HTTP clients, and evaluation. Add the local extra for local inference or serve for the HTTP service. See release validation for the tested scope and remaining model and hardware checks.

For the runnable examples below, download and extract the source distribution. It includes the documentation, examples, tests, and validation tools:

cd kayak-0.4.0
uv sync

Check the client integration without a model, service, or accelerator:

uv run -m examples.mock_integration

This command uses a simulated HTTP response. To run actual inference, use a local model or connect to an existing Kayak service as shown below.

Make a local decision

The default model is CLM-v0.1-8B. The first load downloads approximately 16 GB of encoder weights and 76 MB of projection heads. Runtime memory exceeds the weight size; check the hardware guide before loading.

Save this as decide.py in the checkout:

import kayak
from kayak import Choice

questions = {
    "department": Choice(
        instructions="Which team should handle this request?",
        criteria={
            "billing": "Charges, invoices, and refunds",
            "technical": "Bugs and service outages",
        },
    )
}

with kayak.load(device="auto") as model:
    result = model.decide(
        state="I was charged twice for my subscription.",
        questions=questions,
    )

answer = result.answers["department"]
print(answer.choice)
print(answer.probabilities)
print(result.model.fingerprint)

uv run --extra local decide.py

Keep the model context open to reuse the loaded weights. device="auto" selects available CUDA, then MPS, then CPU. Explicit device, precision, cache, and batch settings are described in the API reference.

Serve once, call from your application

Start a service on a machine with sufficient memory:

uv run --extra serve kayak serve --device auto

After loading and a readiness inference, it listens on http://127.0.0.1:8000. In the quickstart program, replace the model context with a client context:

with kayak.Client(base_url="http://127.0.0.1:8000") as client:
    result = client.decide(
        state="I was charged twice for my subscription.",
        questions=questions,
    )

The base client requires no inference libraries. In another Python project, install it with uv add 'kayak==0.4.0'. Async applications use async with kayak.AsyncClient(...) and await the same operations.

The service admits one inference request at a time and returns 503 when busy. Clients make one attempt per call. Configure authentication, TLS at your network boundary, and timeouts using the serving guide.

Use retrieved evidence

Pass your query and retrieved passages as text state, then ask named questions with judge. The RAG decision example shows Choice, Noul, and Score together. Your application owns retrieval and any text generation. RAG evaluation checks recorded retrieval, reranking, context, and answers; the experiment guide covers configuration, repeated runs, and quality gates.

Evaluate the result

CLM scores are scaled cosine similarities. Probabilities are relative shares among the supplied candidates, not calibrated confidence. A candidate is selected even when every option is unsuitable; application thresholds and fallback rules need evaluation against reviewed labels.

Use the evaluation API for your data and the use-case evaluation map for starter datasets and application checks. Hardware validation, numerical conformance, and task quality are separate evidence; their current scope is recorded in the release checklist.

Documentation

The documentation index organizes guides by integration, operation, evaluation, and contribution.

Contributing

See CONTRIBUTING.md for development setup, checks, bug reports, and pull requests. Code changes follow the engineering conventions.

License

Kayak is licensed under Apache 2.0. It builds on Contrastive Language Models and the Qwen3 encoder. See NOTICE for attribution. Model weights are downloaded separately under their upstream licenses.