Why · Decision rule · Start here · Stateful rules · Patterns · Documentation

OC-Sidememory blocks tokens that violate stateful rules: duplicates, counted predicates, runtime membership, and cross-field relations.

Why OC-Sidememory?

Structure alone can still repeat an action:

{"actions":["search","search","search"]}

With semantic memory:

{"actions":["search","compare","answer"]}

Token acceptance rule

For guide state $\sigma$, token $x$, DFA decision $D$, and semantic decision $S$:

A(\sigma,x)=D(\sigma,x)\land S(\sigma,x)

The semantic layer can remove structurally valid candidates, but it cannot add new candidates:

M_{OC}=M_{DFA}\land\neg M_{deny}

What it enforces

Unsupported schema combinations fail during compilation. See the supported profile and limitations.

How uniqueness changes a mask

After completing "red" in a unique array:

Exact guarantees

Uniqueness:

U(v)\iff C(v)\notin H_a

Counted contains predicate:

c_a=\sum_{i=1}^{n}\mathbf{1}[Q(v_i)],\qquad l\le c_a\le u

Exact equality:

E(v,r)\iff C(v)=C(r)

Imported membership:

I(v,\mathcal{M})\iff C(v)\in\mathcal{M}

Pure probe and rollback:

P_x(\sigma)=\sigma

R_k(T_{x_k}(\cdots T_{x_1}(\sigma)))=\sigma

Installation

Python:

pip install oc-sidememory

For the model-backed walkthrough:

pip install "oc-sidememory[transformers]"

Rust:

cargo add oc-sidememory

From a checkout:

uv sync
uv run maturin develop --release

Start here: prompt to guaranteed JSON

Qwen writes the plan. OC-Sidememory keeps the activities distinct.

import oc_sidememory
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Qwen/Qwen2.5-0.5B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID).cpu().eval()

schema = {
    "type": "object",
    "properties": {
        "city": {"type": "string", "enum": ["Kyoto", "Paris", "Reykjavik"]},
        "pace": {"type": "string", "enum": ["relaxed", "balanced", "busy"]},
        "activities": {
            "type": "array",
            "items": {
                "type": "string",
                "enum": ["temples", "food tour", "museum", "hiking"],
            },
            "minItems": 3,
            "maxItems": 3,
            "uniqueItems": True,
        },
    },
    "required": ["city", "pace", "activities"],
    "additionalProperties": False,
}

prompt = """Create a balanced Kyoto itinerary.
Choose exactly three different activities from temples, food tour, museum, and hiking.
Return only the JSON object."""

messages = [
    {"role": "system", "content": "Return only JSON matching the requested schema."},
    {"role": "user", "content": prompt},
]

runtime = oc_sidememory.from_transformers(model, tokenizer)
generate = oc_sidememory.Generator(runtime, schema)
itinerary = generate(messages, max_new_tokens=128)  # parsed dict, not JSON text

The model, tokenizer vocabulary, and compiled schema are reused. Each call gets independent semantic state, so the same generator also accepts a batch of prompts or chats. Pass a Pydantic-compatible type instead of schema to receive a validated instance.

import json
import torch

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
input_ids = tokenizer(text, return_tensors="pt").input_ids
vocabulary = oc_sidememory.Vocabulary.from_transformers(tokenizer)
compiled = oc_sidememory.compile_schema(
    json.dumps(schema), vocabulary, model.get_output_embeddings().weight.shape[0]
)
guide = oc_sidememory.SidememoryGuide(compiled)

generated = []
past_key_values = None
with torch.inference_mode():
    for _ in range(128):
        result = model(
            input_ids=input_ids,
            past_key_values=past_key_values,
            use_cache=True,
        )
        past_key_values = result.past_key_values

        allowed = torch.tensor(guide.get_tokens(), dtype=torch.long)
        if allowed.numel() == 0:
            raise RuntimeError("schema has no valid continuation")

        logits = result.logits[0, -1]
        token = allowed[torch.argmax(logits[allowed])].item()
        guide.advance(token)

        if token == vocabulary.get_eos_token_id():
            break
        generated.append(token)
        if guide.is_accepting():
            break
        input_ids = torch.tensor([[token]])
    else:
        raise RuntimeError("generation exceeded max_new_tokens")

output = tokenizer.decode(generated)
itinerary = json.loads(output)

Output from the verified CPU run:

{
  "activities": ["temples", "food tour", "museum"],
  "city": "Kyoto",
  "pace": "balanced"
}

Run the complete travel-itinerary example:

python examples/python/00_travel_itinerary.py

The decoding flow

prompt -> model logits -> OC-Sidememory valid token IDs -> choose token
       -> advance model cache + semantic state -> repeat -> valid JSON

Generator compiles once per tokenizer/schema pair and creates one SidememoryGuide per sequence.

Compose stateful rules

One guide can enforce all of these:

• actions must contain three distinct values and include "answer";
• customer must exist in an immutable allowlist loaded for this request;
• confirmation must exactly match the earlier request_id, even when numbers use different valid JSON spellings such as 1 and 1.0.

Prompt:

Build an action plan for customer cust_7 and assign a numeric request ID.
Use search, compare, and answer exactly once. Repeat the request ID in confirmation.
Return only the JSON object.

schema = {
    "type": "object",
    "properties": {
        "request_id": {"type": "number"},
        "customer": {"type": "string"},
        "actions": {
            "type": "array",
            "items": {"enum": ["search", "compare", "answer"]},
            "minItems": 3,
            "maxItems": 3,
            "uniqueItems": True,
            "contains": {"const": "answer"},
        },
        "confirmation": {"type": "number"},
    },
    "required": ["request_id", "customer", "actions", "confirmation"],
    "additionalProperties": False,
}

extensions = {
    "version": 1,
    "objects": [{
        "schemaPath": "$",
        "propertyOrder": ["request_id", "customer", "actions", "confirmation"],
        "captures": [{"name": "rid", "sourceProperty": "request_id"}],
        "relations": [
            {
                "targetProperty": "customer",
                "operator": "memberOf",
                "import": "live_customers",
            },
            {
                "targetProperty": "confirmation",
                "operator": "equal",
                "capture": "rid",
            },
        ],
    }],
}

imports = {
    "identity": "customer-snapshot",
    "version": "catalog-v1",
    "data": {"live_customers": ["cust_7", "cust_9"]},
}
generate_plan = oc_sidememory.Generator(
    runtime, schema, extensions=extensions, imports=imports
)
plan_messages = [
    {"role": "system", "content": "Return only JSON matching the requested schema."},
    {
        "role": "user",
        "content": "Build an action plan for cust_7. Use search, compare, and answer exactly "
                   "once, and repeat the numeric request ID in confirmation.",
    },
]
plan = generate_plan(plan_messages, max_new_tokens=160)

Run the complete model-backed version:

python examples/python/06_customer_action_plan.py

A verified run produced:

{
  "request_id": 1234567890,
  "customer": "cust_7",
  "actions": ["search", "compare", "answer"],
  "confirmation": 1234567890
}

Patterns you can ship

Validate candidates without running a model

The same guide validates tokenized candidates:

schema = {
    "type": "array",
    "items": {"type": "string", "enum": ["red", "green", "blue"]},
    "minItems": 3,
    "maxItems": 3,
    "uniqueItems": True,
}

compiled = oc_sidememory.compile_schema(json.dumps(schema), vocabulary, tokenizer.vocab_size)
guide = oc_sidememory.SidememoryGuide(compiled)

valid = tokenizer.encode('["red","green","blue"]', add_special_tokens=False)
duplicate = tokenizer.encode('["red","red","blue"]', add_special_tokens=False)

assert guide.accepts_tokens(valid, finish=True)
assert not guide.accepts_tokens(duplicate, finish=True)

accepts_tokens, probe, advance, and mask construction use the same Rust transition routine. Python does not contain a second semantic validator.

Command-line interface

oc-sidememory capabilities
oc-sidememory compile --schema schema.json --vocabulary vocabulary.json
oc-sidememory check --schema schema.json --vocabulary vocabulary.json \
  --tokens tokens.json --finish

Commands emit versioned JSON. Vocabulary files store token bytes as base64, so token data does not need to be UTF-8. See the CLI reference.

Performance

Mask latency depends on candidate count, token bytes, emitted events, predicate work, equality work, and current semantic history. Recorded release-mode measurements and their environment are in docs/performance.md. They are engineering measurements, not universal throughput claims.

Documentation

The example ladder covers uniqueness, contains, capture equality, imported membership, and composed rollback using public APIs.

Development

cargo fmt --all --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features
uv run pytest -q

Deterministic correctness tests require no model or network. See CONTRIBUTING.md and SECURITY.md.

Provenance and license

OC-Sidememory is independently developed from an Apache-2.0 outlines-core source import and is not endorsed by its upstream authors. See PROVENANCE.md, MODIFICATIONS.md, NOTICE, and LICENSE.