llm-cage 0.0.1

Cage: run local-LLM components in a network-isolated OS process, hardware access intact, with a friction-free call interface back to the rest of your app.

Cage (llm-cage)

Status: placeholder release — reserving the name while the project is designed. Not yet functional.

Cage runs a local-LLM component of your application inside a separate OS process with network access denied by default (loopback allowed, for talking to a local model server), while keeping local hardware acceleration (GPU/Metal/CUDA) intact and giving you a friction-free way to call into it from the rest of your app:

from cage import Cell

cell = Cell()

@cell.expose
def ask_local_llm(prompt: str) -> str:
    import ollama
    return ollama.chat(model="llama3", messages=[{"role": "user", "content": prompt}])["message"]["content"]

answer = ask_local_llm("what's 2+2?")  # runs inside the network-restricted process

Design details are being worked out as a public RFC series before implementation starts. This release exists solely to reserve the package name.