Accelana

Accelana is an inference-time memory manager for AI models.

Install

Requires Python 3.13. Supported computers are Apple Silicon Macs with macOS 14+ and Windows 11 ARM64 PCs. Windows uses CPU inference; Macs can use Apple GPU acceleration.

Use a fresh virtual environment:

python3.13 -m venv .venv
source .venv/bin/activate
python -m pip install accelana

On Windows, create the environment with py -3.13 -m venv .venv and activate it with .venv\Scripts\Activate.ps1 in PowerShell. Then run the same pip command. Do not install another PyTorch distribution in the Windows environment.

accelana --version
accelana pull Qwen/Qwen2.5-0.5B-Instruct
accelana run Qwen/Qwen2.5-0.5B-Instruct

Lana is bundled. Windows also installs accelana-torch-runtime automatically; Mac installs the official PyTorch package. Models are downloaded separately.

Commands

accelana list

Show cached models and whether each can run chat, plain-text continuation, GGUF, or another supported task:

accelana list

Output includes the Hugging Face model ID and a status such as runnable (chat, plain, ...) or unsupported: reason. Cached GGUF files appear as separate rows with --gguf-file NAME.gguf; without llama.cpp, their status is requires llama.cpp.

accelana pull MODEL [--gguf-file NAME.gguf]

Download a model from Hugging Face into the cache. By default, Accelana downloads selected SafeTensors, tokenizer, and configuration files:

accelana pull microsoft/Phi-4-mini-instruct

MODEL can be a Hugging Face model ID or the short name Phi-4-mini-instruct for microsoft/Phi-4-mini-instruct.

To pull one published GGUF file, name it explicitly. GGUF is outside the bundled SafeTensors flow. Python wheel installations need a separate llama.cpp runner (on macOS, brew install llama.cpp):

accelana pull ggml-org/SmolLM2-135M-GGUF --gguf-file SmolLM2-135M-Q4_K_M.gguf

This downloads only that GGUF file into the Hugging Face cache. It does not convert a SafeTensors model or select a quantization automatically.

accelana run MODEL [options]

Start interactive chat with a cached SafeTensors model (pull it first if it is not in the cache):

accelana run microsoft/Phi-4-mini-instruct
accelana run Phi-4-mini-instruct

run does not download a missing model. Its error shows the matching accelana pull command, including --gguf-file when needed.

Run a cached GGUF file or one already on disk:

accelana run ggml-org/SmolLM2-135M-GGUF --gguf-file SmolLM2-135M-Q4_K_M.gguf
accelana run /path/to/model.gguf

If a cached repository has one GGUF and no SafeTensors checkpoint, run MODEL selects it. Multiple GGUF files require --gguf-file. SafeTensors remains the default when both formats are cached. GGUF runs hand the terminal to llama.cpp, so its prompt and exit controls apply; --plain and --vram-gb are unavailable for GGUF.

For a text generation model without a chat template, continue plain text instead:

accelana pull Intel/tiny-random-gpt2
accelana run Intel/tiny-random-gpt2 --plain

Plain mode keeps one growing text. Each new line follows the previous prompt and generated output; it does not add chat roles. --plain also works with chat models when plain continuation is wanted. Generation stops at an end token or the model's context limit. If the context fills, the next line cannot continue that session.

For SafeTensors runs, enter a message at the >>> prompt. Enter /bye, /exit, exit, or quit to end the session. Ctrl-D also exits; Ctrl-C interrupts.

Options:

Option	Meaning
--gguf-file NAME.gguf	Run this named GGUF file from a cached Hugging Face repository. Omit it when the repository has exactly one GGUF and no SafeTensors checkpoint.
--plain	Continue plain text with a SafeTensors model without requiring a tokenizer chat template.
--vram-gb GB	Set a positive GPU weight budget in decimal GB for a SafeTensors model. Automatically retain as many layers as fit. Without it, the weight budget is the full checkpoint size.

Example:

accelana run microsoft/Phi-4-mini-instruct --vram-gb 8

Accelana automatically chooses how many layers remain loaded between tokens. Every remaining layer still runs: its weights load just before its calculation and release afterward. All model layers participate in every generated token; memory-saving mode does not shrink the model or skip calculations.

Accelana keeps non-layer weights, such as embeddings and the output head, loaded throughout the session. The budget must fit these weights, the retained layers, and the largest remaining layer. Budgets that cannot fit this working set fail before checkpoint weight loading. Automatic selection retains the largest prefix of layers that fits, using their actual sizes. When the entire checkpoint fits, all layers stay loaded.

--vram-gb limits managed weights, not total process or GPU memory. Conversation cache, calculation results, model buffers, and runtime allocations need additional memory. Retaining fewer layers reduces weight memory but increases repeated loading.

accelana delete MODEL [--yes]

Delete all cached revisions for the exact Hugging Face model ID, including any GGUF files in those revisions:

accelana delete microsoft/Phi-4-mini-instruct

Accelana shows the estimated reclaimed space and asks for confirmation. Type y or yes to continue; any other response cancels. Use --yes or -y to skip the prompt:

accelana delete microsoft/Phi-4-mini-instruct --yes

Use the full model ID for deletion. If the model is not cached, the command reports that and exits successfully.

Python API

The Python generate() helper uses chat formatting. The package also exposes task helpers for model snapshots or Hugging Face IDs. These functions load and close the model for each call; pass vram_gib to use limited-memory residency.

from lana_memory.tasks import classify, embed, generate, rank

model = "microsoft/Phi-4-mini-instruct"
reply = generate(model, "Explain gravity in one sentence.")

Task helpers:

from lana_memory.tasks import classify, embed, rank

vectors = embed("owner/embedding-model", "hello", vram_gib=8)
scores = rank("owner/ranking-model", "query", ["candidate one", "candidate two"])
labels = classify("owner/classification-model", "hello")

Replace each owner/... example with a cached model ID suitable for that task, or a local snapshot path.

For reproducible offline use, pass a local Hugging Face snapshot directory instead of a model ID. If given an ID, these helpers require it to be cached first; download it with accelana pull MODEL.

Supported model files and common limits

The SafeTensors path requires a compatible text-only causal architecture supported by the installed Transformers version and a checkpoint Accelana can map for weight residency. Chat also requires a usable tokenizer chat template; plain mode does not. Custom remote model code is disabled. Quantized SafeTensors checkpoints and multimodal chat models are not supported by this path. GGUF uses an installed llama.cpp runner instead; Accelana does not manage its weight residency. Single-file, text-only GGUF models are supported in this path. Hugging Face filter tags alone do not guarantee compatibility. Unsupported models are reported by list or run with a reason.

The default Hugging Face download includes selected model, tokenizer, and configuration files used by Accelana; it does not download arbitrary repository files. pull --gguf-file downloads only the named GGUF file. Internet access is needed for pull; run requires a cached model or a local GGUF file.

Errors and exit behavior

• Invalid command arguments are rejected by the CLI, with help text for the command.
• run returns a nonzero status for missing cached models or runtime errors and reports unsupported snapshots before starting a session.
• Ctrl-C during a run or delete returns status 130.
• A cancelled delete and a delete of a model that is not cached complete without deleting anything.
• If a model cannot be loaded, check accelana list for its support reason. For GGUF, confirm llama.cpp is installed and the selected file is supported by that runner.

Bundled Lana

Accelana wheels include the Lana 4.1.0 Rust executable, compiler bytecode, standard library, and a private copy of its Python bridge. They come from oponite/lana v4.1.0, commit 4e1606a390f44778029ff45bf0b8af7ff9b41272. Lana's Apache-2.0 license is included beside the bundled runtime and bridge.