ollambda 0.1.9

Add your description here

ollambda

Simple Flask based API server to proxy ollama calls to a more performant backend (such as llama.cpp)

Why should I use this package?

Honestly, you probably shouldn't. Not only is it currently nowhere near completion, there are plenty of alternatives.

Ollambda mostly exists as a personal learning tool.

But if you still want to, then continue on to see usage instructions and more info.

If you find a bug, feel free to open an issue, or make a pull request. Contributions are welcome!

How to Use

Installation

To install ollambda, either install using pip install ollambda or clone this repository and use pip install .

venv

Venv usage is encouraged, the simplest way to manage this is to use uv:

cd ollambda
uv venv .venv
source .venv/bin/activate
uv pip install .

Configuration

Currently ollambda proxies requests to an upstream llama.cpp server. The address is configured via environment variables or a YAML config file at ~/.ollambda/config.yaml:

Setting	Config key	Default	Description
Host	HOST	0.0.0.0	Host for bind address.
Port	PORT	5000	Port for bind address.
Proxy IP	PROXY_IP	127.0.0.1	Upstream LLM Server address.
Proxy port	PROXY_PORT	8080	Upstream LLM Server port.
Log Dir	LOG_DIR	~/.ollambda/logs	Directory to store logs.

[!tip] All config values can be overridden with environment vars. For example, to override HOST, simply set OLLAMBDA_HOST=<your host>

[!tip] You can also create a config.yaml file in the working dir where ollambda is running, this file will be preferred.

[!note] Ollambda uses ConfigLite for its config backend, which I also develop.

Running the server

Start with the bundled command (uses Gunicorn internally):

ollambda

This spawns one worker per CPU core and binds to 0.0.0.0:5000 by default.

For manual control, you can also invoke Gunicorn directly:

gunicorn ollambda:app -w 4 -b 0.0.0.0:5000

ollama API coverage

Method	Endpoint	Description	Implemented?
POST	/api/generate	Generate a completion (streaming or non-streaming)	N
POST	/api/chat	Generate a chat completion (with tool calling support)	Y
POST	/api/create	Create a model from another model, GGUF, or safetensors	N (no plans)
GET	/api/tags	List locally available models	Y
POST	/api/show	Show model information (details, modelfile, template, etc.)	Y
POST	/api/copy	Copy a model to a new name	N
DELETE	/api/delete	Delete a model	Y
POST	/api/pull	Download a model from the Ollama library	N
POST	/api/push	Upload a model to a model library	N (no plans)
POST	/api/embed	Generate embeddings from a model	Y
GET	/api/ps	List models currently loaded in memory	Y
GET	/api/version	Retrieve the Ollama server version	Y
HEAD	/api/blobs/:digest	Check if a file blob exists on the server	N (no plans)
POST	/api/blobs/:digest	Push a file blob to the server	N (no plans)

[!IMPORTANT] Current "implementation" may not be fully complete and API aligned.

[!NOTE] Source: https://github.com/ollama/ollama/blob/main/docs/api.md