ollambda
Simple Flask based API server to proxy ollama calls to a more performant backend (such as llama.cpp)
Why should I use this package?
Honestly, you probably shouldn't. Not only is it currently nowhere near completion, there are plenty of alternatives.
Ollambda mostly exists as a personal learning tool.
But if you still want to, then continue on to see usage instructions and more info.
If you find a bug, feel free to open an issue, or make a pull request. Contributions are welcome!
How to Use
Installation
To install ollambda, either install using pip install ollambda or clone this repository and use pip install .
Development
A dev optional is provided for those wishing to make alterations. It includes extra utilities for testing, linting, etc.
venv
Venv usage is encouraged, the simplest way to manage this is to use uv:
cd ollambda
uv venv .venv
source .venv/bin/activate
uv pip install ollambda
Configuration
Currently ollambda proxies requests to an upstream llama.cpp server. The address is configured via environment variables or a YAML config file at ~/.ollambda/config.yaml:
| Setting | Config key | Default | Description |
|---|---|---|---|
| Host | HOST |
0.0.0.0 |
Host for bind address. |
| Port | PORT |
5000 |
Port for bind address. |
| Proxy IP | PROXY_IP |
127.0.0.1 |
Upstream LLM Server address. |
| Proxy port | PROXY_PORT |
8080 |
Upstream LLM Server port. |
| Storage Dir | STORAGE_DIR |
~/.ollambda |
Storage directory. Logs will be stored in a logs subdirectory here |
[!tip] All config values can be overridden with environment vars. For example, to override
HOST, simply setOLLAMBDA_HOST=<your host>
[!tip] You can also create a config.yaml file in the working dir where ollambda is running, this file will be preferred.
[!note] Ollambda uses ConfigLite for its config backend, which I also develop.
Running the server
Start with the bundled command (uses Gunicorn internally):
ollambda
This spawns one worker per CPU core and binds to 0.0.0.0:5000 by default.
For manual control, you can also invoke Gunicorn directly:
gunicorn ollambda:app -w 4 -b 0.0.0.0:5000
ollama API coverage
| Method | Endpoint | Description | Implemented? |
|---|---|---|---|
| POST | /api/generate |
Generate a completion (streaming or non-streaming) | N |
| POST | /api/chat |
Generate a chat completion (with tool calling support) | Y |
| POST | /api/create |
Create a model from another model, GGUF, or safetensors | N (no plans) |
| GET | /api/tags |
List locally available models | Y |
| POST | /api/show |
Show model information (details, modelfile, template, etc.) | Y |
| POST | /api/copy |
Copy a model to a new name | N |
| DELETE | /api/delete |
Delete a model | Y |
| POST | /api/pull |
Download a model from the Ollama library | N |
| POST | /api/push |
Upload a model to a model library | N (no plans) |
| POST | /api/embed |
Generate embeddings from a model | Y |
| GET | /api/ps |
List models currently loaded in memory | Y |
| GET | /api/version |
Retrieve the Ollama server version | Y |
| HEAD | /api/blobs/:digest |
Check if a file blob exists on the server | N (no plans) |
| POST | /api/blobs/:digest |
Push a file blob to the server | N (no plans) |
[!IMPORTANT] Current "implementation" may not be fully complete and API aligned.
[!NOTE] Source: https://github.com/ollama/ollama/blob/main/docs/api.md
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ollambda-0.3.0.tar.gz.
File metadata
- Download URL: ollambda-0.3.0.tar.gz
- Upload date:
- Size: 44.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d1ed05a99bb74b01d7c8fb30c84713a844beab29e042c327af992f7bac7b2f64
|
|
| MD5 |
bd3ee5a6774851b8e86b2b16d63d60f5
|
|
| BLAKE2b-256 |
9c1b54bd143f18461e495276b8200c493b297d783f2b604764f4d19db7b22346
|
File details
Details for the file ollambda-0.3.0-py3-none-any.whl.
File metadata
- Download URL: ollambda-0.3.0-py3-none-any.whl
- Upload date:
- Size: 42.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
369c5f5fca2f3e11d622b615ba7dcfa6b4c58196794fc28b58889c4e275b6d6d
|
|
| MD5 |
606774da0d4d9c98370ec1c5d302f983
|
|
| BLAKE2b-256 |
da8fd403af1cae7b093409ee49f5473bf153a9c1e6f11bc8f3ee8956952bd53e
|
