LocalForge Inference Lab
LocalForge measures and compares LLM inference engines on your own machine. Connect an engine you already run (Ollama, llama.cpp, vLLM, SGLang, an OpenAI-compatible server, and more), build a repeatable workload, run it, and read the results as charts: time to first token, latency percentiles, tokens per second, and host CPU and memory during the run. Compare runs side by side to see which model or setting is faster on your hardware.
Everything stays local. The app listens only on 127.0.0.1, keeps its data in a local SQLite
database, and never downloads models.
Install and run
You need uv and an inference engine with a model, for example
Ollama with ollama pull llama3.2:1b.
uvx localforge-inference-lab
This starts LocalForge and opens it in your browser. Press Ctrl+C in the terminal to stop it.
To keep a localforge command installed instead:
uv tool install localforge-inference-lab
localforge
Options
| Option | Purpose |
|---|---|
--port 8123 |
Listen on a specific port. By default 8000, or a free port if it is taken. |
--data-dir PATH |
Keep the database and datasets in another folder, for example a bigger drive. |
--no-browser |
Start without opening a browser tab. |
--version |
Print the installed version. |
First benchmark
- Engines: add your engine (for Ollama keep the default
http://127.0.0.1:11434), probe it, and pick a model. - Workloads: choose a preset, save the plan, and launch it.
- Runs: follow the run live and read its charts and measurements.
- Experiments: select several runs and compare them.
Source, documentation, and the development setup are at https://github.com/HaisamAbbas/Local-Forge.
1 maintainer
Metadata
1 maintainer
Release files for localforge-inference-lab 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| localforge_inference_lab-0.2.1.tar.gz | 3.3 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| localforge_inference_lab-0.2.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 6.6 MB
Release files / localforge_inference_lab-0.2.1.tar.gz
| Download URL | localforge_inference_lab-0.2.1.tar.gz |
|---|---|
| Size | 3.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
882267b55f9908d8ae3b7b0b7cf1fac5781a6138e5f4ccdb507ec66526c9254e
|
|
BLAKE2b-256 checksum How to use checksums |
1d7c906aa63e4acd19cf87627b3ffa06230b17392bef883d362b3b117b212487
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / localforge_inference_lab-0.2.1-py3-none-any.whl
| Download URL | localforge_inference_lab-0.2.1-py3-none-any.whl |
|---|---|
| Size | 3.3 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b31c97aefde35b2513c64cab1dbcdbb8ebeed6eee1f50065a0662121321f4af0
|
|
BLAKE2b-256 checksum How to use checksums |
4a91efd628c846431fb475fbc611f4f6226e7443a8bc75cb7c3c39b925a48feb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
