Project Omega

A small vision model, served the way production serving actually works: a tiny PyTorch CNN, a FastAPI inference API, Prometheus metrics, structured logs, and a benchmark you can re-run. Repo project-omega, package and command omega.

The numbers, measured

Trained here, on CPU, from a clean checkout with one command. Not a screenshot of somebody else's benchmark:

Reproduce with python -m omega.vision.train --meta models/tinycnn.meta.json and python -m omega.vision.bench --checkpoint models/tinycnn.pt --runs 40. Raw figures live in docs/bench.json.

One reading worth stating plainly: throughput does not improve with batching here (5366 img/s at batch 1, 4303 img/s at batch 32). For a model this small, per-call overhead dominates, so batching gains nothing. That is a property of a 47 kB model, not of batching.

What this is not

This is a tiny CNN trained on synthetic shapes. The 100% validation accuracy is a statement about four synthetic shapes, not about real-world vision. There is no ImageNet result here, no pretrained weights, no GPU, and no accuracy claim I cannot reproduce in front of you. The reason the repo exists is the part around the model that demos usually skip: validation, readiness, metrics, logs, and honest load reporting.

The API

GET  /health    liveness only, deliberately says nothing about the model
GET  /ready     200 with the predictor's real source, 503 when nothing is loaded
GET  /model     what is actually serving: source, classes, sizes, load time
POST /predict   images in, top-k labels and scores out
GET  /metrics   Prometheus text exposition

python -m omega.vision.train                       # writes models/tinycnn.pt
OMEGA_CHECKPOINT=models/tinycnn.pt omega serve     # uvicorn on :8000
curl -s localhost:8000/predict -H 'content-type: application/json' \
  -d '{"images": [[[0, 0.5], [1.0]]], "top_k": 2}' | jq
curl -s localhost:8000/metrics | grep omega_predict

Exit codes: 0 success, 2 a missing dependency (for example omega train without torch installed).

Observability

Two decisions in the instrumentation worth naming. Scrapes of /metrics are not counted, because counting them inflates every dashboard that reads them. Unknown paths are labelled route="unmatched" rather than carrying the requested path, because a raw path is an unbounded label and one scanner can explode your time series. Requests that raise are still counted, as status="500", since the failures are exactly the requests you cannot afford to have invisible.

Logs are one JSON object per line with a request id that is honoured if the caller supplies x-request-id and generated otherwise.

How the torch-free core works

Serving depends on a Predictor protocol, not on PyTorch. create_app(predictor=...) takes any object with predict(batch) -> probabilities, so the whole HTTP surface is tested in under a second with no model, no GPU and no 200MB download. Torch sits behind a single adapter (omega/vision/torch_model.py) and is an optional extra.

The honesty is enforced, not just documented: if a checkpoint is missing, or torch is not installed, the service falls back to a clearly-labelled stub predictor, /model reports source: "stub", and /ready says which. It never pretends real weights are loaded.

CI has two jobs. test runs the torch-free suite on Python 3.9 and 3.12. ml installs CPU torch, runs the torch tests, trains the model, asserts validation accuracy of at least 0.9, and publishes the benchmark it measured as a build artifact.

Limits

• Synthetic four-class shapes only. A real vision task needs a real dataset, and this repo does not have one.
• Single process, single thread per request, no batching queue. Throughput numbers above are CPU on one machine.
• The stub exists for tests and for a clearly-labelled degraded mode. It is not a model.
• Python 3.9+ and CPU only. Nothing here has been run on a GPU.

License

MIT.