momahub 0.2.0

Distributed AI inference hub — Mixture of Models on Ollama

MoMa Hub

Distributed AI inference on consumer GPUs — Mixture of Models on Ollama

pip install momahub

---

What is MoMa Hub?

MoMa Hub is the infrastructure layer for the MoMa (Mixture of Models on Ollama) vision: a federated network where anyone with a gaming GPU can contribute inference capacity to the global AI commons — and route tasks to the right model on the right GPU automatically.

Inspiration	What they shared	MoMa Hub equivalent
SETI@home	Idle CPU cycles	Idle GPU cycles
Airbnb	Spare bedrooms	Spare VRAM
Docker Hub	Container images	Ollama runtime configs
GitHub	Source code	SPL prompt scripts

A GTX 1080 Ti (11 GB VRAM, ~$150 used) runs any 7B model in real time. There are millions sitting idle in gaming PCs. MoMa Hub organises them.

---

Quick Start

# 1. Install
pip install momahub

# 2. Start the hub server
momahub serve --port 8765

# 3. Register your Ollama node (in another terminal)
momahub register --node-id home-gpu-0 \
                 --url http://localhost:11434 \
                 --gpu "GTX 1080 Ti" \
                 --vram 11 \
                 --models qwen2.5:7b --models mistral:7b

# 4. Check nodes
momahub nodes

# 5. Run inference through the hub
momahub infer --model qwen2.5:7b --prompt "Explain attention mechanisms"

Python SDK

import asyncio
from momahub import MoMaHub, NodeInfo, InferenceRequest

hub = MoMaHub()
hub.register(NodeInfo(
    node_id="home-gpu-0",
    url="http://localhost:11434",
    gpu_model="GTX 1080 Ti",
    models=["qwen2.5:7b"],
))

resp = asyncio.run(hub.infer(InferenceRequest(
    model="qwen2.5:7b",
    prompt="Hello from MoMa Hub!",
)))
print(resp.content)

---

Architecture

Consumer GPUs (GTX 1080 Ti × N)
  │  Ollama serve (one per GPU)
  │
  ▼
MoMa Hub  ──  FastAPI registry + round-robin router
  │
  ▼
Client CLI / Python SDK / SPL scripts

See docs/DESIGN.md for the full roadmap and hardware reference.

---

Roadmap

Version	Milestone
v0.1 (now)	Local MVP: register nodes, round-robin routing, CLI + SDK
v0.2	Persistent registry, capability-aware routing, heartbeat daemon
v0.3	LAN mDNS discovery, SPL integration (USING HUB momahub://...)
v0.4	Internet federation, auth, public hub registry

---

Contributing

MoMa Hub is planned as open-source (Apache 2.0). Once the MVP is proven on 4× GTX 1080 Ti at home, the repo will go public. Follow the GitHub for updates.

https://github.com/digital-duck/momahub

---

License

Apache 2.0