modelpulse 0.1.1

End-to-end partial-weight transfer pipeline.

ModelPulse 🚀

End-to-end partial-weight transfer pipeline.

Device A serves model shards → Device B reconstructs the GGUF in RAM and runs inference with no persistent GGUF reconstruction on disk.

Device A                                      Device B
────────────────────                          ───────────────────────────────────
modelpulse server ./shards                   modelpulse bridge run http://100.101.102.103:8000
  │                                         │
  ├── GET /manifest  ─────────────────────► │  1. fetch manifest
  ├── GET /shards/*  ─────────────────────► │  2. pull all shards (streaming)
  │                                         │  3. assemble GGUF in RAM → /dev/shm
  │                                         │  4. llama.cpp loads from /dev/shm
  │                                         │  5. run inference, stream tokens
  └── POST /metrics  ◄────────────────────  │  6. send collected metrics

---

📸 Screenshots

Server (Device A)

Bridge (Device B)

Inference in Progress

Metrics Sent Back

---

📦 Install

Install ModelPulse as a Python package directly from GitHub:

pip install git+https://github.com/MdSufiyan005/Model-Pulse.git

🔄 Workflow

1 — Prepare shards on Device A

Use gguf_to_shards.py from the companion tools to convert your GGUF model:

python tools/gguf_to_shards.py convert model.gguf ./shards/

2 — Start the server on server

modelpulse server run ./shards --host 0.0.0.0 --port 8000

3.0 - getting talescale ip

curl -fsSL https://tailscale.com/install.sh | sh

sudo tailscale up # signup on the page

tailscale ip # get the ip address - eg: 100.101.102.103

3.1 — Run inference on edge device

modelpulse bridge run http://100.101.102.103:8000

---

📋 Commands

Device A (Server)

modelpulse server run <shards_dir> [options]

Option	Default	Description
--port	8000	Server port
--host	0.0.0.0	Bind address
--metrics-log	metrics.jsonl	Metrics log file

Device B (Client)

modelpulse bridge run <host> [options]
modelpulse bridge status <host> [--all]

Command	Description
run	Full pipeline: pull → infer → report
status	Display latest metrics from Device A

Bridge run options

Flag	Default	Description
--prompt / -p	(interactive)	Prompt string
--max-tokens	256	Tokens to generate
--temp / -t	0.7	Sampling temperature
--ctx	2048	Context window
--no-report	false	Skip sending metrics

---

💾 Zero-Disk Strategy

shard_data  ─── assemble_gguf_bytes() ──► gguf_bytes (RAM)
                                              │
                                       write_bytes()
                                              │
                                        /dev/shm/sb_<pid>.gguf   ← tmpfs, never touches physical disk
                                              │
                                       del gguf_bytes            ← Python bytes freed
                                              │
                                    Llama(model_path=...)        ← mmap from tmpfs
                                              │
                                       cleanup() → unlink()

This keeps the model file temporarly on the ram, while still satisfying llama.cpp’s file-path requirement.

The system prioritizes /dev/shm and /run/shm by checking for existence and write access, falling back to $TMPDIR or /tmp if no RAM-backed filesystem is available.

---

📁 Project Layout

modelpulse/
├── pyproject.toml          # Packaging config
├── README.md               # This doc
├── Images/                 # Screenshots (server.png, bridge.png, etc.)
├── modelpulse/             # Core package
│   ├── __init__.py
│   ├── main.py            # Unified CLI: modelpulse bridge, modelpulse server
│   ├── shared/            # Shared models
│   │   ├── __init__.py
│   │   └── models.py      # ShardManifest, InferenceMetrics
│   ├── server/          # Server side
│   │   ├── __init__.py
│   │   └── server.py      # FastAPI server
│   └── client/          # Client side
│       ├── __init__.py
│       ├── cli.py         # Bridge CLI
│       ├── bridge.py      # RAM GGUF assembly + llama.cpp
│       └── shard_client.py # Async HTTP client
├── tools/                 # Utilities
│   ├── gguf_parser.py
│   └── gguf_to_shards.py  # GGUF → shard converter