private-edge-os 0.8.0

Private Edge OS — Federated edge compute orchestration for enterprise AI inference

Private Edge OS v0.8

Federated edge compute orchestration for enterprise AI inference.
Zero data egress. On-premises. OpenAI-compatible API.

© 2026 Honeypotz Inc. — private-edgeai.com | [e-mail]

---

What It Does

Private Edge OS turns idle enterprise compute (workstations, lab servers, VM clusters) into a private, federated AI inference fabric. Inference stays 100% on-premises — no data leaves your network.

v0.8 Milestone Features

Layer	Feature	Status
L1 Agent	Daemon, hardware fingerprint, idle detection, heartbeat, uninstall	✅
L2 Runtime	ONNX/PyTorch, llama.cpp LLM, INT4/INT8, request batching, streaming	✅
L3 Orchestration	Scheduler, load balancer, failover controller	✅
L5 Security	Zero-egress enforcement, RBAC v1, JWT auth, AES-256 at rest	✅
L7 Interface	OpenAI-compatible REST API, CLI (pedge)	✅

---

Quick Start

1. Install

# Requires Python 3.10+
pip install ".[llama]"          # with LLM support
# or
pip install .                   # ONNX only

For production (root required):

sudo bash scripts/install.sh

2. Start the API server

pedge start --port 8080
# or
python -m private_edge.api.server

3. Load a model

# Download a GGUF model (e.g. from Hugging Face — run offline in production)
pedge models load /path/to/llama-3-8b-instruct.Q4_K_M.gguf

# Or via API
curl -X POST http://localhost:8080/api/v1/models/load \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model_id":"llama3-8b","path":"/path/to/model.gguf","format":"gguf"}'

4. Run inference (OpenAI-compatible)

# CLI
pedge infer "Summarise our Q2 earnings report" --model llama3-8b

# Python — drop-in OpenAI SDK replacement
python - <<'EOF'
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="your-pedge-token",
)

response = client.chat.completions.create(
    model="llama3-8b",
    messages=[{"role": "user", "content": "Hello, Private Edge!"}],
)
print(response.choices[0].message.content)
EOF

5. Multi-node cluster

On each worker node:

PEDGE_ORCHESTRATOR_URL=http://orchestrator:8080 \
PEDGE_ORG_ID=acme \
pedge agent

---

Configuration

All settings are environment variables prefixed PEDGE_. Copy /etc/private-edge/pedge.env and edit.

Variable	Default	Description
PEDGE_API_PORT	8080	API server port
PEDGE_API_SECRET_KEY	change-me…	JWT signing key — change in production
PEDGE_ORCHESTRATOR_URL	http://localhost:8080	Control plane URL (agent → orchestrator)
PEDGE_MODEL_DIR	/var/lib/private-edge/models	Where model files live
PEDGE_ZERO_EGRESS_ENABLED	true	Block non-private outbound connections
PEDGE_ALLOWED_EGRESS_CIDRS	RFC-1918	Allowed CIDRs for egress policy
PEDGE_LLAMA_N_GPU_LAYERS	0	GPU layers for llama.cpp (0 = CPU-only)
PEDGE_RBAC_ENABLED	true	Enable JWT + RBAC
PEDGE_ENCRYPTION_AT_REST	true	AES-256-GCM encryption for stored data

---

API Reference

The API is 100% OpenAI-compatible. Any OpenAI SDK client works by changing base_url.

Method	Path	Description
POST	/v1/chat/completions	Chat inference (streaming ✓)
POST	/v1/embeddings	Text embeddings
GET	/api/v1/models	List available models
POST	/api/v1/models/load	Load a model
POST	/api/v1/models/unload	Unload current model
GET	/api/v1/nodes	List cluster nodes
POST	/api/v1/nodes/register	Register a node (agent)
POST	/api/v1/nodes/{id}/heartbeat	Node heartbeat
POST	/api/v1/auth/token	Get JWT token
GET	/health	Health check

Enable interactive docs (debug mode only): PEDGE_API_LOG_LEVEL=debug pedge start → http://localhost:8080/docs

---

Architecture

┌─────────────────────────────────────────────────────────┐
│  Enterprise Network (zero egress enforced)               │
│                                                          │
│  ┌─────────────┐    mTLS     ┌──────────────────────┐   │
│  │  Orchestrator│◄──────────►│  Node Agent (L1)     │   │
│  │  API Server  │            │  • Fingerprint        │   │
│  │  Scheduler   │            │  • Idle detect        │   │
│  │  Load Balancer│           │  • Heartbeat          │   │
│  └──────┬───────┘            └──────────┬────────────┘   │
│         │                               │                 │
│  ┌──────▼────────────────────────────── ▼────────────┐   │
│  │           Inference Runtime (L2)                   │   │
│  │   llama.cpp (GGUF/INT4/INT8) │ ONNX Runtime        │   │
│  │   Request Batcher            │ GPU Routing          │   │
│  └────────────────────────────────────────────────────┘   │
│                                                          │
│  ┌──────────────────────────────────────────────────┐    │
│  │  Security Layer (L5)                              │    │
│  │  Zero-Egress │ RBAC/JWT │ AES-256 at rest         │    │
│  └──────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────┘

---

Running Tests

pip install ".[dev]"
pytest tests/ -v

---

Roadmap

• v0.9 — mTLS, Prometheus metrics, model registry, GPU routing
• v1.0 — HIPAA BAA, AD/LDAP, distributed tracing, SOC 2 prep
• v1.5 — Federated learning API, differential privacy, multi-tenant isolation

---

Built in Greenwich, CT by the Honeypotz Inc. team.