InferiaLLM acts as the authoritative execution layer between your applications and your AI infrastructure. It governs how LLMs are accessed, secured, routed, and run on compute.

Installation

pip install inferiallm

Quick Start

InferiaLLM requires a .env file in your current working directory to configure connections to PostgreSQL and Redis.

# 1. Initialize your environment
# Create a .env file with your DATABASE_URL and Redis settings
cp .env.sample .env

# 2. Bootstrap the platform
# This creates the database, roles, and applying schemas
inferiallm init

# 3. Launch all services
# Starts API, Orchestration, Inference, and the Dashboard in one go
inferiallm start all

Configuration

The CLI manages configuration through environment variables. The most critical settings are:

1. Database & Security

CLI Reference

inferiallm init

Bootstraps the unified database environment. Output:

[inferia:init] Connecting as admin to bootstrap inferia
[inferia:init] Creating role: inferia
[inferia:init] Creating database: inferia
[inferia:init] Applying schema: global_schema
[inferia:init] Bootstrap complete

For existing databases, apply incremental schema updates manually:

psql "$DATABASE_URL" -f db/migrations/20260212_add_inference_logs_ip.sql

inferiallm start

Starts all InferiaLLM gateways (API, Orchestration, Inference) and the Dashboard.

You can also start specific services:

• inferiallm start all: Starts all core services (default).
• inferiallm start api-gateway: Starts only the API Gateway.
• inferiallm start orchestration: Starts the Orchestration stack (API, Sidecar, Worker).
• inferiallm start inference: Starts the Inference engine.

Package Structure

The inferia package is a monorepo-style library that contains all backend services:

package/src/inferia/
├── cli.py                  # Entry point for the CLI
├── services/
│   ├── api_gateway/        # Security & Policy Service (Port 8000)
│   ├── inference/          # Inference Proxy Service (Port 8001)
│   ├── orchestration/      # Compute Control Plane (Port 8080)
│   ├── data/               # Data Engine (Port 8003)
│   └── guardrail/          # Guardrail Engine (Port 8002)

Core Capabilities

• Unified Control Plane: Orchestrate LLMs across heterogeneous compute (K8s, DePIN, VPS).
• Policy Enforcement: Centralized RBAC, safety guardrails, and budget controls.
• Execution Boundary: Authority-based routing between applications and infrastructure.

For full documentation, architecture diagrams, and deployment guides, visit the main repository.