dlab-ai-forward 0.1.0

AI Forward - Expose local LLM services (Ollama, LM Studio) to the cloud

DLab AI Forward CLI

Command-line interface for AI Forward - expose your local LLM services (Ollama, LM Studio) to the cloud via secure WebSocket tunnel.

Features

• 🔌 Easy Tunneling - Connect local LLM services to cloud infrastructure
• 🤖 Multi-Service Support - Works with Ollama and LM Studio
• 📊 System Monitoring - Reports CPU, RAM, and GPU metrics
• 🔄 Auto-Discovery - Automatically detects available models
• 🔒 Secure - Token-based authentication
• 🖥️ Cross-Platform - Works on macOS, Linux, and Windows

Installation

pip install dlab-ai-forward

For NVIDIA GPU monitoring support:

pip install dlab-ai-forward[nvidia]

Or using pipx (recommended for CLI tools):

pipx install dlab-ai-forward
pipx install dlab-ai-forward[nvidia]  # With NVIDIA support

Usage

# Basic usage
dlab-ai-forward --name demo --server https://ai.example.com --token YOUR_TOKEN

# With custom ports
dlab-ai-forward --name my-node --server https://ai.example.com --token YOUR_TOKEN \
    --ollama-port 11434 --lmstudio-port 1234

# Expose only specific services
dlab-ai-forward --name ollama-only --server https://ai.example.com --token YOUR_TOKEN \
    --services ollama

# Verbose mode (debug logging)
dlab-ai-forward --name demo --server https://ai.example.com --token YOUR_TOKEN --verbose

Options

Option	Description	Default
--name	Tunnel name (unique identifier)	Required
--server	Backend server URL	Required
--token	Authentication token (32 chars)	Required
--ollama-port	Ollama local port	11434
--lmstudio-port	LM Studio local port	1234
--services	Services to expose	ollama lmstudio
--verbose, -v	Enable debug logging	False

System Requirements

• Python 3.8+
• Ollama (optional) - running on localhost:11434
• LM Studio (optional) - running on localhost:1234

GPU Monitoring

GPU metrics are automatically detected and reported:

• NVIDIA GPUs: Install with pip install dlab-ai-forward[nvidia]
• AMD GPUs (Windows): Install with pip install dlab-ai-forward[amd]
• Apple Silicon (M1/M2/M3): Automatically supported on macOS
• Other GPUs: Not currently supported

AMD GPU on Windows

For AMD GPU monitoring on Windows, you have two options:

1. Using WMI (built-in, no extra install):

   • Works with most AMD GPUs
   • Uses Windows Performance Counters
   • May require running as Administrator for full metrics

2. Using pyadl (optional):

   pip install dlab-ai-forward[amd]
   

   • Uses AMD Display Library (ADL)
   • More accurate metrics
   • Requires AMD drivers installed

How It Works

1. CLI connects to backend via WebSocket with authentication token
2. CLI detects local LLM services and available models
3. CLI reports system metrics (CPU, RAM, GPU)
4. Cloud services can proxy requests through the backend to your local models

Example: Call Local Model from Cloud

# Cloud service makes request to backend
curl https://ai.example.com/api/demo/ollama/api/tags \
    -H "Authorization: Bearer YOUR_TOKEN"

# Or for LM Studio
curl https://ai.example.com/api/demo/lmstudio/v1/models \
    -H "Authorization: Bearer YOUR_TOKEN"

License

MIT License