Python client for benchmarking and validating ML models on real devices via RunLocal API.

🎯 Key Features

• ⚡ Real Hardware Testing - No simulators or emulators. Test on real devices maintained in our devices lab
• 🌍 Cross-Platform Coverage - Access MacBooks, iPhones, iPads, Android, and Windows devices from a single API
• 🔧 Multiple ML Formats - Support for CoreML, ONNX, OpenVINO, TensorFlow Lite, and GGUF models. More frameworks coming soon.
• 📊 Detailed Metrics - Measure inference time, memory usage, and per-layer performance data
• 🚦 CI/CD Ready - Integrate performance and accuracy testing into your deployment pipeline

🔍 Evaluate Results

All benchmarks performed through the python client can be evaluated on the web platform by logging into your account. Check out our public demo for comprehensive benchmark evaluation across different devices and model formats.

🛠 Installation

Using uv

Add to your project with uv:

uv add runlocal-hub

Then run:

uv sync

Using pip

pip install runlocal-hub

Development Installation

For development or to install from source using uv:

git clone https://github.com/neuralize-ai/runlocal_hub.git
cd runlocal_hub
uv sync

🔑 Authentication

Get your API key from the RunLocal dashboard:

1. Log in to RunLocal
2. Click your avatar → User Settings
3. Navigate to "API Keys"
4. Click "Create New API Key"
5. Save your key securely

export RUNLOCAL_API_KEY=<your_api_key>

🕹 Usage Guide

Simple Benchmark

from runlocal_hub import RunLocalClient, display_benchmark_results

client = RunLocalClient()

# Benchmark on any available device
response = client.benchmark("model.mlpackage")
display_benchmark_results(response.results)

Device Filtering

Target specific devices with intuitive filters:

from runlocal_hub import DeviceFilters, RunLocalClient

client = RunLocalClient()

# High-end MacBooks with M-series chips
mac_filters = DeviceFilters(
    device_name="MacBook",
    soc="Apple M",        # Matches M1, M2, M3, etc.
    ram_min=16,           # At least 16GB RAM
    year_min=2021         # Recent models only
)

# Latest iPhones with Neural Engine
iphone_filters = DeviceFilters(
    device_name="iPhone",
    year_min=2022,
    compute_units=["CPU_AND_NE"]
)

# Run benchmarks
response = client.benchmark(
    "model.mlpackage",
    device_filters=[mac_filters, iphone_filters],
    count=None  # Use all matching devices
)

🧮 Running Predictions

Test your model with real inputs:

import numpy as np

# Prepare input
image = np.random.rand(1, 3, 224, 224).astype(np.float32)
inputs = {"image": image}

# Run prediction on iPhone
response = client.predict(
    inputs=inputs,
    model_path="model.mlpackage",
    device_filters=DeviceFilters(device_name="iPhone 15", compute_units=["CPU_AND_NE"])
)

outputs = response.results.outputs

tensors = outputs["CPU_AND_NE"]
for name, path in tensors.items():
    tensor = np.load(path)
    print(f"  {name}: {tensor.shape} ({tensor.dtype})")
    print(f"  First values: {tensor.flatten()[:5]}")

📚 Examples

Check out the example scripts:

• bench.py - Simple benchmarking example
• predict.py - Prediction with custom inputs, serialised outputs

To run the examples:

# Make sure you have set your API key
export RUNLOCAL_API_KEY=<your_api_key>

# Run benchmark example
uv run examples/bench.py

# Run prediction example
uv run examples/predict.py

💠 Supported Formats

More frameworks coming soon.

📜 License

This project is licensed under the MIT License - see the LICENSE.txt file for details.