Introduction

PhysicalAI Library is a Python SDK for training, evaluating, and deploying Vision-Language-Action (VLA) policies. It provides implementations of imitation learning algorithms built on PyTorch Lightning, with a focus on robotic manipulation tasks. The library supports the full ML lifecycle: from training on demonstration data to deploying optimized models for real-time inference.

Key Features

• Simple and modular API and CLI for training, inference, and benchmarking.
• Built on Lightning for reduced boilerplate and distributed training support.
• Export models to OpenVINO, ONNX, or Torch formats for accelerated inference.
• Benchmark policies on standardized environments like LIBERO, PushT, and RoboCasa.
• Unified inference API across all export backends.

Supported Policies

Installation

pip install physicalai-train

# Ubuntu
sudo apt-get install -y ffmpeg

# macOS
brew install ffmpeg

git clone https://github.com/open-edge-platform/physical-ai-studio.git
cd physical-ai-studio/library

# Create virtual environment and install
uv venv
source .venv/bin/activate

# Choose one matching your hardware:
uv sync --extra cpu --extra all     # CPU
# uv sync --extra cu128 --extra all # NVIDIA GPU (CUDA)
# uv sync --extra xpu --extra all   # Intel GPU (XPU)

Training

PhysicalAI supports both API and CLI-based training. Checkpoints are saved to experiments/lightning_logs/ by default.

API

from physicalai.data import LeRobotDataModule
from physicalai.policies import ACT
from physicalai.train import Trainer

# Initialize components
datamodule = LeRobotDataModule(repo_id="lerobot/aloha_sim_transfer_cube_human")
model = ACT()
trainer = Trainer(max_epochs=100)

# Train
trainer.fit(model=model, datamodule=datamodule)

CLI

# Train with config file
physicalai fit --config configs/physicalai/act/pusht/default.yaml

# Train with CLI arguments
physicalai fit \
    --model physicalai.policies.ACT \
    --data physicalai.data.LeRobotDataModule \
    --data.repo_id lerobot/aloha_sim_transfer_cube_human

# Override config values
physicalai fit \
    --config configs/physicalai/act/pusht/default.yaml \
    --trainer.max_epochs 200 \
    --data.train_batch_size 64

Benchmark

Evaluate trained policies on standardized simulation environments.

API

from physicalai.benchmark.gyms import LiberoBenchmark
from physicalai.policies import ACT

# Load trained policy (path from training output)
policy = ACT.load_from_checkpoint("experiments/lightning_logs/version_0/checkpoints/last.ckpt")
policy.eval()

# Run benchmark
benchmark = LiberoBenchmark(task_suite="libero_10", num_episodes=20)
results = benchmark.evaluate(policy)

# View results
print(results.summary())
results.to_json("results.json")

RoboCasa requires a dedicated virtual environment. Install it with bash library/scripts/benchmark/install_robocasa.sh, then swap in RoboCasaBenchmark:

from physicalai.benchmark.gyms import RoboCasaBenchmark

benchmark = RoboCasaBenchmark(task="atomic_seen", num_episodes=20)
results = benchmark.evaluate(policy)

CLI

# Basic benchmark
physicalai benchmark \
    --benchmark physicalai.benchmark.gyms.LiberoBenchmark \
    --benchmark.task_suite libero_10 \
    --policy physicalai.policies.ACT \
    --ckpt_path ./checkpoints/model.ckpt

# With video recording
physicalai benchmark \
    --benchmark physicalai.benchmark.gyms.LiberoBenchmark \
    --benchmark.task_suite libero_10 \
    --benchmark.video_dir ./videos \
    --benchmark.record_mode failures \
    --policy physicalai.policies.ACT \
    --ckpt_path ./checkpoints/model.ckpt

Export

Export trained policies to optimized formats for deployment.

API

from physicalai.policies import ACT

# Load and export
policy = ACT.load_from_checkpoint("checkpoints/model.ckpt")
policy.export("./exports", backend="openvino")

CLI

physicalai export \
    --policy physicalai.policies.ACT \
    --ckpt_path checkpoints/model.ckpt \
    --backend openvino \
    --output_dir ./exports

Supported Backends

Inference

Deploy exported models with a unified inference API.

API

from physicalai.inference import InferenceModel

# Load exported model (auto-detects backend)
policy = InferenceModel("./exports")

# Run inference loop
obs, info = env.reset()
policy.reset()
done = False

while not done:
    action = policy.select_action(obs)
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

The inference API is consistent across all export backends, making it easy to switch between OpenVINO, ONNX, and Torch depending on your deployment target.

Documentation

• Getting Started - Installation, quickstart, first benchmark, first deployment
• How-To Guides - Goal-oriented guides for specific tasks
• Explanation - Architecture and design documentation

See Also

• Main Repository - Project overview
• Application - GUI for data collection and training

Development Type Checking

# From library/
uv run pyrefly check -c pyproject.toml