VLAStudio: A Modular Learning Playground for Robotics

demos generated by policies trained in VLAStudio

ACT - RLBench	Diffusion Policy - ALOHA-Sim	ACT - Square	FlowMatching - Transport	SmolVLA - MetaWorld
Koch - Pick & Place -Inference	SO101 - Fold Tower -Inference	BC_MLP - AdroitHandDoor	Pi0 - LIBERO	ACT - RoboTwin

VLAStudio is a Python library for robot learning. Write your own run.py to load demonstrations, train imitation-learning and vision-language-action policies, evaluate in simulation, and deploy on real robots. Compose these steps with ordinary Python functions and run your file with python run.py.

• Train and fine-tune ACT, Diffusion Policy, SmolVLA, Pi0/Pi0.5, and other policies with a shared Python API.
• Combine data sources through HDF5, LeRobot, and RLDS loaders and shared data processing.
• Evaluate and deploy locally or through a policy server, with simulation benchmarks and real robot integrations.
• Extend the library with your own policy, dataset, simulator, or robot without rewriting the pipeline.
• Choose your own Python environment. Use vlastudio env to install and select model or simulator runtimes.

🛠️ Installation

Install the library, then install ACT / DP / ALOHA-Sim into your current environment:

python -m pip install vlastudio
vlastudio env install
python run.py

For development, use pip install -e . before vlastudio env install. Alternatively, pip install "vlastudio[base]" installs the base Python dependencies.

Create a separate model or simulator environment:

vlastudio env create -n pi05 --policy openpi
vlastudio env run pi05 -- python train.py
vlastudio env create -n sim --env libero
vlastudio env run sim -- python evaluate.py
vlastudio env list

Selectors accept built-in names or config paths. Use -i URL for a custom index; otherwise existing uv settings and pip index environment variables apply. Cache location defaults to ~/.cache/vlastudio; override it with VLASTUDIO_CACHE. See the environment guide for RLBench, BEHAVIOR, RoboTwin and validation.

import vlastudio stays lightweight and never installs dependencies. Ordinary Python scripts use their own interpreter; env run explicitly selects one.

🚀 Write Your Own run.py

ACT on ALOHA-Sim

Prepare the demonstrations at data/sim_transfer_cube_scripted, or use a task YAML with your own dataset path. The built-in starting point is sim_transfer_cube_scripted.yaml.

Save this as run.py in your project:

import vlastudio as vla


def main():
    dataset = vla.load_dataset("sim_transfer_cube_scripted")
    policy = vla.load_policy("act")
    result = vla.train(
        policy, dataset, "default", output_dir="checkpoints/act_aloha"
    )
    benchmark = vla.load_env("aloha_transfer")
    evaluation = benchmark.evaluate(result.policy, output_dir="results/act_aloha")
    print(evaluation.metrics)


if __name__ == "__main__":
    main()

python run.py

The script uses your current Python environment. Paths resolve relative to your working directory; pass your own YAML paths or built-in names to the loaders. You can freely add preprocessing, logging, evaluation logic, or other library calls to this file.

The ACT example provides the same workflow:

python examples/01_train_and_eval_act_on_aloha/train_and_evaluate.py

Diffusion Policy

The default installation above already includes Diffusion Policy and ALOHA-Sim. Write run_dp.py:

import vlastudio as vla


def main():
    dataset = vla.load_dataset("sim_transfer_cube_scripted")
    policy = vla.load_policy("diffusion_policy")
    vla.train(
        policy, dataset, "dp", output_dir="checkpoints/dp_aloha",
        overrides={"training.max_steps": 200000, "training.save_steps": 10000},
    )
    vla.load_env("aloha_transfer").evaluate(policy, output_dir="results/dp_aloha")


if __name__ == "__main__":
    main()

python run_dp.py

SmolVLA on MetaWorld

In a separate Python 3.10 environment, run python -m pip install -e . -r requirements/smolvla.txt. The built-in task uses the LeRobot MetaWorld dataset. Provide access to that dataset and the pretrained policy weights.

Write train_smolvla.py:

import vlastudio as vla


def main():
    dataset = vla.load_dataset("metaworld")
    policy = vla.load_policy("smolvla")
    vla.train(
        policy, dataset, "smolvla", output_dir="checkpoints/smolvla_metaworld",
        overrides={"training.max_steps": 100000,
                   "training.per_device_train_batch_size": 64},
    )


if __name__ == "__main__":
    main()

python train_smolvla.py

Use the policy-server workflow below to evaluate with a simulator in another environment.

Runnable examples

Each example shows its own environment setup and Python files. Run from the repository root after installing the required dependencies.

Overview

🌐 Policy Server

Write serve.py in the model environment:

import vlastudio as vla


if __name__ == "__main__":
    policy = vla.load_policy("smolvla", checkpoint="checkpoints/smolvla_metaworld")
    vla.serve(policy, address="127.0.0.1:5000")

In a separate simulation environment, run python -m pip install -e . -r requirements/metaworld.txt, then write evaluate.py:

import vlastudio as vla


if __name__ == "__main__":
    policy = vla.connect_policy("127.0.0.1:5000")
    benchmark = vla.load_env("metaworld.easy")
    result = benchmark.evaluate(
        policy, output_dir="results/smolvla_metaworld", num_rollout=10, batch_size=10
    )
    print(result.metrics)

Start python serve.py in the model environment, wait for the server to listen, then start python evaluate.py in the simulation environment. The server handles inference while the client handles observations and actions. See Example 02 for runnable files.

🤖 Real Robots and Data Collection

For SO101, create a dedicated Python 3.10 environment and install the model and motor SDK together:

python -m pip install -e . -r requirements/smolvla.txt "lerobot[feetech]==0.3.3"

Customize serial ports, camera devices, IDs, and calibration paths in your robot and teleoperator YAMLs. Starting points: so101.yaml, so101_leader.yaml.

Write collect.py:

import vlastudio as vla


if __name__ == "__main__":
    vla.collect(
        "so101", teleop="so101_leader", output_dir="data/so101_pick_place",
        dataset_format="hdf5", task="pick up the cube",
    )

Write infer.py to use an already running policy server:

import vlastudio as vla


if __name__ == "__main__":
    policy = vla.connect_policy("127.0.0.1:5000")
    vla.infer("so101", policy)

Run python collect.py to collect demonstrations or python infer.py to deploy. Pass custom YAML paths instead of built-in names for your own robot. See the SO101 guide.

🎨 Policy Gallery

🏆 Simulation Benchmark Gallery

🦾 Robot Gallery

• SO101
• BimanualSO101
• Kochv1.1
• AgilexAloha (CobotMagic)

📊 Datasets and Extensions

VLAStudio provides HDF5, LeRobot, and RLDS loaders. Data processing and batch collation adapt each source to the selected policy. Your run.py can select built-in names or your own task and policy YAML files:

import vlastudio as vla

dataset = vla.load_dataset("path/to/task.yaml")
policy = vla.load_policy("path/to/policy.yaml")

Built-in configurations live in src/vlastudio/configs. User configurations can live anywhere.

Start from the examples above to write your own components. A policy supplies load_model and a model with select_action; optional data processors, collators, and trainers customize training. The developer guide describes package and runtime internals.

🩹 Troubleshooting

• Missing or conflicting dependencies: inspect vla.environment_plan(...) and use its pip command in the intended environment, or install the corresponding requirements file.
• Headless MuJoCo on Linux: set os.environ["MUJOCO_GL"] = "egl" near the start of your Python file. CPU rendering needs system OSMesa libraries and the osmesa backend; a desktop display can use glfw.
• Old CMake projects fail with “Compatibility with CMake < 3.5 has been removed”: set CMAKE_POLICY_VERSION_MINIMUM=3.5 in the terminal before running pip for that optional simulator.
• Model or simulator resources are missing: Python dependencies, pretrained weights, datasets, and native simulator assets are separate setup steps. Follow the relevant example's README.

📫 Contact

• Wang Z. zwang@stu.xmu.edu.cn
• Yuxin G. yuxinxmu@163.com

🙏 Acknowledgements

This repo is built on the open source codebases below. Thanks to the authors' wonderful contributions.

• Lerobot

• ACT

• Octo-pytorch

• Openpi

• OpenVLA

• DiVLA

• RoboTwin

• RLBench

• StarVLA