safari-sdk 2.103.0

Safari SDK: the SDK for Google DeepMind Gemini Robotics models

Safari SDK: the SDK for Google DeepMind Gemini Robotics models 🦓🦄🐘🐒🐍

Disclaimer

This is not an officially supported Google product.

Safari SDK provides full lifecycle toolings necessary for using Gemini Robotics models, including but not limited to, access checkpoint, serving a model, evaluate the model on robot and in sim, upload data, finetuning the model, download the finetuned checkpoint, etc. Most of the functionality requires you to join Gemini Robotics Trusted Tester Program to use. See details in Gemini Robotics main page.

Source Code

The source code can be found in GitHub.

Note: Unless stated otherwise, the commands below assume that your working directory is the root of your repository clone (ie, the same directory which contains the pyproject.toml file).

Installing the SDK

Installing Pip Dependencies

The pyproject.toml file specifies the dependencies for the Safari SDK, including allowed version ranges for some packages. The requirements.txt file specifies exact versions and hashes of each of those dependencies; internal unit tests use these versions, and installing them in a virtual environment before installing the SDK is recommended to avoid failures due to upstream package changes:

pip install -r requirements.txt

Updating Pip Dependencies

A script is provided which updates the requirements.txt file to the most recent versions of dependencies which satisfy the constraints in the pyproject.toml file. This allows those updates to be done at discrete times (rather than whenever those updates are pushed to PyPi) and tested in isolation before being pushed to all users. To run that script:

scripts/update_pip_dependencies.sh

Installing from PyPi

The Safari SDK can be easily installed via PyPI.

pip install safari_sdk

Installing from Source Code

The Safari SDK can also be installed from the source code.

pip install -e .[dev]

Building the Wheel

To build a Python wheel, run the following command from the root of the repository.

scripts/build_wheel.sh

This script will build a pip installable wheel for the Safari SDK, and print the file's path to stdout.

Model support

Safari SDK aims to support all models in the Gemini Robotics model series.

Trusted Testers can access the Gemini Robotics On Device model from SDK v2.4.1.

Libraries

Libraries related to robot data logging is in safari/logging.

Libraries related to model inference and interface with model servers are in safari/model.

Libraries and binary related to accessing model checkpoints, upload data and request of model finetune can be found in safari/flywheel.

Examples, including robot and simulation evaluation of models are in examples/. Aloha specific eval code are in examples/aloha.

Flywheel CLI

The flywheel CLI is a convenient CLI tool available after installation of the pip package. It provides a set of commands to interact with the Gemini Robotics platform, such as training models, serving models, managing data, and downloading artifacts.

To use the CLI

flywheel-cli <command> [--flags] [--flags]

Supported commands are:

• train: Train a model. Requires specifying task ID, start date, and end date.
• serve: Serve a model. Requires specifying the training job ID.
• list: List available training jobs.
• list_serve: List available serving jobs.
• data_stats: Show data statistics available for training.
• download: Download artifacts from a training job or a specific artifact ID.
• upload_data: Upload data to the data ingestion service.
• version: Show the version of the SDK.
• help: Show this help message with all the available commands and flags.

Agent

The Safari SDK includes a comprehensive agent framework for building interactive robotics agents powered by Gemini models. See YouTube Video: Gemini Robotics 1.5: Using agentic capabilities. The framework is located in safari/agent/framework and provides a modular architecture for creating agents that can perceive their environment, reason about tasks, and control robot hardware.

Key Components

Agents (safari/agent/framework/agents/): Base agent classes that integrate with the Gemini Live API to provide conversational interaction and tool use capabilities.

Embodiments (safari/agent/framework/embodiments/): Hardware-specific interfaces that connect agents to physical robot systems (e.g., Aloha robot). Each embodiment provides tools for robot control.

Tools (safari/agent/framework/tools/): Modular capabilities that agents can use, including:

• Run instruction
• Success detection
• Scene description
• etc.

Event Bus (safari/agent/framework/event_bus/): Asynchronous publish-subscribe system for communication between agent components.

Configuration (safari/agent/framework/config.py): Centralized configuration management using AgentFrameworkConfig, supporting both programmatic configuration and flag-based setup.

Aloha Agent Example

The examples/aloha/agent/ directory contains agent implementations for the Aloha robot platform.

The primary example is simple_agent.py, which provides a conversational agent that can control the Aloha robot using natural language instructions.

To run the Aloha agent, use the provided run.py script:

python examples/aloha/agent/run.py --agent_name=simple_agent

The Aloha agent demonstrates integration of vision-based robot control, multi-camera perception, and conversational interaction with Gemini models.

Alternatively, you build your own agent using the agent framework.

The codebase is still in active development. We will update our most updated user guide with Trusted Testers of Gemini Robotics.