upkie 1.4.0

Python module to control Upkie wheeled bipeds.

Upkie wheeled biped robot

Build instructions and software for Upkie wheeled bipeds. Develop on Linux 🐧 or macOS 🍏, deploy to the robot's Raspberry Pi 🍓. Questions are welcome in the Discussions forum or on the Chat.

Quick sim

If you have Python and a C++ compiler (setup one-liners: Fedora, Ubuntu), you can run an Upkie simulation right from the command line. It won't install anything on your machine, as everything will run locally from the repository:

git clone https://github.com/tasts-robots/upkie.git
cd upkie
./start_wheel_balancer.sh

Click on the robot in the simulator window to apply external forces 😉

Running a spine

Upkie's code is organized into spines, which communicate with the simulation or mjbots actuators, and agents, the programs that implement robot behaviors. We use Bazel to build spines, both for simulation on your development computer or for running on the robot's Raspberry Pi. Bazel builds everything locally, does not install anything on your system, and makes sure that all versions of all dependencies are correct. Check out this introduction for more details.

Simulation spine

In the example above we ran an agent called "wheel balancer". We could also start the simulation spine independently, and let it run waiting for agents to connect:

./start_simulation.sh

This script is just an alias for a Bazel run command:

./tools/bazelisk run -c opt //spines:bullet -- --show

The -c opt flag selects the optimized compilation mode. It is actually the default in this project, we just show it here for example.

Robot spine

To run a spine on the robot, we first build it locally and upload it to the onboard Raspberry Pi:

make build
make upload ROBOT=your_upkie

Next, log into the Pi and run a pi3hat spine:

$ ssh foo@robot
foo@robot:~$ cd upkie
foo@robot:upkie$ make run_pi3hat_spine

Once the spine is running, you can run any agent in a separate shell on the robot, for example the wheel balancer:

foo@robot:upkie$ make run_wheel_balancer

Running a Python agent

We develop Python agents using the upkie interface distributed on PyPI. This interface is already fast enough for real-time control. To install it:

pip install upkie

The repository ships a number of agents that have been tested on several Upkie's. You will find them in the agents/ directory. To run an agent, call its main Python script:

python agents/<agent_name>/main.py <args>

For instance, to run the PPO balancer on the foobar policy:

python agents/ppo_balancer/main.py foobar

Example of a custom agent

You can develop your own agent using the environments distributed in upkie.envs. For example, run a spine and try executing the following code in a Python interpreter:

import gymnasium as gym
import upkie.envs

upkie.envs.register()

with gym.make("UpkieWheelsEnv-v4", frequency=200.0) as env:
    observation = env.reset()
    action = 0.0 * env.action_space.sample()
    for step in range(1_000_000):
        observation, reward, done, _ = env.step(action)
        if done:
            observation = env.reset()
        pitch = observation[0]
        action[0] = 10.0 * pitch

With a simulation spine, this code will reset the robot's state and execute the policy continuously. In a pi3hat spine, this code will control the robot directly. You can check out the examples/ directory for more examples.

Code overview

Agents

MPC balancer

An agent that balances the robot in place, using wheels only, by closed-loop model predictive control. It performs better than the Wheel balancer with significantly less hacks ;-)

Pink balancer

A more capable agent that combines wheeled balancing with inverse kinematics computed by Pink. This is the controller that runs in the first two videos of Upkie.

PPO balancer

An agent trained by reinforcement learning to balance with straight legs. Training uses the UpkieWheelsEnv gym environment and the PPO implementation from Stable Baselines3.

Wheel balancer

A baseline agent designed to check out Upkie's physical capabilities. The robot balances with its wheels only, following PD feedback from the head pitch and wheel odometry to wheel velocities, plus a feedforward non-minimum phase trick for smoother transitions from standing to rolling.

Environments

UpkieServosEnv

Upkie with full observation and joint position-velocity-torque actions.

UpkieWheelsEnv

Upkie with full observation but only wheel velocity actions.

Environments are single-threaded rather than vectorized. In return, they run as-is on the real robot.

Observers

Floor contact

Detect contact between the wheels and the floor. The pink and wheel balancers use contact as a reset flag for their integrators, to avoid over-spinning the wheels while the robot is in the air.

Wheel contact

Detect contact between a given wheel and the floor.

Wheel odometry

Measure the relative motion of the floating base with respect to the floor. Wheel odometry is part of their secondary task (after keeping the head straight), which is to stay around the same spot on the floor.

Spines

Bullet

Spawn Upkie in a Bullet simulation. Resetting this spine moves the robot back to its initial configuration in this world.

pi3hat

Spine is made to be called from a Raspberry Pi with an onboard mjbots pi3hat. Servos are stopped when the spine is stopped, and switch to position mode (which is a position-velocity-torque controller) when the spine idles. Check out the spine state machine for details.