Skip to main content

Hugging Face 🤗 x Stable-baselines3 v3.2

Note: this is a fork of https://github.com/huggingface/huggingface_sb3 with fixes for newer SB3 versions

A library to load and upload Stable-baselines3 models from the Hub with Gymnasium and Gymnasium compatible environments.

Installation

With pip

pip install sb3-team-hf-integration

Examples

We wrote a tutorial on how to use 🤗 Hub and Stable-Baselines3 here

If you use Colab or a Virtual/Screenless Machine, you can check Case 3 and Case 4.

Case 1: I want to download a model from the Hub

You will need to set the TRUST_REMOTE_CODE environment variable to True to allow the use of pickle.load():

import os
import gymnasium as gym

from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy

# Allow the use of `pickle.load()` when downloading model from the hub
# Please make sure that the organization from which you download can be trusted
os.environ["TRUST_REMOTE_CODE"] = "True"

# Retrieve the model from the hub
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename = name of the model zip file from the repository
checkpoint = load_from_hub(
    repo_id="sb3/demo-hf-CartPole-v1",
    filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)

# Evaluate the agent and watch it
eval_env = gym.make("CartPole-v1")
mean_reward, std_reward = evaluate_policy(
    model, eval_env, render=False, n_eval_episodes=5, deterministic=True, warn=False
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")

Case 2: I trained an agent and want to upload it to the Hub

With package_to_hub() we'll save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub. It currently works for Gym and Atari environments. If you use another environment, you should use push_to_hub() instead.

First you need to be logged in to Hugging Face:

  • If you're using Colab/Jupyter Notebooks:
from huggingface_hub import login

login()
  • Else:
huggingface-cli login

For more details about authentication, check out this guide.

Then

With package_to_hub():

import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import package_to_hub

# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)

# Create the evaluation env
eval_env = make_vec_env(env_id, n_envs=1)

# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)

# Train the agent
model.learn(total_timesteps=int(5000))

# This method save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub
package_to_hub(model=model, 
               model_name="ppo-LunarLander-v3",
               model_architecture="PPO",
               env_id=env_id,
               eval_env=eval_env,
               repo_id="ThomasSimonini/ppo-LunarLander-v3",
               commit_message="Test commit")

With push_to_hub(): Push to hub only push a file to the Hub, if you want to save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub, use package_to_hub()

import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import push_to_hub

# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)

# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)

# Train it for 10000 timesteps
model.learn(total_timesteps=10_000)

# Save the model
model.save("ppo-LunarLander-v3")

# Push this saved model .zip file to the hf repo
# If this repo does not exists it will be created
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename: the name of the file == "name" inside model.save("ppo-LunarLander-v3")
push_to_hub(
    repo_id="ThomasSimonini/ppo-LunarLander-v3",
    filename="ppo-LunarLander-v3.zip",
    commit_message="Added LunarLander-v3 model trained with PPO",
)

Case 3: I use Google Colab with Classic Control/Box2D Gym Environments

  • You can use xvbf (virtual screen)
!apt-get install -y xvfb python-opengl > /dev/null 2>&1
  • Just put your code inside a python file and run
!xvfb-run -s "-screen 0 1400x900x24" <your_python_file>

Case 4: I use a Virtual/Remote Machine

  • You can use xvbf (virtual screen)
xvfb-run -s "-screen 0 1400x900x24" <your_python_file>

Case 5: I want to automate upload/download from the Hub

If you want to upload or download models for many environments, you might want to automate this process. It makes sense to adhere to a fixed naming scheme for models and repositories. You will run into trouble when your environment names contain slashes. Therefore, we provide some helper classes:

import gymnasium as gym
from huggingface_sb3.naming_schemes import EnvironmentName, ModelName, ModelRepoId

env_name = EnvironmentName("seals/Walker2d-v0")
model_name = ModelName("ppo", env_name)
repo_id = ModelRepoId("YourOrganization", model_name)

# prints 'seals-Walker2d-v0'. Notice how the slash is removed so you can use it to 
# construct file paths if you like.
print(env_name)

# you can still access the original gym id if needed
env = gym.make(env_name.gym_id)  

# prints `ppo-seals-Walker2d-v0`
print(model_name)  

# prints: `ppo-seals-Walker2d-v0.zip`. 
# This is where `model.save(model_name)` will place the model file
print(model_name.filename)  

# prints: `YourOrganization/ppo-seals-Walker2d-v0`
print(repo_id)

Metadata

Release files for sb3-team-hf-integration 3.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sb3-team-hf-integration 3.2
File Size Uploaded Provenance Actions
sb3_team_hf_integration-3.2.tar.gz 12.8 kB With provenance Details for sb3_team_hf_integration-3.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sb3-team-hf-integration 3.2
File Interpreter ABI Platform Actions
sb3_team_hf_integration-3.2-py3-none-any.whl Python 3 none any Details for sb3_team_hf_integration-3.2-py3-none-any.whl

Total release size: 23.3 kB

Release files / sb3_team_hf_integration-3.2.tar.gz

Download URL sb3_team_hf_integration-3.2.tar.gz
Size 12.8 kB
Tags Source
SHA-256 checksum
How to use checksums
c0e1c1d04ce36363bf92b5801d7e687fa0dd7be418d59bb6b6e5c31aecd8ecbd
BLAKE2b-256 checksum
How to use checksums
5092c3a95018298292457b64b66fdb8eb33b7995f83a4b9865a4195c450f383f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Release files / sb3_team_hf_integration-3.2-py3-none-any.whl

Download URL sb3_team_hf_integration-3.2-py3-none-any.whl
Size 10.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1a2ed610190d78d223a62b61824e74d05fb24926f3e369956311d4d9f1f33494
BLAKE2b-256 checksum
How to use checksums
07a76506a5643764ac9e800691a1bc2b231b36a5d6d0298dc6debc480d2c110a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

Security for sb3-team-hf-integration 3.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sb3_team_hf_integration-3.2.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

3.2 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page