Hugging Face 🤗 x Stable-baselines3 v3.2
Note: this is a fork of https://github.com/huggingface/huggingface_sb3 with fixes for newer SB3 versions
A library to load and upload Stable-baselines3 models from the Hub with Gymnasium and Gymnasium compatible environments.
Installation
With pip
pip install sb3-team-hf-integration
Examples
We wrote a tutorial on how to use 🤗 Hub and Stable-Baselines3 here
If you use Colab or a Virtual/Screenless Machine, you can check Case 3 and Case 4.
Case 1: I want to download a model from the Hub
You will need to set the TRUST_REMOTE_CODE environment variable to True to allow the use of pickle.load():
import os
import gymnasium as gym
from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
# Allow the use of `pickle.load()` when downloading model from the hub
# Please make sure that the organization from which you download can be trusted
os.environ["TRUST_REMOTE_CODE"] = "True"
# Retrieve the model from the hub
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename = name of the model zip file from the repository
checkpoint = load_from_hub(
repo_id="sb3/demo-hf-CartPole-v1",
filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)
# Evaluate the agent and watch it
eval_env = gym.make("CartPole-v1")
mean_reward, std_reward = evaluate_policy(
model, eval_env, render=False, n_eval_episodes=5, deterministic=True, warn=False
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
Case 2: I trained an agent and want to upload it to the Hub
With package_to_hub() we'll save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub.
It currently works for Gym and Atari environments. If you use another environment, you should use push_to_hub() instead.
First you need to be logged in to Hugging Face:
- If you're using Colab/Jupyter Notebooks:
from huggingface_hub import login
login()
- Else:
huggingface-cli login
For more details about authentication, check out this guide.
Then
With package_to_hub():
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import package_to_hub
# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)
# Create the evaluation env
eval_env = make_vec_env(env_id, n_envs=1)
# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)
# Train the agent
model.learn(total_timesteps=int(5000))
# This method save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub
package_to_hub(model=model,
model_name="ppo-LunarLander-v3",
model_architecture="PPO",
env_id=env_id,
eval_env=eval_env,
repo_id="ThomasSimonini/ppo-LunarLander-v3",
commit_message="Test commit")
With push_to_hub():
Push to hub only push a file to the Hub, if you want to save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub, use package_to_hub()
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import push_to_hub
# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)
# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)
# Train it for 10000 timesteps
model.learn(total_timesteps=10_000)
# Save the model
model.save("ppo-LunarLander-v3")
# Push this saved model .zip file to the hf repo
# If this repo does not exists it will be created
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename: the name of the file == "name" inside model.save("ppo-LunarLander-v3")
push_to_hub(
repo_id="ThomasSimonini/ppo-LunarLander-v3",
filename="ppo-LunarLander-v3.zip",
commit_message="Added LunarLander-v3 model trained with PPO",
)
Case 3: I use Google Colab with Classic Control/Box2D Gym Environments
- You can use xvbf (virtual screen)
!apt-get install -y xvfb python-opengl > /dev/null 2>&1
- Just put your code inside a python file and run
!xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
Case 4: I use a Virtual/Remote Machine
- You can use xvbf (virtual screen)
xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
Case 5: I want to automate upload/download from the Hub
If you want to upload or download models for many environments, you might want to automate this process. It makes sense to adhere to a fixed naming scheme for models and repositories. You will run into trouble when your environment names contain slashes. Therefore, we provide some helper classes:
import gymnasium as gym
from huggingface_sb3.naming_schemes import EnvironmentName, ModelName, ModelRepoId
env_name = EnvironmentName("seals/Walker2d-v0")
model_name = ModelName("ppo", env_name)
repo_id = ModelRepoId("YourOrganization", model_name)
# prints 'seals-Walker2d-v0'. Notice how the slash is removed so you can use it to
# construct file paths if you like.
print(env_name)
# you can still access the original gym id if needed
env = gym.make(env_name.gym_id)
# prints `ppo-seals-Walker2d-v0`
print(model_name)
# prints: `ppo-seals-Walker2d-v0.zip`.
# This is where `model.save(model_name)` will place the model file
print(model_name.filename)
# prints: `YourOrganization/ppo-seals-Walker2d-v0`
print(repo_id)
Metadata
Release files for sb3-team-hf-integration 3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sb3_team_hf_integration-3.2.tar.gz | 12.8 kB | With provenance | Details for sb3_team_hf_integration-3.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sb3_team_hf_integration-3.2-py3-none-any.whl | Python 3 | none | any | Details for sb3_team_hf_integration-3.2-py3-none-any.whl |
Total release size: 23.3 kB
Release files / sb3_team_hf_integration-3.2.tar.gz
| Download URL | sb3_team_hf_integration-3.2.tar.gz |
|---|---|
| Size | 12.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c0e1c1d04ce36363bf92b5801d7e687fa0dd7be418d59bb6b6e5c31aecd8ecbd
|
|
BLAKE2b-256 checksum How to use checksums |
5092c3a95018298292457b64b66fdb8eb33b7995f83a4b9865a4195c450f383f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.
Transparency logRelease files / sb3_team_hf_integration-3.2-py3-none-any.whl
| Download URL | sb3_team_hf_integration-3.2-py3-none-any.whl |
|---|---|
| Size | 10.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1a2ed610190d78d223a62b61824e74d05fb24926f3e369956311d4d9f1f33494
|
|
BLAKE2b-256 checksum How to use checksums |
07a76506a5643764ac9e800691a1bc2b231b36a5d6d0298dc6debc480d2c110a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.
Transparency logSecurity for sb3-team-hf-integration 3.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for sb3_team_hf_integration-3.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 26, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.