Metadata-Version: 2.4
Name: sb3-team-hf-integration
Version: 3.2
Summary: Additional code for Stable-baselines3 to load and upload models from the Hub.
Author-email: "Thomas Simonini, Omar Sanseviero and Hugging Face Team" <thomas.simonini@huggingface.co>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/huggingface/huggingface_sb3
Keywords: reinforcement learning,deep reinforcement learning,RL
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: huggingface_hub<3.0,>=2.0
Requires-Dist: pyyaml~=6.0
Requires-Dist: wasabi
Requires-Dist: cloudpickle>=1.6
Requires-Dist: stable-baselines3
Requires-Dist: moviepy
Provides-Extra: quality
Requires-Dist: ruff; extra == "quality"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: requests; extra == "test"
Requires-Dist: gymnasium[classic-control]; extra == "test"
Provides-Extra: dev
Requires-Dist: ruff; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: requests; extra == "dev"
Requires-Dist: gymnasium[classic-control]; extra == "dev"

# Hugging Face 🤗 x Stable-baselines3 v3.2

*Note: this is a fork of https://github.com/huggingface/huggingface_sb3 with fixes for newer SB3 versions*

A library to load and upload Stable-baselines3 models from the Hub with Gymnasium and Gymnasium compatible environments.

## Installation
### With pip
```
pip install sb3-team-hf-integration
```

## Examples
We wrote a tutorial on how to use 🤗 Hub and Stable-Baselines3 [here](https://colab.research.google.com/github/huggingface/deep-rl-class/blob/master/notebooks/unit1/unit1.ipynb)

If you use **Colab or a Virtual/Screenless Machine**, you can check Case 3 and Case 4.

### Case 1: I want to download a model from the Hub

You will need to set the `TRUST_REMOTE_CODE` environment variable to `True` to allow the use of `pickle.load()`:

```python
import os
import gymnasium as gym

from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy

# Allow the use of `pickle.load()` when downloading model from the hub
# Please make sure that the organization from which you download can be trusted
os.environ["TRUST_REMOTE_CODE"] = "True"

# Retrieve the model from the hub
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename = name of the model zip file from the repository
checkpoint = load_from_hub(
    repo_id="sb3/demo-hf-CartPole-v1",
    filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)

# Evaluate the agent and watch it
eval_env = gym.make("CartPole-v1")
mean_reward, std_reward = evaluate_policy(
    model, eval_env, render=False, n_eval_episodes=5, deterministic=True, warn=False
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
```

### Case 2: I trained an agent and want to upload it to the Hub
With `package_to_hub()` **we'll save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub**.
It currently **works for Gym and Atari environments**. If you use another environment, you should use `push_to_hub()` instead.

First you need to be logged in to Hugging Face:
- If you're using Colab/Jupyter Notebooks:
```python
from huggingface_hub import login

login()
```
- Else:
```
huggingface-cli login
```

For more details about authentication, check out [this guide](https://huggingface.co/docs/huggingface_hub/quick-start#authentication).

Then

**With `package_to_hub()`**:

```python
import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import package_to_hub

# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)

# Create the evaluation env
eval_env = make_vec_env(env_id, n_envs=1)

# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)

# Train the agent
model.learn(total_timesteps=int(5000))

# This method save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub
package_to_hub(model=model, 
               model_name="ppo-LunarLander-v3",
               model_architecture="PPO",
               env_id=env_id,
               eval_env=eval_env,
               repo_id="ThomasSimonini/ppo-LunarLander-v3",
               commit_message="Test commit")
```


**With `push_to_hub()`**:
Push to hub only **push a file to the Hub**, if you want to save, evaluate, generate a model card and record a replay video of your agent before pushing the repo to the hub, use `package_to_hub()`

```python
import gymnasium as gym

from stable_baselines3 import PPO
from stable_baselines3.common.env_util import make_vec_env
from huggingface_sb3 import push_to_hub

# Create the environment
env_id = "LunarLander-v3"
env = make_vec_env(env_id, n_envs=1)

# Instantiate the agent
model = PPO("MlpPolicy", env, verbose=1)

# Train it for 10000 timesteps
model.learn(total_timesteps=10_000)

# Save the model
model.save("ppo-LunarLander-v3")

# Push this saved model .zip file to the hf repo
# If this repo does not exists it will be created
## repo_id = id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name})
## filename: the name of the file == "name" inside model.save("ppo-LunarLander-v3")
push_to_hub(
    repo_id="ThomasSimonini/ppo-LunarLander-v3",
    filename="ppo-LunarLander-v3.zip",
    commit_message="Added LunarLander-v3 model trained with PPO",
)
```
### Case 3: I use Google Colab with Classic Control/Box2D Gym Environments
- You can use xvbf (virtual screen)
```
!apt-get install -y xvfb python-opengl > /dev/null 2>&1
```
- Just put your code inside a python file and run
```
!xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
```

### Case 4: I use a Virtual/Remote Machine
- You can use xvbf (virtual screen)

```
xvfb-run -s "-screen 0 1400x900x24" <your_python_file>
```

### Case 5: I want to automate upload/download from the Hub
If you want to upload or download models for many environments, you might want to 
automate this process. 
It makes sense to adhere to a fixed naming scheme for models and repositories.
You will run into trouble when your environment names contain slashes.
Therefore, we provide some helper classes:

```python
import gymnasium as gym
from huggingface_sb3.naming_schemes import EnvironmentName, ModelName, ModelRepoId

env_name = EnvironmentName("seals/Walker2d-v0")
model_name = ModelName("ppo", env_name)
repo_id = ModelRepoId("YourOrganization", model_name)

# prints 'seals-Walker2d-v0'. Notice how the slash is removed so you can use it to 
# construct file paths if you like.
print(env_name)

# you can still access the original gym id if needed
env = gym.make(env_name.gym_id)  

# prints `ppo-seals-Walker2d-v0`
print(model_name)  

# prints: `ppo-seals-Walker2d-v0.zip`. 
# This is where `model.save(model_name)` will place the model file
print(model_name.filename)  

# prints: `YourOrganization/ppo-seals-Walker2d-v0`
print(repo_id)
```
