Metadata-Version: 2.3
Name: holosoma
Version: 0.0.1
Summary: Holosoma: core training framework for humanoid robot reinforcement learning (locomotion + whole-body tracking).
Keywords: humanoid,robotics,reinforcement-learning,locomotion,whole-body-tracking,sim2real
Author: Amazon FAR Team
License: Apache-2.0
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Requires-Dist: astor
Requires-Dist: easydict
Requires-Dist: importlib-resources ; python_full_version < '3.9'
Requires-Dist: ipdb
Requires-Dist: joblib
Requires-Dist: loguru
Requires-Dist: lxml
Requires-Dist: lxml-stubs
Requires-Dist: matplotlib
Requires-Dist: meshcat
Requires-Dist: mypy==1.14.1
Requires-Dist: notebook
Requires-Dist: numpy-stl
Requires-Dist: numpy>=1.23.5,<2
Requires-Dist: omegaconf
Requires-Dist: onnx
Requires-Dist: onnxruntime
Requires-Dist: open3d
Requires-Dist: opencv-python
Requires-Dist: plotly
Requires-Dist: pydantic
Requires-Dist: pygame
Requires-Dist: pynput
Requires-Dist: pytest
Requires-Dist: rich
Requires-Dist: scipy
Requires-Dist: shapely
Requires-Dist: smart-open[s3]
Requires-Dist: tensorboard
Requires-Dist: tensordict
Requires-Dist: termcolor
Requires-Dist: tqdm
Requires-Dist: trimesh
Requires-Dist: types-pynput
Requires-Dist: types-python-dateutil
Requires-Dist: types-requests
Requires-Dist: types-tqdm
Requires-Dist: tyro>=1.0.0
Requires-Dist: wandb==0.22.0
Requires-Dist: warp-lang>=1.10
Requires-Dist: yourdfpy>=0.0.58
Requires-Dist: zmq
Requires-Dist: far-booster-sdk==0.1.1 ; extra == 'booster'
Requires-Dist: far-unitree-sdk==0.1.6 ; extra == 'unitree'
Requires-Python: >=3.8
Provides-Extra: booster
Provides-Extra: ros2
Provides-Extra: unitree
Description-Content-Type: text/markdown

# Holosoma Training Framework

Core training framework for humanoid robot reinforcement learning with support for locomotion (velocity tracking) and whole-body tracking tasks.

| **Category** | **Supported Options** |
|-------------|----------------------|
| **Simulators** | IsaacGym, IsaacSim, MJWarp (training) \| Mujoco (evaluation) |
| **Algorithms** | PPO, FastSAC |
| **Robots** | Unitree G1, Booster T1 |

## Training

All training/eval scripts support `--help` for discovering available flags, e.g. `python src/holosoma/holosoma/train_agent.py --help`.

> **Note:** Video recording is enabled by default with `logger:wandb`. On headless servers, you may need to disable video or configure rendering. See [Video Recording](#video-recording) below.

### Locomotion (Velocity Tracking)

Train robots to track velocity commands.

```bash
# G1 with FastSAC on IsaacGym
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    logger:wandb \
    --training.seed 1

# T1 with PPO on IsaacSim
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:t1-29dof \
    simulator:isaacsim \
    logger:wandb \
    --training.seed 1
```

Once checkpoints are saved, you can evaluate policies using [In-Training Evaluation](#in-training-evaluation) (same simulator as training) or cross-simulator evaluation in MuJoCo (see [holosoma_inference](../holosoma_inference/README.md)).

### MJWarp Training for Locomotion (Velocity Tracking)

Train using the MJWarp simulator (GPU-accelerated MuJoCo). **Note: MJWarp support is in beta.**

```bash
# G1 with FastSAC
source scripts/source_mujoco_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:mjwarp \
    logger:wandb

# G1 with PPO
source scripts/source_mujoco_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof \
    simulator:mjwarp \
    logger:wandb

# T1 with FastSAC
source scripts/source_mujoco_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:t1-29dof-fast-sac \
    simulator:mjwarp \
    logger:wandb

# T1 with PPO
source scripts/source_mujoco_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:t1-29dof \
    simulator:mjwarp \
    logger:wandb \
    --terrain.terrain-term.scale-factor=0.5  # required to avoid training instabilities

```

> **Note:**
> - MJWarp uses `nconmax=96` (maximum contacts per environment) by default. This can be adjusted via `--simulator.config.mujoco-warp.nconmax-per-env=96` if needed.
> - These examples use `--training.num-envs=4096`, but you may need to adjust this value based on your hardware.
> - When training T1 with PPO on mixed terrain, use `--terrain.terrain-term.scale-factor=0.5` to avoid training instabilities.


### Whole-Body Tracking

Train robots to track full-body motion sequences.

**Note**: Currently only supported for Unitree G1 / IsaacSim.

```bash
# G1 with FastSAC
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-wbt-fast-sac \
    logger:wandb

# G1 with PPO
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-wbt \
    logger:wandb

# Custom motion file
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-wbt \
    logger:wandb \
    --command.setup_terms.motion_command.params.motion_config.motion_file="holosoma/data/motions/g1_29dof/whole_body_tracking/<your file>.npz"

# Visualize the motion file in isaacsim before training
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/replay.py \
    exp:g1-29dof-wbt \
    --training.headless=False \
    --training.num_envs=1
```

Once checkpoints are saved, you can evaluate policies using [In-Training Evaluation](#in-training-evaluation) (same simulator as training) or cross-simulator evaluation in MuJoCo (see [holosoma_inference](../holosoma_inference/README.md)).

---

## Evaluation

### In-Training Evaluation

For evaluating policies with the exact same configuration used during training (same simulator, environment settings, etc.):

```bash
# Evaluate checkpoint from Wandb
python src/holosoma/holosoma/eval_agent.py \
    --checkpoint=wandb://<ENTITY>/<PROJECT>/<RUN_ID>/<CHECKPOINT_NAME>
# e.g., --checkpoint=wandb://username/fastsac-t1-locomotion/abcdefgh/model_0010000.pt

# Evaluate local checkpoint
python src/holosoma/holosoma/eval_agent.py \
    --checkpoint=<CHECKPOINT_PATH>
# e.g., --checkpoint=/home/username/checkpoints/fastsac-t1-locomotion/model_0010000.pt
```

This evaluation mode:
- Automatically loads the training configuration from the checkpoint
- Runs evaluation in the same simulator and environment as training
- Can export policies to ONNX format (via `--training.export_onnx=True`)
- For locomotion evaluation, supports interactive velocity commands via keyboard (when simulator window is active):
  - `w`/`a`/`s`/`d`: linear velocity commands
  - `q`/`e`: angular velocity commands
  - `z`: zero velocity command

### Cross-Simulator Evaluation (MuJoCo)

For testing trained policies in MuJoCo simulation or deploying to real robots, see the [holosoma_inference documentation](../holosoma_inference/README.md). This covers:
- Sim-to-sim evaluation (IsaacGym/IsaacSim → MuJoCo)
- Real robot deployment (both locomotion and WBT)

**Note**: ONNX policies are typically exported alongside `.pt` checkpoints during training, but can also be generated using the in-training evaluation script above.

## Advanced Configuration

The training system uses a hierarchical configuration system. The `exp` config serves as the main entry point with default configurations tuned for each algorithm and robot. You can customize training by overriding parameters on the command line.

> **Tip**: When composing Tyro configs, pass the `exp:<name>` preset before any other config fragments (e.g., `logger:wandb`). Tyro expects the base experiment to be declared first, and reversing the order can lead to confusing resolution errors.

### Logging with Weights & Biases

```bash
source scripts/source_isaacsim_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof \
    simulator:isaacsim \
    --training.seed 1 \
    --algo.config.use-symmetry=False \
    logger:wandb \
    --logger.project locomotion-g1-29dof-ppo \
    --logger.name ppo-without-symmetry-seed1
```

### Video Recording

Video recording is **enabled by default** when using `logger:wandb`. Videos are recorded periodically and uploaded to Weights & Biases.

**Configuration:**
```bash
# Disable video recording
--logger.video.enabled False

# Adjust recording interval (episodes)
--logger.video.interval 10

# Change resolution
--logger.video.width 640 --logger.video.height 360
```

**Troubleshooting Headless Environments:**

If training fails on headless servers with display/rendering errors (e.g., `GLXBadFBConfig`, `eglInitialize failed`, `GLFW initialization failed`):

- **IsaacSim:** Disable video with `--logger.video.enabled False`, or force EGL with `DISPLAY= python ...`, or use virtual display with `xvfb-run -a python ...`
- **MJWarp/MuJoCo:** Set environment variable before training: `export MUJOCO_GL=egl`. See [MuJoCo docs](https://mujoco.readthedocs.io/en/stable/programming/index.html#using-opengl)
- **IsaacGym:** Usually works in headless environments. If issues occur, disable video with `--logger.video.enabled False`

### Terrain

```bash
# Use plane terrain instead of mixed terrain
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    terrain:terrain-locomotion-plane
```

### Multi-GPU Training

```bash
source scripts/source_isaacgym_setup.sh
torchrun --nproc_per_node=4 src/holosoma/holosoma/train_agent.py \
    exp:t1-29dof-fast-sac \
    simulator:isaacgym \
    --training.num-envs 16384  # global/total number of environments
```

### Custom Reward Weights

```bash
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --reward.terms.tracking-lin-vel.weight=2.5 \
    --reward.terms.feet-phase.params.swing-height=0.12
```

### Observation Noise

```bash
# Disable observation noise
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --observation.groups.actor-obs.enable-noise=False
```

### Observation History Length

Some policies benefit from stacking multiple timesteps of observations. You can increase the history length used during training with:

```bash
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --observation.groups.actor_obs.history-length 4
```

Make sure to pass the same history length when running inference so the exported ONNX policy receives inputs with the correct shape.

### Curriculum Learning

```bash
# Disable curriculum
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --curriculum.setup-terms.penalty-curriculum.params.enabled=False

# Custom curriculum threshold (for shorter episodes)
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --simulator.config.sim.max-episode-length-s=10.0 \
    --curriculum.setup-terms.penalty-curriculum.params.level-up-threshold=350
```

### Domain Randomization

```bash
source scripts/source_isaacgym_setup.sh
python src/holosoma/holosoma/train_agent.py \
    exp:g1-29dof-fast-sac \
    simulator:isaacgym \
    --randomization.setup-terms.push-randomizer-state.params.enabled=False \
    --randomization.setup-terms.randomize-base-com-startup.params.enabled=True \
    --randomization.setup-terms.mass-randomizer.params.added-mass-range=[-1.0,3.0]
```
