Metadata-Version: 2.4
Name: isaacsim-mcp-server
Version: 0.5.1
Summary: Control NVIDIA Isaac Sim robotics simulator through MCP -- 41 tools for scene management, robot control, sensors, and simulation
Project-URL: Homepage, https://github.com/whats2000/isaacsim-mcp-server
Project-URL: Repository, https://github.com/whats2000/isaacsim-mcp-server
Project-URL: Issues, https://github.com/whats2000/isaacsim-mcp-server/issues
Author: omni-mcp, whats2000
License-Expression: MIT
License-File: LICENSE
License-File: LICENSE_HEADER.py
License-File: LICENSE_README.md
Keywords: isaac-sim,mcp,nvidia,robotics,simulation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: mcp[cli]
Description-Content-Type: text/markdown

# Isaac Sim MCP Server

<!-- mcp-name: io.github.whats2000/isaacsim-mcp-server -->

[![PyPI version](https://img.shields.io/pypi/v/isaacsim-mcp-server)](https://pypi.org/project/isaacsim-mcp-server/)
[![Isaac Sim 5.1.0](https://img.shields.io/badge/Isaac_Sim-5.1.0-76b900)](https://developer.nvidia.com/isaac-sim)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![MCP Quality](https://archestra.ai/mcp-catalog/api/badge/quality/whats2000/isaacsim-mcp-server)](https://archestra.ai/mcp-catalog/api/badge/quality/whats2000/isaacsim-mcp-server)

> Natural language control for NVIDIA Isaac Sim through the [Model Context Protocol](https://modelcontextprotocol.io/) (MCP).

Connect any MCP-compatible IDE (Cursor, VS Code, Claude Code, Windsurf, JetBrains) to a running Isaac Sim instance and control it with plain-English prompts -- create robots, build scenes, run simulations, and debug physics all from your editor.

<a href="https://youtu.be/U6QaE8nX-DA">
  <img src="https://img.youtube.com/vi/U6QaE8nX-DA/0.jpg" alt="Demo Video" width="600">
</a>

---

## Highlights

- **42 tools** across 9 categories -- scene, objects, lighting, robots, sensors, materials, assets, simulation, graphs
- **107+ robots** auto-discovered from the Isaac Sim asset library (Franka, UR, Unitree, Boston Dynamics, and more)
- **Step-and-observe** debugging -- step the simulation and inspect prim positions, joint states, and physics in one call
- **Hot-reload** -- iterate on Python controllers without restarting Isaac Sim
- **Multi-instance** -- run multiple Isaac Sim sessions side by side on different ports
- Built for **Isaac Sim 5.1.0** with a modular adapter layer for version isolation

---

## Installation

### Option A: pip install (recommended)

```bash
pip install isaacsim-mcp-server
```

This installs the MCP server and the `isaacsim-mcp-server` CLI. You still need the Isaac Sim extension from the repo (see [Launching Isaac Sim](#2-launch-isaac-sim-with-the-extension) below).

### Option B: From source

```bash
git clone https://github.com/whats2000/isaacsim-mcp-server
cd isaacsim-mcp-server
./scripts/setup_python_env.sh
```

### Requirements

| Requirement | Version |
|-------------|---------|
| NVIDIA Isaac Sim | `5.1.0` |
| Python | `3.10+` |
| `uv` | latest (for source install) |
| Platform | Linux (Ubuntu 22.04+) |

> [!IMPORTANT]
> Currently only **Linux** is supported. Windows support is planned.
> macOS is not supported because NVIDIA Isaac Sim does not run on macOS.

> [!NOTE]
> We are welcoming contributions to support other Isaac Sim versions. 
> The adapter layer is designed for easy version isolation.

---

## Quick Start

### 1. Set up the environment

If you installed from source:

```bash
./scripts/setup_python_env.sh
```

### 2. Launch Isaac Sim with the extension

```bash
./scripts/run_isaac_sim.sh
```

You should see in the logs:

```
Registered 41 command handlers
Isaac Sim MCP server started on localhost:8766
```

<details>
<summary>Optional: Beaver3D / NVIDIA API keys for 3D generation</summary>

```bash
export BEAVER3D_MODEL="<your beaver3d model name>"
export ARK_API_KEY="<your beaver3d api key>"
export NVIDIA_API_KEY="<your nvidia api key>"
```

</details>

### 3. Connect your IDE

Add the MCP server to your editor. Replace the path with your actual repo location.

<details>
<summary><strong>Claude Code (CLI)</strong></summary>

```bash
claude mcp add isaac-sim /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh
```

Or edit `~/.claude.json` / `.mcp.json`:

```json
{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}
```

</details>

<details>
<summary><strong>VS Code</strong></summary>

Create `.vscode/mcp.json` in your workspace:

```json
{
  "servers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}
```

</details>

<details>
<summary><strong>Cursor</strong></summary>

Open **Cursor Settings > MCP**, or edit `~/.cursor/mcp.json`:

```json
{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}
```

</details>

<details>
<summary><strong>Claude Desktop</strong></summary>

Edit the config file for your platform:

- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
- Linux: `~/.config/Claude/claude_desktop_config.json`

```json
{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}
```

</details>

<details>
<summary><strong>Windsurf</strong></summary>

Open **Windsurf Settings > MCP** or edit `~/.codeium/windsurf/mcp_config.json`:

```json
{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    }
  }
}
```

</details>

<details>
<summary><strong>JetBrains IDEs</strong></summary>

Go to **Settings > Tools > AI Assistant > MCP Servers** and add the server. See the [JetBrains MCP docs](https://www.jetbrains.com/help/idea/model-context-protocol.html) for details.

</details>

### 4. Start prompting

```text
Check the connection with get_scene_info.
If the scene is empty, create a physics scene.
Add a Franka robot at the origin and a Go1 quadruped at [2, 0, 0].
```

---

## Architecture

```text
MCP Client (IDE)
      |
      v
isaacsim-mcp-server          (PyPI package / CLI)
      |
      v  TCP socket (localhost:8766)
      |
isaac.sim.mcp_extension      (Omniverse extension)
      |
      v
Handlers -> Adapter -> Isaac Sim 5.1.0 APIs
```

---

## Tools

42 tools across 9 categories:

| Category | Count | What you can do |
|----------|------:|-----------------|
| **Scene** | 7 | Inspect scenes, create physics, list/load environments, browse prims |
| **Objects** | 4 | Create, delete, transform, and clone primitives |
| **Lighting** | 2 | Create and tune lights |
| **Robots** | 6 | Spawn 107+ robots, inspect joints, set positions, refresh library |
| **Sensors** | 4 | Create cameras/LiDAR, capture images, get point clouds |
| **Materials** | 2 | Create and apply materials |
| **Assets** | 4 | Import URDF, load/search USD, generate 3D models |
| **Graphs** | 2 | Build and edit Action Graphs (OnPlaybackTick, ScriptNode, script file attachment) |
| **Simulation** | 11 | Play/pause/stop/step, execute Python, inspect physics, hot-reload |

<details>
<summary>Full tool list</summary>

**Scene:** `get_scene_info` `create_physics_scene` `clear_scene` `list_prims` `get_prim_info` `list_environments` `load_environment`

**Objects:** `create_object` `delete_object` `transform_object` `clone_object`

**Lighting:** `create_light` `modify_light`

**Robots:** `create_robot` `list_available_robots` `refresh_robot_library` `get_robot_info` `set_joint_positions` `get_joint_positions`

**Sensors:** `create_camera` `capture_image` `create_lidar` `get_lidar_point_cloud`

**Materials:** `create_material` `apply_material`

**Assets:** `import_urdf` `load_usd` `search_usd` `generate_3d`

**Graphs:** `create_action_graph` `edit_action_graph`

**Simulation:** `play_simulation` `pause_simulation` `stop_simulation` `step_simulation` `set_physics_params` `get_isaac_logs` `get_simulation_state` `get_physics_state` `get_joint_config` `execute_script` `reload_script`

</details>

---

## Example Prompts

**Scene bootstrap**
```text
Check the connection with get_scene_info. If the scene is empty, create a physics scene.
Add stronger lighting and place a camera that looks at the workspace.
```

**Robot layout**
```text
Create three Franka robots in a row at [0,0,0], [2,0,0], and [4,0,0].
Then add a Go1 robot at [1, 3, 0].
```

**Environment loading**
```text
List available environments, choose a warehouse-like one, and load it.
Create a camera and capture an image.
```

**Asset search and 3D generation**
```text
Search for a rusty desk, load the best result near [0, 5, 0], scaled to [2, 2, 2].
```

---

## Advanced Usage

### Multiple Instances

Run multiple Isaac Sim sessions side by side. Each uses a different port (auto-assigned from `8766`).

```bash
# First instance (default port 8766)
claude mcp add isaac-sim /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh

# Second instance (port 8767)
claude mcp add isaac-sim-2 -e ISAAC_MCP_PORT=8767 -- /path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh
```

<details>
<summary>JSON config for multiple instances</summary>

```json
{
  "mcpServers": {
    "isaac-sim": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh"
    },
    "isaac-sim-2": {
      "command": "/path/to/isaacsim-mcp-server/scripts/run_mcp_server.sh",
      "env": { "ISAAC_MCP_PORT": "8767" }
    }
  }
}
```

</details>

### Desktop Launcher (Linux)

Install a dedicated **Isaac Sim MCP** application icon:

```bash
./scripts/install_desktop_entry.sh
```

This creates a launcher that auto-assigns ports, waits for the extension socket, and cleans up on exit.

### Recommended Workflow

1. Start with `get_scene_info` to verify the connection
2. Create a physics scene if the stage is empty
3. Prefer purpose-built tools before `execute_script`
4. Use `list_available_robots` / `list_environments` before loading
5. Use `create_action_graph` to wire OnPlaybackTick → ScriptNode controllers
6. Use `step_simulation` with `observe_prims` and `observe_joints` for debugging
7. Use `reload_script` to iterate on controllers without restarting

---

## Demo: Franka Pick-and-Place

A ready-to-run demo at `demo/franka_pick_place.py` using RMPflow for motion planning:

```text
Please use the Isaac MCP tool complete this:

Create a physics scene with a ground plane, then spawn a Franka FR3 robot at the origin.

Add two textured tables with a gap along Y. Place a small textured cube with physics enabled on top of the first table.

Use `create_action_graph` to wire `OnPlaybackTick` → `ScriptNode`, and write a pick-and-place controller script using RMPflow for motion planning. Save the script to the `demo/` directory.

Use `get_prim_info` to query actual positions and sizes of the tables and cube before writing the controller — do not hardcode coordinates.

Start the simulation with Play. The robot should pick the cube from table 1 and place it on table 2. Verify the process using `step_simulation` with `observe_prims` on the cube to confirm it reaches table 2.
```

Uses `create_action_graph` with `script_file` for one-step Action Graph + ScriptNode setup, plus the observability tools: `get_joint_config`, `step_simulation` with `observe_prims`, `get_physics_state`, and `edit_action_graph` for script hot-reload.

---

## Development

```bash
# Run the MCP inspector
./.venv/bin/python -m mcp dev ./isaac_mcp/server.py
```

The inspector is available at `http://localhost:5173`.

### Setup Notes

| Script | Purpose | Default |
|--------|---------|---------|
| `setup_python_env.sh` | Create venv and install package | Python 3.10 |
| `run_isaac_sim.sh` | Launch Isaac Sim with extension | `$HOME/isaacsim` |
| `run_mcp_server.sh` | Start the MCP server | Port 8766 |
| `launch_isaac_sim_mcp.sh` | Combined launcher | Auto-assigns port |
| `dev_mcp_server.sh` | Dev server with hot-reload | Port 8766 |

Override defaults:

```bash
PYTHON_SPEC=3.11 ./scripts/setup_python_env.sh
ISAACSIM_ROOT=/opt/isaacsim ./scripts/run_isaac_sim.sh
```

<details>
<summary>Troubleshooting</summary>

If Isaac Sim says `Can't find extension with name: isaac.sim.mcp_extension`:

```bash
# Make sure you're in the repo root
pwd
test -f ./isaac.sim.mcp_extension/config/extension.toml && echo OK
```

Note: `--ext-folder` must point to the **repo root**, not to `isaac.sim.mcp_extension/` directly.

</details>

---

## Contributing

Pull requests are welcome. Improvements to tools, docs, adapters, and tests are all useful.

## License

MIT License. Copyright (c) 2023-2025 omni-mcp, Copyright (c) 2026 whats2000. See [LICENSE](LICENSE).
