Metadata-Version: 2.5
Name: omniverse-mcp
Version: 0.5.1
Summary: MCP server for AI control of NVIDIA Omniverse applications (Isaac Sim 5.x and up, USD Composer, and more)
Project-URL: Homepage, https://github.com/serionist/omniverse-mcp
Project-URL: Repository, https://github.com/serionist/omniverse-mcp
Project-URL: Documentation, https://github.com/serionist/omniverse-mcp/tree/main/docs
Project-URL: Issues, https://github.com/serionist/omniverse-mcp/issues
Author: Soma Osvay
License-Expression: MIT
License-File: LICENSE
Keywords: ai,isaac-sim,mcp,model-context-protocol,nvidia,omniverse,robotics,simulation,usd
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Requires-Dist: mcp==2.3.0
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/serionist/omniverse-mcp/main/docs/images/logo.svg" width="96" alt="">
</p>

<h1 align="center">Omniverse MCP</h1>

<p align="center">
  Let an AI assistant work in NVIDIA Isaac Sim and other Omniverse Kit apps:<br>
  look at the scene, edit it, run physics, drive robots and record what happens.
</p>

<p align="center">
  <a href="https://pypi.org/project/omniverse-mcp/"><img alt="PyPI" src="https://img.shields.io/pypi/v/omniverse-mcp"></a>
  <a href="https://pypi.org/project/omniverse-mcp/"><img alt="Python versions" src="https://img.shields.io/pypi/pyversions/omniverse-mcp"></a>
  <a href="https://github.com/astral-sh/uv"><img alt="uv" src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json"></a>
  <a href="https://github.com/serionist/omniverse-mcp/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/serionist/omniverse-mcp/actions/workflows/ci.yml/badge.svg?branch=main"></a>
  <a href="https://github.com/serionist/omniverse-mcp/actions/workflows/codeql.yml"><img alt="CodeQL" src="https://github.com/serionist/omniverse-mcp/actions/workflows/codeql.yml/badge.svg?branch=main"></a>
  <a href="https://github.com/serionist/omniverse-mcp/blob/main/LICENSE"><img alt="Licence: MIT" src="https://img.shields.io/github/license/serionist/omniverse-mcp?label=licence"></a>
  <a href="https://glama.ai/mcp/servers/serionist/omniverse-mcp"><img alt="Glama score" src="https://glama.ai/mcp/servers/serionist/omniverse-mcp/badges/score.svg"></a>
  <a href="https://mcpservers.org/servers/serionist/omniverse-mcp"><img alt="Listed on mcpservers.org" src="https://mcpservers.org/badge.svg"></a>
</p>

<!-- The official MCP registry trusts the PyPI package by this line, as server.json names it. -->
<!-- mcp-name: io.github.serionist/omniverse-mcp -->

<p align="center">
  <img src="https://raw.githubusercontent.com/serionist/omniverse-mcp/main/docs/images/hero-viewport.jpg" width="49%" alt="A Franka arm on a table with a red, a green and a blue block and a yellow ball, as capture_viewport returns it from Isaac Sim">
  <img src="https://raw.githubusercontent.com/serionist/omniverse-mcp/main/docs/images/hero-segmentation.png" width="49%" alt="The same view's instance segmentation: each link of the arm, the table, each block and the ball in its own colour">
</p>
<p align="center">
  <sub>One <code>capture_viewport</code> call in Isaac Sim 6.2: the frame, and every object in its own colour, with the
  legend and the screen boxes that tell the assistant what it sees.</sub>
</p>

Omniverse MCP connects an AI client, such as Claude Code, Cursor or VS Code, to a running Kit app. It comes in two
parts, from one release:

- **the server,** `omniverse-mcp`: the [MCP](https://modelcontextprotocol.io/) server your AI client starts, from
  PyPI;
- **the bridge,** `ovmcp.bridge`: the Kit extension inside the Kit app, listed in Kit as *Omniverse MCP Bridge*, and
  in Kit's community extension registry.

Claude Code users get both from one plugin, with skills for each kind of work.

```
AI client  --MCP, stdio-->  omniverse-mcp  --HTTP, with a token-->  ovmcp.bridge
                            (the server)                         (inside the Kit app, port 8211)
```

## Requirements

- **A Kit app of Kit 107 or later,** on Windows or Linux. Tested on Isaac Sim 6.2 (Windows and Linux) and 5.1
  (Linux); USD Composer and other Kit apps should work, but are untested. On Linux, Isaac Sim 5.1 runs on NVIDIA's
  580 drivers; on a 595 driver its renderer crashes as it starts.
- **[uv](https://docs.astral.sh/uv/getting-started/installation/),** which runs the server. Install it, then open a
  new terminal and restart your AI client, so both find `uvx`:
  - macOS and Linux: `curl -LsSf https://astral.sh/uv/install.sh | sh`
  - Windows: `powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"`
- **An MCP client that can read files:** the tools write images and long answers to files and return their paths.
- **Kit and the client on the same machine,** or see [Kit on another machine](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#kit-on-another-machine).

## Install

### Claude Code: the plugin

In Claude Code:

```
/plugin marketplace add serionist/omniverse-mcp
/plugin install omniverse-mcp@omniverse-mcp
```

If it asks you to, run `/reload-plugins`. From a terminal, the same is `claude plugin marketplace add
serionist/omniverse-mcp`, then `claude plugin install omniverse-mcp@omniverse-mcp`, and the plugin loads when
Claude Code next starts.

Then run `/omniverse-mcp:setup`. It installs the bridge of the plugin's release, tells you what to click in Kit, and
checks the connection. The plugin also brings skills for scenes, geometry, simulation, scripts and bug reports, which
Claude uses when a request fits ([the plugin](https://github.com/serionist/omniverse-mcp/blob/main/plugin/README.md)).

### Other MCP clients

1. **Install the bridge:**

   ```
   uvx omniverse-mcp extension install
   ```

   It copies `ovmcp.bridge` into your user data folder, then prints the folder Kit needs, under "Kit finds it once
   its extension search paths hold this folder". Or install the bridge in Kit itself, from Kit's community registry,
   and give the server its version in step 3
   ([From Kit's community registry](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#from-kits-community-registry)).
2. **Turn it on in the Kit app,** once:
   1. **Window > Extensions**, the menu (☰) at the top right, **Settings**.
   2. Under **Extension Search Paths**, the green **+**, then paste the folder. The **+** adds an empty row: click it
      to type.
   3. Search for `ovmcp` under **THIRD PARTY**, and turn **Omniverse MCP Bridge** on, with **Autoload**.

   The bridge starts at once, and Kit's console shows `[ext: ovmcp.bridge-<version>] startup`. Kit saves the search
   path and Autoload when it exits, so quit it once with **File > Exit**. A headless Kit takes
   `--ext-folder <that folder> --enable ovmcp.bridge` instead ([headless Kit](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#headless-kit)).
3. **Add the server to your client's MCP config,** and restart the client. In most clients' JSON:

   ```json
   {
     "mcpServers": {
       "omniverse": { "command": "uvx", "args": ["omniverse-mcp"], "env": { "PYTHONPATH": "" } }
     }
   }
   ```

   Each client's file and form, VS Code's and Codex's included, are in [MCP clients](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#mcp-clients). The
   empty `PYTHONPATH` keeps a Python path that Isaac Sim's environment sets out of the server's Python.

Your client gets the server's short instructions with it. If it ignores them, see
[docs/AI_GUIDE.md](https://github.com/serionist/omniverse-mcp/blob/main/docs/AI_GUIDE.md), which also points at the plugin's skills as plain Markdown files.

### Check that it works

With the Kit app running, ask your assistant to *"call get_sim_state"*. It answers with lines such as:

```
state = stopped
up_axis = Z
...
extension_version = 0.5.1
server_version = 0.5.1
```

Something else? See [Troubleshooting](#troubleshooting).

> **Used the extension before 0.2.0?** It came from a clone of this repo, and before that it was
> `isaacsim.mcp.bridge`. Turn the old one off and remove its search path, so Kit loads one copy. See the
> [changelog](https://github.com/serionist/omniverse-mcp/blob/main/CHANGELOG.md).

## What you can ask

> "What's in the scene? Capture the viewport and tell me where each object is."

> "Put a table at the origin with a red, a green and a blue cube on it, and light it so the capture isn't black."

> "Show me /World/Table from its four top corners, with everything else hidden."

> "How many faces does /World/Robot have, and which parts have the most?"

> "Add a Franka arm on the table, and list its joints and their limits."

> "Drop a cube from one metre, record two seconds of it falling, and show me the last frame."

> "Export /World/Table as a self-contained USD file."

> "That failed. Show me Kit's warnings and errors."

## The tools

| Area | Tools | What they do |
|---|---|---|
| **Scene** | `get_scene_tree`, `get_prim_properties`, `get_prim_bounds`, `set_prim_transform`, `create_prim`, `delete_prim`, `clone_prim`, `set_visibility`, `set_prim_attribute`, `save_scene`, `open_scene`, `new_scene` | Read the scene's hierarchy and properties; create, move, copy, hide and delete objects; set an attribute such as a light's intensity; save, open a file, or start an empty scene |
| **Camera** | `capture_viewport`, `inspect_prim`, `capture_sweep`, `set_camera`, `viewport_light` | Capture the viewport with each object's screen box and colour, see one object from several sides, capture many viewpoints and variant states in one job and again after a change, aim the camera at a point or an object, light the view |
| **Materials** | `set_material`, `update_material_paths` | Give an object a colour and surface; fix material paths after files move |
| **Simulation** | `sim_control`, `get_sim_state`, `get_performance` | Play, pause, stop and step physics; the state, the up axis and the versions; measure the frame rate, the physics step and GPU memory |
| **Physics** | `set_physics_properties`, `apply_force`, `raycast` | Mass, friction and bounce; make a collider; push an object; cast a ray |
| **Robots** | `create_robot`, `get_robot_info`, `set_joint_targets` | Add a robot from Isaac Sim's library (Franka, UR10, G1, Spot and more), read its joints and where they are, set joint targets |
| **Recording** | `start_recording`, `stop_recording`, `get_recording_frame` | Record frames from any camera at any size, and objects' state, while physics plays, then look back at any frame |
| **Geometry** | `get_mesh_stats`, `compare_prims` | Count faces and vertices and find the heaviest parts, compare two objects or variants |
| **USD files and variants** | `flatten_usd`, `export_prim_as_file`, `create_variant_structure`, `set_variant_selection` | Write a stage as one file, export an object, make and switch variant sets |
| **Kit** | `get_logs`, `manage_extensions`, `draw_debug` | Read Kit's log, turn Kit extensions on and off, draw lines and points in the viewport |
| **Scripts** | `execute_script`, `run_script_job`, `get_job`, `cancel_job` | Run Python inside Kit for anything the other tools don't do; run a long script as a job while Kit renders, read its progress and cancel it |

Every tool, with its parameters, is in the [tool reference](https://github.com/serionist/omniverse-mcp/blob/main/docs/TOOLS.md).

<p align="center">
  <img src="https://raw.githubusercontent.com/serionist/omniverse-mcp/main/docs/images/inspect-orbit.jpg" width="100%" alt="The workcell from four corners, as inspect_prim frames it with everything else hidden">
</p>
<p align="center">
  <sub><code>inspect_prim(prim_path="/World/Workcell", angles=["top_front_right", "top_back_right", "top_back_left",
  "top_front_left"], isolate=True)</code>: one call, every side, framed to fit, the rest of the scene hidden.</sub>
</p>

## Updating

The server and the bridge come from one release, so update them together, then restart Kit and your client.

- **The plugin:** `claude plugin marketplace update omniverse-mcp`, then
  `claude plugin update omniverse-mcp@omniverse-mcp`, and restart Claude Code. Then run `/omniverse-mcp:setup`, which
  installs the matching bridge.
- **Other clients:** `uvx omniverse-mcp@latest extension install` fetches the newest server and installs its bridge.
  On Windows a running Kit holds the bridge's folder: quit it first. Unpinned, uvx can also move to a new release
  by itself; the server then names the command that installs the matching bridge. To stay on one release, pin it
  ([Updating](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#updating)).
- **The bridge from Kit's community registry:** update it in Kit's Extensions window, and the server's pin with it.

## Configuration

### The server

```
omniverse-mcp [--host HOST] [--port PORT] [--output-dir DIR]    the MCP server, over stdio
omniverse-mcp extension install                                 install the bridge of this release
omniverse-mcp extension path                                    print the folder Kit's search paths need
```

| Variable | Flag | Default | What |
|---|---|---|---|
| `OMNIVERSE_MCP_HOST` | `--host` | `127.0.0.1` | where the bridge listens |
| `OMNIVERSE_MCP_PORT` | `--port` | `8211` | the bridge's port |
| `OMNIVERSE_MCP_OUTPUT_DIR` | `--output-dir` | `mcp_output` in the working folder | where images and long answers go |
| `OMNIVERSE_MCP_TOKEN` | | the token file | the bridge token, for a Kit app on another machine |
| `OMNIVERSE_MCP_STATE_DIR` | | per system | the token file's folder, set the same on both sides |

The plugin's server reads them from the environment Claude Code starts in. Add `mcp_output/` to your project's
`.gitignore`.

### The bridge

| Setting | Default | What |
|---|---|---|
| `/exts/ovmcp.bridge/host` | `127.0.0.1` | the address it listens on |
| `/exts/ovmcp.bridge/port` | `8211` | its port |
| `/exts/ovmcp.bridge/token` | empty: a new token at every start | a fixed token, for a remote MCP server |

Set them on Kit's command line, such as `--/exts/ovmcp.bridge/port=8212`, and give the server the same port in
`OMNIVERSE_MCP_PORT`. Don't edit the installed extension's `extension.toml`: the next `extension install` replaces it.

### The bridge token

The bridge runs any Python the server sends, so it answers only a server that holds its token. At every start it
writes a new token to a file only your user can read, and the server on the same machine reads it: nothing to set up.
Requests from web pages are refused. [Kit on another machine](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#kit-on-another-machine) says how to set a
token by hand, and [Architecture](https://github.com/serionist/omniverse-mcp/blob/main/docs/ARCHITECTURE.md#the-bridge-token) where the file lives.

### Context7 for scripts

Models are not trained on Omniverse and Isaac Sim APIs, and guess their names. With [Context7](https://context7.com)
as a second MCP server, the assistant looks the API up (library `isaac-sim/isaacsim`) before writing `execute_script`
code; the server's instructions and the `script` skill ask it to.

## Troubleshooting

**"Missing or wrong bridge token"**: Kit runs as another user, in a container or on another machine. Set a token by
hand ([Kit on another machine](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#kit-on-another-machine)).

**"The Kit extension is older than this MCP server", or "speaks bridge protocol"**: the server and the bridge come
from different releases. Run the command the message names, then restart Kit.

**"Kit app connection failed", refused or timed out**: is the Kit app running, with the bridge on? Its console shows
`[ext: ovmcp.bridge-<version>] startup`. The server looks on port 8211 unless `OMNIVERSE_MCP_PORT` says otherwise. On
its first start Isaac Sim compiles shaders for several minutes, and answers slowly until then.

**"Kit at ... stopped answering"**: a call got no answer in 120 s, and `/health` none in 5 s either. Kit may be busy
with a long script that never awaits, or hung: Kit 110 (Isaac Sim 6.2) can hang for good inside NVIDIA's renderer
while it sets up a new camera view ([Known issues](#known-issues) has NVIDIA's workaround). Later calls fail in 5 s
while Kit stays silent, and go again once it answers. If it stays silent for minutes, restart the Kit app.

**Kit logs `[MCP Bridge] Failed to start server`**: another program, or a second Kit app, holds port 8211. Give this
Kit app another port, and the server the same one ([The bridge](#the-bridge)).

**The server doesn't start**: Claude Code's `/mcp` shows its error; other clients have an MCP log. Is `uvx` on the
PATH the client starts with? Restart the client after installing uv.

**A capture comes back black, or its prims do**: the answer warns, with the lighting mode. `viewport_light(action="set",
mode="Default")` lights the view, or `mode="camera"` for a light that follows the camera; Kit apps have no default
light.

**A tool returns an error**: `get_logs(min_level="warn")` shows what Kit reported. `get_robot_info` reads joint
positions while physics plays or is paused, and `raycast` needs it to have played once; the robot tools need Isaac Sim.

More in [SETUP.md](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#troubleshooting).

## Known issues

- **Isaac Sim 6.2 (Kit 110) can hang for good while it sets up a new camera view,** inside NVIDIA's renderer, its log
  silent from then on, and tools fail with "Kit at ... stopped answering"
  ([#114](https://github.com/serionist/omniverse-mcp/issues/114)). The bridge keeps one render between captures, and
  records through it at its size, so it sets up few new views, but a long session can still meet it. NVIDIA's
  workaround is a flag on Kit's command line, `--/rtx/hydra/supportMultiTickRate=false`
  ([isaac-sim/IsaacSim#920](https://github.com/isaac-sim/IsaacSim/issues/920); where it goes, with Kit's window or
  without: [Headless Kit](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md#headless-kit)): NVIDIA's repro ran 1,000 new views twice with it, and hung
  after 600 without. Its cost: repeated captures of one view differ a little more. Two captures of Replicator's
  walking agents, with the timeline stopped, came a mean of about 3.2 of 255 a channel apart with the flag, against
  about 1.2 without, in our tests. NVIDIA hasn't said yet what it does to sensors that render at different rates.
  Kit's renderer reads the flag only as it starts: set from a script while Kit runs, it doesn't reconfigure the
  renderer, and it misleads Replicator's `step()`, which reads it on each call
  ([#189](https://github.com/serionist/omniverse-mcp/issues/189)).
- **Isaac Sim 6.2 (Kit 110.4) can stop answering after a long session of stage changes,** its main thread busy inside
  Fabric's population code (`usdrt.population.plugin`), with none of the bridge's code running: seen in a test order
  that opens many stages, plays physics with colliders and captures with segmentation, never in a short session.
  Restart Kit; nothing of the bridge's can prevent it ([#162](https://github.com/serionist/omniverse-mcp/issues/162)).
  NVIDIA's flag for the new-view hang, above, avoided this one too in that order: 2 runs of 2 passed with it, where
  Kit hung 9 times of 9 without.

Report another as an [issue](https://github.com/serionist/omniverse-mcp/issues). When a tool misbehaves, your
assistant asks whether to report it, then drafts the report with the versions, the call and Kit's log, and files it
with the GitHub CLI once you agree to the draft, or leaves it for you to post (the plugin's
[report-bug](https://github.com/serionist/omniverse-mcp/blob/main/plugin/skills/report-bug/SKILL.md) skill).

## Security

The bridge runs any Python the server sends, by design: `execute_script` and `run_script_job` are tools. The token
keeps other programs and web pages out, but not your own AI client, so keep it asking before `execute_script`,
`run_script_job` and `manage_extensions`.
Report a vulnerability privately, as [SECURITY.md](https://github.com/serionist/omniverse-mcp/blob/main/SECURITY.md) says.

## Documentation

- [Setup in depth](https://github.com/serionist/omniverse-mcp/blob/main/docs/SETUP.md): each client's config, headless Kit, ports, remote setups, uninstalling
- [Tool reference](https://github.com/serionist/omniverse-mcp/blob/main/docs/TOOLS.md): every tool, its parameters and hints
- [The plugin](https://github.com/serionist/omniverse-mcp/blob/main/plugin/README.md): what the Claude Code plugin brings, and its skills
- [Without the plugin](https://github.com/serionist/omniverse-mcp/blob/main/docs/AI_GUIDE.md): the context and workflows for other clients
- [Architecture](https://github.com/serionist/omniverse-mcp/blob/main/docs/ARCHITECTURE.md): how the server and the bridge work, for contributors
- [Changelog](https://github.com/serionist/omniverse-mcp/blob/main/CHANGELOG.md), [Security](https://github.com/serionist/omniverse-mcp/blob/main/SECURITY.md), [Contributing](https://github.com/serionist/omniverse-mcp/blob/main/CONTRIBUTING.md)

Until 1.0, a minor release can change the tools; the changelog lists it under Breaking. Questions, ideas and bugs:
[open an issue](https://github.com/serionist/omniverse-mcp/issues/new/choose).

## Licence

[MIT](https://github.com/serionist/omniverse-mcp/blob/main/LICENSE). Not affiliated with or endorsed by NVIDIA; Omniverse and Isaac Sim are NVIDIA's trademarks.
