Metadata-Version: 2.4
Name: unreal-llm-sandbox
Version: 0.0.2
Summary: LLM Sandbox For Unreal Engine
Home-page: https://github.com/NeuralVFX/unreal-llm-sandbox
Author: NeuralVFX
Author-email: neuralvfx@gmail.com
License: MIT License
Keywords: nbdev jupyter notebook python
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastcore
Requires-Dist: requests
Requires-Dist: litellm
Requires-Dist: lisette
Requires-Dist: python-fasthtml
Requires-Dist: mistune
Requires-Dist: pillow
Requires-Dist: starlette
Requires-Dist: ddgs
Provides-Extra: dev
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Unreal LLM Sandbox


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
<p align="center">
  <img src="https://raw.githubusercontent.com/NeuralVFX/unreal-llm-sandbox-plugin/main/assets/demo.gif" alt="Demo">
</p>

#### This project has two components:
- Web-Interface: [unreal-llm-sandbox](https://github.com/NeuralVFX/unreal-llm-sandbox)  <--- You are here
- Unreal Plugin: [unreal-llm-sandbox-plugin](https://github.com/NeuralVFX/unreal-llm-sandbox-plugin)
  
# What is this?

An Unreal Engine plugin that exposes a Python execution server, enabling:
- A web-based coding interface connected directly to Unreal
- LLM-assisted code review with full context of your code and errors
- Creation and deployment of agentic tools that operate inside the engine

## Features
- **Code Execution** - All Code is executed directly in Unreal Engine
- **LLM Execution** - Ask an LLM for help, with your code/errors in context
- **Agentic Tool Use** - LLM can use tools directly in Unreal Engine
- **User Tools** - Build and register custom agentic tools instantly

# Installation


### Clone and Install
Create a new `conda` env, then:
```
git clone https://github.com/NeuralVFX/unreal-llm-sandbox 
cd unreal-llm-sandbox
pip install -e .
```
### Set your `OPENAI_API_KEY`
In Windows, open `Powershell` and then:
```
setx OPENAI_API_KEY *Your_api_key*
```
In Linux, I bet you'll figure it out...

# Unreal Server

Follow instructions at **[unreal-llm-sandbox-unreal](https://github.com/NeuralVFX/unreal-llm-sandbox-unreal)** to install Unreal side

# Usage

### Starting the Server

From the menu bar:
`LLM Sandbox → Start Server`
- The server runs at `http://127.0.0.1:8765`
    
### Starting the Web Interface
- Start `unreal-llm-sandbox` from command line ( outside of Unreal )
- Open `http://localhost:5001/notebook/notebook.ipynb` ( or any `ipynb` name )
- If the notebook doesn't exist, a blank one is created
- Notebooks auto-save every 2 seconds
- Standard `.ipynb` format, compatible with Jupyter


The web app provides:
- **Notebook Interface** - Jupyter-style interface
- `Code Cells` - Write and executing Python code in Unreal
- `Markdown Cells` - Write notes in Markdown
- `LLM Prompt Cells` - Chat with LLMs that have full context of your notebook + agentic control of Unreal

# Registering Custom Agent Tools

Syntax to register a new agentic tool:
```python

@register_tool
def spawn_cube(location_x: float, location_y: float, location_z: float):
    """Spawn a cube at the specified world location.
    
    Args:
        location_x: X coordinate
        location_y: Y coordinate  
        location_z: Z coordinate
    
    Returns:
        Name of the spawned actor
    """
    import unreal
    # Your Unreal Python code here
    ...
```
#### Either:
- Run this in a `Code Cell`
- Or create a new python file in your project's `Content/Python/tools` directory
#### Tool Discovery:
- Tools registered in `Code Cells` are instantly availible to the LLM
- Tools added to `Content/Python/tools` are discovered on project restart
#### To use:
- Open a `Prompt Cell`, Click the 🛠️ icon to activate Unreal tools, and write a prompt!

# Requirements

- Unreal Engine 5.6
- A conda env for the web-server
- An Open AI API key

