uni-robot-agent

A unified harness for robot use.

01

Agent

Environment context

Prompts & Tool descriptions

Tool contributions and history policies

Agent backends

Codex / Claude Code

Task reasoning, tool calls, native collaboration

Tool calls
02

Orchestration

MCP adapters

HTTP / Claude SDK

ToolExecutor

Serial / parallel execution
Resource exclusivity

Toolkit

ToolDef registration
Schema + handler

Unified call and result contract
03

Tools

One shared implementation

Shared Episode; arm and camera parameters
from environment descriptors.

GripperTools

Motion & grasping

move·grasp
release·holdactor_act

SceneTools

Scene perception

view_env_state
capture_scene·locate

SkillTools

Skill library

list_skills
read_skill
write_skill

EpisodeTools

Episode completion

finish
04

Services

Episode → EnvironmentClient

EnvironmentService

Step, state, snapshots,
execution receipts

GripperTools → ActionBinding

ActorService

ActorClient: action proposals
and batching

SceneTools → PerceptionClient

PerceptionService

Image segmentation
and batching

Backend contract Backend contract Backend contract
05

Backends

EnvironmentBackend

RoboUse / RoboDojo adapters

Native simulation runtimes

Pi05Backend

OpenPI / JAX

RoboDojo Pi0.5

Sam3Backend

SAM3 inference

Text / point prompts; image cache