Metadata-Version: 2.4
Name: videosdk-teleop
Version: 0.1.2
Summary: Robot teleoperation over WebRTC: frame-accurate observation/action correlation, a non-bypassable safety envelope, and recording that exports to any dataset format
Author: VideoSDK
License-Expression: Apache-2.0
Project-URL: Homepage, https://docs.videosdk.live
Project-URL: Documentation, https://docs.videosdk.live
Keywords: robotics,teleoperation,webrtc,ros2,lerobot,so-101,imitation-learning,robot-learning,dataset
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Networking
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: videosdk>=0.3.8
Requires-Dist: vsaiortc>=0.0.17
Requires-Dist: numpy
Requires-Dist: av
Requires-Dist: pillow
Provides-Extra: lerobot
Requires-Dist: lerobot[feetech]==0.6.1; extra == "lerobot"
Provides-Extra: ros2
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Dynamic: license-file

# videosdk-teleop

**Drive a robot arm over the internet, and know exactly what the operator was
looking at when they moved it.**

A WebRTC transport for robot teleoperation, with frame-accurate
observation/action correlation, a safety envelope that cannot be bypassed, and
a recorder that turns every session into training data.

![Torq architecture: physical robots behind follower adapters, a VideoSDK room carrying synced action and video, and leader adapters in front of a human operator, VR headset, simulation or AI agent](https://cdn.videosdk.live/teleop/sdk-overview.webp)

---

## Install

```bash
pip install videosdk-teleop               # transport + cameras, ready to run
pip install "videosdk-teleop[lerobot]"    # + the lerobot adapters
pip install "videosdk-teleop[ros2]"       # + the ROS 2 nodes
```

Python **3.10-3.12**. The first line is enough to teleoperate: the WebRTC
transport and the camera stack are core dependencies, not extras.

## Your robot, in three methods

Everything else is inherited: lease negotiation, clamping, the watchdog, the
e-stop latch, correlation, recording.

```python
from videosdk_teleop import Follower, SafetyConfig, single_group_descriptor

class MyArm(Follower):
    def descriptor(self):            return single_group_descriptor(...)
    def read_joints(self):           return {"shoulder": 12.4, "elbow": -3.1}
    def write_joints(self, joints):  ...        # RAISE on failure

MyArm("abcd-efgh-ijkl", safety=SafetyConfig(slew=12.0)).run(hz=50)
```

Already on lerobot? Wrap the device instead of subclassing:

```python
from videosdk_teleop.adapters.lerobot import (SO101FollowerAdapter,
                                              SO101LeaderAdapter)

follower = SO101FollowerAdapter("abcd-efgh-ijkl", robot=arm, token=TOKEN)
```

On ROS 2, `teleop_bridge_node` does the same against your driver topics.

Meeting ids come from the VideoSDK API, or `videosdk_teleop.rooms.create_room()`.

## Run it locally, no robot needed

Runnable examples live in
[videosdk-teleops-examples](https://github.com/videosdk-live/videosdk-teleops-examples):

```bash
git clone https://github.com/videosdk-live/videosdk-teleops-examples
cd videosdk-teleops-examples

python quickstart/remote_teleop.py --ticks 100   # full chain, in-process
python quickstart/my_arm.py                      # port your own arm
python quickstart/protocol_demo.py               # the wire format, live
```

`remote_teleop.py` runs a real follower against a real leader: real gates,
real clamps, real observation log. Only the network is swapped for a loopback.

## You own the loop

```python
leader.tick()                 # one control period, never blocks
leader.run(hz=50)             # or let it pace the loop for you
```

```python
for _ in leader.ticks(hz=50):
    obs = leader.observation()          # joints + pixels, same instant
    print(obs.observation_id, obs.joints, obs.images)

    frame = leader.peek_synced()        # watch only: does not arm the echo
```

## Safety

```python
SafetyConfig(
    slew=12.0,                          # max change per joint per tick
    watchdog_timeout_s=0.5,             # silence for this long -> failsafe
    max_staleness_ms=100.0,             # older commands are dropped
    require_deadman=True,
    on_starvation=FailsafeAction.HOLD,
    resume_requires_reclaim=False)
```

```python
follower.estop()          # unauthenticated by design: anyone can stop an arm
follower.clear_estop()    # latched: only a human on the follower host resumes
```

Every command is checked before it reaches the motors: who sent it, whether it
arrived too late, and how far it moves the arm. If the operator stops sending,
the arm holds where it is.

## Knowing what the operator saw

Every action is tagged with the exact camera frame the operator was looking at
when they sent it. The tag travels with the command, so the two machines never
need their clocks in sync.

## Recording

```python
follower.start_recording("./sessions")
follower.start_episode("pick up the cube")
...                                          # your loop runs
follower.end_episode(success=True)
follower.stop_recording()
```

On the follower this captures pixels before an encoder or a network touched
them. Episodes open and close on the deadman by default.

VideoSDK can also record the session in the cloud:

```python
follower = MyArm("abcd-efgh-ijkl", cloud_recording=True)

follower.cloud_recording_state    # idle / requested / starting / ready / failed
```

If the cloud recorder never comes up, recording carries on locally and the
state says so.

## Watching a session

```python
from videosdk_teleop import attach

attach(follower, host="0.0.0.0", port=8080)   # live page + /snapshot.json
```

## On ROS 2

One node, `teleop_bridge_node`, one role per host. It never talks to your
motors directly: it reads your driver's joint-state topic and writes to your
driver's joint-command topic, exactly like any other ROS node.

```bash
ros2 launch videosdk_teleop_ros bridge.launch.py \
    role:=follower_side robot_id:=my_arm cameras:=cam_a,cam_b labels:=wrist,front

ros2 service call /videosdk_bridge/enable std_srvs/srv/SetBool "{data: true}"
```

The bridge sits idle until `enable` is called, so launching it moves nothing.

Install into the same Python that ROS 2 uses, not whatever `python3` your shell
resolves to:

```bash
/usr/bin/python3 -m pip install --user videosdk-teleop
```

---

## Documentation

Full documentation at **[docs.videosdk.live](https://docs.videosdk.live)**.

---

Apache-2.0 · [docs.videosdk.live](https://docs.videosdk.live) · [examples](https://github.com/videosdk-live/videosdk-teleops-examples)
