Metadata-Version: 2.4
Name: irtsp
Version: 0.1.1
Summary: Friendly Python client for iRTSP — typed, time-synced iPhone camera + IMU / GPS / LiDAR-depth / ARKit-pose streams
Project-URL: Homepage, https://github.com/ryanrudes/irtsp-python
Project-URL: Documentation, https://github.com/ryanrudes/irtsp-support/blob/main/INTEGRATION.md
Project-URL: Support, https://ryanrudes.github.io/irtsp-support/
Project-URL: Source, https://github.com/ryanrudes/irtsp-python
Author-email: Ryan Rudes <ryanrudes@gmail.com>
License: MIT
License-File: LICENSE
Keywords: arkit,camera,imu,ios,iphone,lidar,robotics,rtsp,sensors,slam,vio,visual-inertial-odometry
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Multimedia :: Video :: Capture
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: all
Requires-Dist: av>=10; extra == 'all'
Requires-Dist: numpy>=1.22; extra == 'all'
Requires-Dist: zeroconf>=0.39; extra == 'all'
Provides-Extra: discovery
Requires-Dist: zeroconf>=0.39; extra == 'discovery'
Provides-Extra: numpy
Requires-Dist: numpy>=1.22; extra == 'numpy'
Provides-Extra: video
Requires-Dist: av>=10; extra == 'video'
Requires-Dist: numpy>=1.22; extra == 'video'
Description-Content-Type: text/markdown

# irtsp

**Typed, SI-unit, one-shared-clock sensor streams from an iPhone.**

[iRTSP](https://ryanrudes.github.io/irtsp-support/) turns an iPhone into a streaming
sensor rig: RTSP video plus side channels for fused IMU, GNSS, compass heading,
barometric altitude, ARKit 6-DOF pose, camera intrinsics, and LiDAR metric depth.
This library is the Python client. Every sample arrives as a small, frozen,
pattern-matchable dataclass in SI units, carrying two timestamps on a **single clock
anchor captured once per session** — the same anchor that drives the video's RTP/RTCP
timeline — so video and odometry align with **no time offset to estimate**. The core
is pure stdlib, Python ≥ 3.10.

## Install

| Command | What you get |
|---|---|
| `pip install irtsp` | The core client — odometry + depth channels, all record types. Zero dependencies. |
| `pip install "irtsp[numpy]"` | `DepthFrame.meters` arrays and `point_cloud()` back-projection. |
| `pip install "irtsp[discovery]"` | `irtsp.discover()` — find phones over Bonjour/mDNS (zeroconf). |
| `pip install "irtsp[video]"` | Decoded video frames + `synced()` bundles (PyAV + numpy). **Experimental.** |
| `pip install "irtsp[all]"` | Everything above. |

## Quickstart

```python
import irtsp

with irtsp.connect("192.168.1.24") as phone:   # or "ryans-iphone.local"
    for imu in phone.imu:
        print(imu.gyro, imu.accel)             # rad/s, m/s²
```

Start streaming in the iRTSP app, point `connect()` at the phone's address, iterate.
Records are regular frozen dataclasses — unpack them, stick them in a queue, feed them
to your filter.

## Pattern-match the whole odometry channel

`phone.odometry` yields every record in arrival order (including depth frames when the
depth channel is open), and every record type supports structural pattern matching:

```python
import irtsp

with irtsp.connect("192.168.1.24", depth=True) as phone:
    for rec in phone.odometry:
        match rec:
            case irtsp.IMU(gyro=g, accel=a):
                propagate(g, a, rec.host_ts)
            case irtsp.Pose(position=p, tracking=irtsp.Tracking.NORMAL):
                update(p, rec.unix_ts)
            case irtsp.GNSS(lat=lat, lon=lon) as fix if fix.h_accuracy is not None:
                fuse_gps(lat, lon, fix.h_accuracy)
            case irtsp.DepthFrame() as d:
                enqueue_depth(d)
```

Unknown record types from a newer app version arrive as `irtsp.Unknown` (with the raw
payload bytes) instead of raising — old clients keep working.

## The streams

Each property on the session (`phone.imu`, `phone.gnss`, …) is a **fresh, independent
iterator** with its own buffer — create as many as you like, they don't compete.

| Record | Stream | Rate | SI fields | Wire-native view |
|---|---|---|---|---|
| `IMU` | `phone.imu` | ≤ ~100 Hz | `gyro` rad/s · `accel` m/s² (gravity **included**; face-up at rest ≈ (0, 0, −9.81)) · `quat` (x, y, z, w), body→world | `accel_g` |
| `RawGyro` / `RawAccel` | `phone.raw_gyro` / `phone.raw_accel` | raw sensor mode | rad/s · m/s² | `accel_g` |
| `Intrinsics` | `phone.intrinsics` | on zoom/lens change | `fx fy cx cy` px at `width×height` · `matrix` (3×3 K) · `scaled()` | — |
| `GNSS` | `phone.gnss` | ~1 Hz | `lat`/`lon` deg · `altitude` m · `speed` m/s · `h_accuracy`/`v_accuracy` m · `course_deg` | `course_rad` |
| `Altitude` | `phone.altitude` | ~1 Hz | `relative_altitude` m · `pressure` **Pa** | `pressure_kpa`, `pressure_hpa` |
| `Heading` | `phone.heading` | event-driven | `true_deg` · `magnetic_deg` · `accuracy_deg` | `true_rad`, `magnetic_rad` |
| `Pose` | `phone.pose` | ~60 Hz (AR mode) | `position` m (gravity-aligned world frame) · `orientation` quat · `tracking` | — |
| `DepthFrame` | `phone.depth` | ≤ 30 Hz | half-float **meters**; `meters` (ndarray), `at(x, y)`, `point_cloud(K)` | — |

Unit conventions, in one breath: SI everywhere by default — the wire's g-units become
m/s² (× 9.80665) and its kPa becomes Pa at decode time, with the wire-native value
always one property away. Angles conventionally spoken in degrees (lat/lon, compass,
GNSS course) stay degrees under explicit `*_deg` names, with `*_rad` properties.
Fields CoreLocation marks invalid (negative on the wire) decode to `None`, and an
attitude-off session's zeroed quaternion slots decode to `quat=None` — no sentinel
values ever reach your code.

Every record also carries `host_ts`, `unix_ts`, `seq`, `gap`, and a `time` property
(`unix_ts` as an aware UTC `datetime`). More on the two clocks below.

## Discovery

```python
import irtsp                       # pip install "irtsp[discovery]"

for device in irtsp.discover():
    print(device.name, device.host, device.ports)
    # iPhone 192.168.1.24 {'video': 8554, 'imu': 8555, 'depth': 8556}

phone = irtsp.discover()[0].connect(depth=True)
```

Only devices advertising the iRTSP odometry service (`_irtsp-imu._tcp`) are returned,
so a random RTSP camera on your network won't show up. No zeroconf installed? Just
connect by IP.

## Callbacks and `run()`

If you'd rather push than pull:

```python
import irtsp

phone = irtsp.connect("192.168.1.24", reconnect=True)
phone.on(irtsp.GNSS, lambda fix: print(f"{fix.lat:.6f}, {fix.lon:.6f}"))
phone.on((irtsp.IMU, irtsp.Pose), sink.write)
phone.run()   # blocks until the session closes; Ctrl-C exits cleanly
```

Callbacks run on the reader thread, so keep them quick; a callback that raises is
logged and never kills the reader. `phone.latest(irtsp.Intrinsics, wait=2.0)` gives
you the most recent record of a type — handy for intrinsics, which the server replays
to late joiners on connect.

## asyncio

The async client is a native asyncio implementation (not a thread wrapper) with the
same shapes:

```python
import asyncio, irtsp

async def main():
    async with await irtsp.connect_async("192.168.1.24") as phone:
        async for imu in phone.imu:
            print(imu.gyro, imu.accel)

asyncio.run(main())
```

## Depth → numpy → point cloud

```python
import irtsp                       # pip install "irtsp[numpy]"

with irtsp.connect("192.168.1.24", depth=True) as phone:
    K = phone.latest(irtsp.Intrinsics, wait=2.0)      # replayed on connect

    for frame in phone.depth:
        depth = frame.meters                          # (H, W) float32, meters
        center = frame.at(frame.width // 2, frame.height // 2)  # stdlib, no numpy
        pts = frame.point_cloud(K, stride=4)          # (N, 3) float32, camera frame
```

`point_cloud()` accepts intrinsics at any resolution (the stream's intrinsics are for
the video) and rescales them to the depth map automatically. The camera frame is the
standard pinhole convention: +X right, +Y down, +Z forward. Non-finite depths are
dropped.

## Video + `synced()` — EXPERIMENTAL

With the `video` extra, the session can decode the RTSP video and hand you each frame
bundled with the odometry that belongs to it, all on one clock:

```python
import irtsp                       # pip install "irtsp[video]"

with irtsp.connect("192.168.1.24", video=True, depth=True) as phone:
    for f in phone.synced():
        f.image        # (H, W, 3) RGB uint8
        f.timestamp    # unix seconds — same axis as every record's unix_ts
        f.imu          # IMU records since the previous frame (pre-integration ready)
        f.pose         # Pose SLERP-interpolated at f.timestamp, or None
        f.depth        # nearest DepthFrame within tolerance, or None
        f.intrinsics   # latest camera matrix
        f.approx_clock # ← read the note below
```

**The honest note.** Frame times are exact only when FFmpeg exposes the RTCP
wall-clock anchor (`start_time_realtime`); iRTSP builds its RTCP Sender Reports from
the same anchor as the odometry, so when that value is available the alignment is as
good as the record timestamps. When FFmpeg *doesn't* expose it, the stream falls back
to anchoring the first frame at local receive time — alignment is then off by network
plus decode latency (typically a few tens of ms) and every frame is flagged
`approx_clock=True`. Check that flag before trusting tight sync, and if you need
guaranteed-exact video timing, consume the RTSP stream with your own RTCP-aware
pipeline as described in the
[integration guide](https://github.com/ryanrudes/irtsp-support/blob/main/INTEGRATION.md) §4.

## How the synchronization works

At the start of each streaming session the phone captures **one anchor pair** — a
host-clock reading (`mach_absolute_time`, seconds) and the Unix wall time at the same
instant — and *every* stream derives its timestamps from it. That's the whole trick:
the streams were never on different clocks, so there is no offset to estimate and
nothing to cross-correlate.

Every record carries both axes:

- **`host_ts`** — seconds on the phone's monotonic host clock. The same axis as the
  video/audio presentation timestamps, CoreMotion, ARKit, and the depth frames.
  Cleanest for intra-session alignment; not comparable across reboots.
- **`unix_ts`** — wall-clock seconds, `unix_ts = wall_anchor + (host_ts − host_anchor)`.
  The same axis as the video's RTCP Sender-Report NTP timeline, and comparable across
  machines.

The handshake ships both anchors; `phone.clock` is a `StreamClock` that converts
either way (`to_unix()` / `to_host()`). Because the anchor is frozen at session start,
no mid-session NTP adjustment will ever warp your timeline. The full derivation —
64-byte record layout, depth framing, RTP→wall-time math — is in the
[integration guide](https://github.com/ryanrudes/irtsp-support/blob/main/INTEGRATION.md).

## Reliability notes

- **Slow consumers can't stall capture.** Each iterator/callback subscription has its
  own bounded buffer (default 8192 records); a consumer that falls behind drops its
  *own* oldest records and never affects the reader or other consumers. The count is
  on `stream.dropped`.
- **Nothing is lost silently.** Each channel carries a wire sequence number; if
  records were lost upstream (or dropped before your consumer attached), the next
  record's `gap` field says how many went missing right before it (`gap=0` means
  none).
- **Reconnects.** By default a dropped connection closes the session (iterators end,
  `run()` returns). With `connect(..., reconnect=True)` the client redials with
  backoff and re-reads the handshake — picking up fresh clock anchors if the phone
  restarted its stream.
- **Bad bytes fail loudly.** Connecting to a non-iRTSP port or desyncing raises
  `irtsp.ProtocolError` rather than yielding garbage.

## Links

- iRTSP app + support: <https://ryanrudes.github.io/irtsp-support/>
- Wire protocol & synchronization spec: [INTEGRATION.md](https://github.com/ryanrudes/irtsp-support/blob/main/INTEGRATION.md)

## License

MIT — see [LICENSE](LICENSE).
