Metadata-Version: 2.4
Name: pyroboframes
Version: 1.3.1
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: numpy>=1.24
Requires-Dist: pyarrow>=14
Requires-Dist: pytest==8 ; extra == 'dev'
Requires-Dist: numpy==1.24 ; extra == 'dev'
Requires-Dist: maturin==1.7 ; extra == 'dev'
Requires-Dist: mlx==0.20 ; extra == 'mlx'
Provides-Extra: dev
Provides-Extra: mlx
License-File: LICENSE
Summary: High-performance dataloader for robot learning datasets. LeRobot format support, hardware-accelerated video decode via VideoToolbox, MLX array output. Zero-copy data pipeline optimized for Apple Silicon and edge robotics.
Keywords: robotics,robot-learning,ml-dataloader,lerobot,dataset-loader,computer-vision,video-processing,mlx,pytorch,jax,numpy,data-pipeline,machine-learning,reinforcement-learning,edge-ai,apple-silicon,video-decode,zero-copy,deep-learning,neural-networks,training-framework,data-preprocessing
Author-email: Georgi Mammen Mullassery <mullassery@gmail.com>
Maintainer-email: Georgi Mammen Mullassery <mullassery@gmail.com>
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Bug Tracker, https://github.com/Mullassery/PyRoboFrames/issues
Project-URL: Changelog, https://github.com/Mullassery/PyRoboFrames/releases
Project-URL: Discussions, https://github.com/Mullassery/PyRoboFrames/discussions
Project-URL: Documentation, https://github.com/Mullassery/PyRoboFrames#readme
Project-URL: Homepage, https://github.com/Mullassery/PyRoboFrames
Project-URL: Repository, https://github.com/Mullassery/PyRoboFrames
Project-URL: Source Code, https://github.com/Mullassery/PyRoboFrames/tree/main

# PyRoboFrames

[![PyPI](https://img.shields.io/pypi/v/pyroboframes)](https://pypi.org/project/pyroboframes/)
[![Python](https://img.shields.io/pypi/pyversions/pyroboframes)](https://pypi.org/project/pyroboframes/)
[![License: Proprietary](https://img.shields.io/badge/License-Proprietary-red.svg)](./LICENSE)
[![Tests](https://img.shields.io/badge/tests-245%20passing-brightgreen)]()
![Status: v1.3 Production Ready](https://img.shields.io/badge/Status-v1.3%20Production%20Ready-brightgreen)

**Intelligent data pipeline for robot learning. Zero-copy, quality-aware loading with hardware acceleration and dataset composition tracking.**

PyRoboFrames is a **foundation library for robot learning intelligence** — load any robot learning dataset with quality validation, accelerate video decode with hardware (VideoToolbox/NVDEC), track data provenance, and stream to NumPy/MLX/PyTorch/JAX. The heavy lifting runs in a Rust engine; Python is the ergonomic surface.

**Architectural Role:** Owns robot learning data pipelines. Loads, validates, and optimizes training data with quality metadata and data composition tracking. Quality metadata flows downstream to training systems.

**For autonomous driving perception and foundation models, see [PyRoboVision](https://github.com/Mullassery/PyRoboVision).**

## Why Star This?

- **10 speedup without code changes** — Hardware-accelerated video decode (VideoToolbox on macOS, NVDEC on NVIDIA)
- **Zero-copy data transfer** — Stream directly to MLX, PyTorch, or JAX without intermediate copies
- **Quality validation built-in** — Detect corrupted frames, missing episodes, and data drift before training
- **Multi-dataset composition** — Mix LeRobot, HDF5, NetCDF, and cloud datasets in one training loop

## Why PyRoboFrames?

| Feature | PyRoboFrames | PyTorch DataLoader | Standard approach |
|---------|---------|---------|---------|
| **Video decode speed** | 10 (HW-accelerated) | Baseline | Baseline |
| **Memory (1000 frames)** | ~100MB (streaming) | ~500MB+ (loaded) | ~500MB+ (loaded) |
| **Zero-copy transfer** | Direct to MLX/torch | Copy required | Copy required |
| **Quality validation** | Built-in | Manual | Manual |
| **Dataset support** | 5+ formats | Custom code | DIY |
| **Multi-dataset mixing** | Easy composition | Manual | DIY |
| **Episode prefetch** | Built-in optimization | Manual | None |
| **S3/GCS support** | Native streaming | fsspec wrapper | DIY |
| **Setup time** | Minutes | Hours | Days |

---

## Installation

```bash
# Latest stable (v1.3.0)
pip install pyroboframes

# Or with uv
uv add pyroboframes

# Specific version
pip install pyroboframes==1.3.0
```

**Latest:** v1.3.0 (2026-07-26) — Advanced I/O, memory efficiency, sensor fusion, cross-platform training 
**Requires:** Python  3.10 
**Prebuilt wheels:** macOS (Apple Silicon), Linux (x86_64) 
**From source:** Rust 1.78+ required

Optional extras (install as needed):

```bash
pip install pyroboframes h5py # HDF5 datasets
pip install pyroboframes xarray netCDF4 # NetCDF datasets
pip install pyroboframes tensorflow-datasets # RLDS / Open X-Embodiment
pip install pyroboframes fsspec s3fs # S3 remote streaming
pip install pyroboframes fsspec gcsfs # GCS remote streaming
pip install pyroboframes ray # Ray distributed loading
```

---

## Quick Start

### Load LeRobot Datasets

```python
import pyroboframes as prf

ds = prf.RoboFrameDataset.from_path("/path/to/lerobot_dataset")

loader = ds.loader(
 batch_size=64,
 cameras=["observation.images.top"],
 output="torch", # or "mlx", "numpy", "jax"
 num_workers=4,
 cache_size=4096, # LRU frame cache (frames)
 episode_prefetch=True,
)

for batch in loader:
 state = batch["observation.state"] # [64, state_dim]
 frames = batch["observation.images.top"] # [64, H, W, 3]
 action = batch["action"] # [64, action_dim]
```

### Proprioceptive-Only (No Video) for 10 Speedup

```python
loader = prf.ProprioceptiveLoader(
 dataset_path="/path/to/lerobot_dataset",
 batch_size=256,
 device="mlx",
)

for batch in loader:
 state = batch["state"] # [256, state_dim]
 action = batch["action"] # [256, action_dim]
```

### Temporal Windows for Sequence Models

```python
loader = ds.loader(
 batch_size=32,
 chunk_size=16,
 delta_timestamps={"observation.state": [-0.2, -0.1, 0.0]},
 output="mlx",
)
for batch in loader:
 seq = batch["observation.state"] # [32, 3, state_dim]
```

---

## What's New in v1.1

### Video Codec Selection — 40–50% Storage Savings

```python
# Write with HEVC (H.265) instead of the H.264 default
prf.write_lerobot_dataset(
 path="/out/dataset",
 features={"observation.state": state_arr, "action": action_arr},
 episode_lengths=[500, 500],
 video_codec="hevc", # "h264" | "hevc" | "av1"
 video_crf=23, # lower = better quality, larger file
)

# Standalone video encoding
prf.encode_video_frames(frames, "output.mp4", codec="av1", crf=30)
```

### Data Validation Toolkit

```python
from pyroboframes import DatasetValidator

validator = DatasetValidator(
 ds,
 check_frames=True, # frame count vs. metadata
 check_temporal=True, # timestamp gap detection
 check_codec=True, # sample-decode health check
 sample_rate=0.1, # probe 10% of episodes
)
report = validator.validate()
print(report.summary())
report.raise_if_errors()
```

### Episode-Level Caching for Repeated Epochs

```python
from pyroboframes import EpisodeCache

cache = EpisodeCache(ds, max_episodes=8)

for epoch in range(10):
 for ep_idx in range(ds.num_episodes()):
 ep = cache.get_episode(ep_idx) # decoded once, cached after
 states = ep["observation.state"] # [T, D]

cache.prefetch([0, 1, 2, 3]) # background pre-decode
```

### Cross-Dataset Quality Comparison

```python
from pyroboframes import EpisodeScorer, DatasetQualityProfile, CrossDatasetComparator

scorer = EpisodeScorer()
profile_a = DatasetQualityProfile.from_scores("dataset_a", scorer.score_episodes(df_a))
profile_b = DatasetQualityProfile.from_scores("dataset_b", scorer.score_episodes(df_b))

comparator = CrossDatasetComparator(reference=profile_a)
print(comparator.compare(profile_b)) # Cohen's d, percentile overlap
print(comparator.recommend_mixing_ratio(profile_b)) # curriculum mixing weight
```

### HDF5 / NetCDF / RLDS Format Support

```python
# HDF5 (ROBOMIMIC, ACT, custom) — pip install h5py
from pyroboframes import HDF5Dataset, convert_hdf5
convert_hdf5("robomimic.hdf5", "/out/lerobot")

# NetCDF (scientific/simulation datasets) — pip install xarray netCDF4
from pyroboframes import NetCDFDataset, convert_netcdf
convert_netcdf("sim_data.nc", "/out/lerobot", episode_breaks=[0, 500, 1200])

# RLDS / Open X-Embodiment — pip install tensorflow-datasets
from pyroboframes import RLDSDataset, convert_rlds
convert_rlds("fractal20220817_data", "/out/lerobot", split="train")
```

### Remote S3/GCS Streaming + Ray Distributed Loading

```python
# Stream from S3
from pyroboframes import RemoteDataset
ds = RemoteDataset.from_s3("s3://my-bucket/lerobot_dataset").open()
ds.prefetch_episodes([0, 1, 2, 3]) # background download
loader = ds.loader(batch_size=32)

# Ray distributed — pip install ray
from pyroboframes import RayDistributedLoader, shard_episodes
loader = RayDistributedLoader(
 "/path/to/dataset", num_workers=4, rank=0, world_size=4, batch_size=32
)

# Or just shard episodes yourself
my_episodes = shard_episodes(total_episodes=200, world_size=4, rank=0)
#  [0, 4, 8, , 196]
```

---

## What's New in v1.2

### GPU-Accelerated Image Transforms

Transform frames on NVIDIA (CuPy), Apple Silicon (MLX), or CPU with automatic fallback:

```python
from pyroboframes.gpu_acceleration import GPUTransforms

transforms = GPUTransforms(device="auto") # Picks best available: cuda  mlx  cpu

# Resize + normalize on GPU
resized = transforms.resize(frame, size=(224, 224), interpolation="bilinear")
normalized = transforms.normalize(
 resized,
 mean=[0.485, 0.456, 0.406],
 std=[0.229, 0.224, 0.225]
)
```

### Temporal Consistency for Video Stitching

Smooth stitched panoramas and reduce flickering with optical flow and temporal filtering:

```python
from pyroboframes.gpu_acceleration import OpticalFlowEstimator, TemporalFilter

# Optical flow for seam tracking
flow = OpticalFlowEstimator.estimate_lucas_kanade(frame1, frame2)

# Temporal smoothing (exponential moving average)
frames = [frame1, frame2, frame3, ...]
smoothed = TemporalFilter.apply_temporal_smoothing(frames, alpha=0.7)

# Median filtering
denoised = TemporalFilter.apply_median_filter(frames, kernel_size=3)
```

### Real-World Autonomous Driving Datasets

Load Waymo, nuScenes, and KITTI with unified interface:

```python
from pyroboframes.dataset_loaders import (
 WaymoDatasetLoader,
 nuScenesDatasetLoader,
 KITTIDatasetLoader,
)

# Waymo Open Dataset
waymo = WaymoDatasetLoader("/path/to/waymo")
image, metadata = waymo.get_frame(scene_idx=0, frame_idx=10, camera="FRONT")
print(f"Camera calibration: fx={metadata.calibration.fx}")

# nuScenes
nuscenes = nuScenesDatasetLoader("/path/to/nuscenes")
image, metadata = nuscenes.get_frame(scene_idx=0, frame_idx=10, camera="CAM_FRONT")

# KITTI
kitti = KITTIDatasetLoader("/path/to/kitti", split="training")
image, metadata = kitti.get_frame(seq_idx=0, frame_idx=10, camera=0)
```

### Occupancy Grid Mapping for 3D Perception

Convert point clouds and 3D bounding boxes to occupancy grids for path planning:

```python
from pyroboframes.occupancy_3d import OccupancyGrid, OccupancyGridConfig, LiDARProcessor

# Create occupancy grid
config = OccupancyGridConfig(size_x=100.0, size_y=100.0, resolution=0.1)
grid = OccupancyGrid(config)

# Add LiDAR point cloud
points = lidar_points[:, :2] # [N, 2] XY coordinates
grid.add_point_cloud(points)

# Add 3D bounding boxes
bbox = {"x": 0, "y": 0, "width": 2.0, "length": 4.0, "height": 2.0}
grid.add_bounding_box(bbox)

# Morphological operations for smoothing
grid.dilate(kernel_size=3)
grid.erode(kernel_size=5)

# Get results
free_space = grid.get_free_space_mask() # [H, W] binary mask
occupied_cells = grid.get_occupied_cells() # List of (x, y) cells
```

### LiDAR Processing & Radar Fusion

Process 3D point clouds and fuse with radar for velocity estimates:

```python
from pyroboframes.occupancy_3d import LiDARProcessor, RadarFusionProcessor

# Filter points
points = LiDARProcessor.filter_by_distance(lidar_points, max_distance=100.0)
points = LiDARProcessor.filter_by_height(points, min_height=-1.0, max_height=3.0)

# Ground segmentation
ground, non_ground = LiDARProcessor.ground_segmentation(points, threshold=0.1)

# Clustering
clusters = LiDARProcessor.cluster_points(points, distance_threshold=0.2, min_points=5)

# Compute normals for surface analysis
normals = LiDARProcessor.compute_normals(points, k=10)

# Radar-LiDAR fusion
radar_detections = [{"x": 0, "y": 0, "z": 0, "vx": 1.0, "vy": 0, "vz": 0}]
fused = RadarFusionProcessor.fuse_radar_lidar(
 lidar_points,
 radar_detections,
 distance_threshold=1.0
)
```

---

## What's New in v1.3

### Advanced I/O & Ecosystem Integration

**Parquet Export with Compression Options:**
```python
from pyroboframes import ParquetWriter, ParquetWriteOptions

writer = ParquetWriter(
    ParquetWriteOptions(
        compression="snappy",  # or gzip, brotli, zstd, lz4
        row_group_size=10000,
    )
)
paths = writer.write_dataframe(robotics_df, output_dir="./data")
```

**LeRobot Write-Back for Custom Datasets:**
```python
# Export aligned multi-sensor data back to LeRobot v3.0 format
from pyroboframes import write_from_robotics_dataframe

write_from_robotics_dataframe(
    dataframe=robotics_df,  # RoboticsDataFrame from MCAP/ROS2
    output_path="./custom_lerobot",
    reference_topic="/camera/rgb",  # For episode detection
    fps=30.0,
)
# Now train on your custom dataset like any LeRobot dataset
```

**Hugging Face Hub Direct Loading:**
```python
from pyroboframes import from_huggingface_hub

# Auto-detects LeRobot or multi-topic format
ds = from_huggingface_hub("lerobot/aloha_mobile_cabinet")
# or load any robotics dataset from HF Hub

loader = ds.loader(batch_size=32)
```

### Memory Efficiency & Synchronization

**Memory-Mapped Parquet for 100GB+ Datasets:**
```python
from pyroboframes import LazyParquetDataset

# Load huge files without loading into RAM
ds = LazyParquetDataset("/path/to/large.parquet")

# Slice rows [start:end) without loading full file
batch = ds.slice(start=10000, end=10064, columns=["state", "action"])

# Iterate in memory-efficient batches
for batch in ds.iter_batches(batch_size=256):
    # process batch
```

**Multi-Camera Frame Synchronization:**
```python
from pyroboframes import VideoSynchronizer, CameraTimeline

# Align camera streams with dropped frame handling
ref_timeline = CameraTimeline("cam_0", timestamps_ns, frame_indices)
sync = VideoSynchronizer(ref_timeline, window_size_ns=100_000_000)

other_timeline = CameraTimeline("cam_1", other_times, other_frames)
mapping = sync.sync_camera(other_timeline)
# mapping[ref_idx] → corresponding frame in other camera (-1 if missing)
```

**Jitter Filtering for Smooth Video:**
```python
from pyroboframes.video_sync import JitterFilter

jitter = JitterFilter(alpha=0.7)  # EMA smoothing
smooth_timestamps = jitter.filter_timestamps(raw_timestamps_ns)
```

### Multi-Rate Sensor Fusion

**Adaptive Sensor Rate Matching:**
```python
from pyroboframes import MultiRateFusionEngine

engine = MultiRateFusionEngine(reference_rate_hz=30.0)

# Detect rates from timestamps
rates = engine.detect_rates({
    "camera": camera_timestamps_ns,  # ~30Hz
    "imu": imu_timestamps_ns,  # ~100Hz
})

# Align all sensors to 30Hz reference
aligned = engine.align_to_reference(
    reference_topic_times=camera_times,
    sensor_data={
        "imu_accel": (imu_times, imu_accel_data),
    },
    strategy="linear"  # or "nearest_neighbor"
)
```

**Kalman Filtering for State Estimation:**
```python
# Smooth noisy IMU readings while preserving fast dynamics
filtered = engine.kalman_filter_state(
    measurements=noisy_gyro,
    process_variance=1e-5,
    measurement_variance=1e-2
)
```

**Weighted Multi-Sensor Fusion:**
```python
# Fuse redundant sensors with per-sensor confidence weights
fused = engine.weighted_fusion(
    sensor_readings={
        "encoder_pos": enc_position,
        "imu_accel": imu_accel,
    },
    weights={"encoder_pos": 0.7, "imu_accel": 0.3}
)
```

### Cross-Platform Training Parity

**Framework-Agnostic Tensor Adapter:**
```python
from pyroboframes import ToTensorAdapter, detect_best_framework

# Auto-detects best framework: CUDA → MLX → PyTorch → JAX → NumPy
adapter = ToTensorAdapter(framework="auto")
batch = adapter.batch_to_tensors(numpy_batch)  # Works on any device

# Or explicitly target device
adapter = ToTensorAdapter(framework="mlx", device="mps")

# Single training loop works on M3, RTX 5090, H100, CPU
framework = detect_best_framework()
print(f"Running on: {framework}")
```

**On-Device MLX Transforms (Apple Silicon):**
```python
from pyroboframes import MLXTransforms

mlx_t = MLXTransforms()
if mlx_t.is_available():
    # Keeps data in unified memory (no host copies)
    resized = mlx_t.resize(image_mx, size=(224, 224))
    normalized = mlx_t.normalize(resized, mean=[0.5]*3, std=[0.5]*3)
    
    # Compose transforms
    result = mlx_t.compose(image_mx, [
        {"op": "normalize", "args": {"mean": [0.5]*3, "std": [0.5]*3}},
        {"op": "resize", "args": {"size": (224, 224)}},
    ])
```

**On-Device MPS Transforms (PyTorch):**
```python
from pyroboframes import MPSTransforms

mps_t = MPSTransforms()
if mps_t.is_available():
    # Same API as MLX for cross-platform scripts
    resized = mps_t.resize(image_torch, size=(224, 224), interpolation="bilinear")
    normalized = mps_t.normalize(resized)
```

---

## Full Feature Table

| Feature | Status | Notes |
|---|---|---|
| **LeRobot v3.0 loading** | | Full schema support |
| **Video decode** | | FFmpeg + VideoToolbox + NVDEC |
| **Proprioceptive loader** | | 10 speedup (no video) |
| **Temporal windows** | | Multi-timestep sequences |
| **Multi-camera batching** | | Arbitrary camera combinations |
| **Output formats** | | NumPy, MLX, PyTorch, JAX |
| **Parallel prefetch** | | num_workers for async loading |
| **Data augmentation** | | Rotate, flip, crop, color jitter |
| **Video codec selection** | | H.264 / HEVC / AV1 + CRF control |
| **Dataset validation** | | Temporal gaps, missing frames, codec health |
| **Episode caching** | | RAM-based LRU cache, background prefetch |
| **MCAP ingestion** | | JSON, protobuf, CDR support |
| **ROS 2 bag ingestion** | | .db3 native format |
| **HDF5 ingestion** | | ROBOMIMIC, ACT, custom layouts |
| **NetCDF ingestion** | | Scientific/simulation datasets |
| **RLDS / Open X-Embodiment** | | tensorflow-datasets integration |
| **Episode quality scoring** | | Diversity, sharpness, state variance |
| **Cross-dataset comparison** | | Cohen's d, percentile ranking, mixing ratio |
| **S3/GCS streaming** | | fsspec-backed remote datasets |
| **Ray distributed loading** | | Episode sharding across Ray workers |
| **Streaming ingestion** | | Kafka, MQTT real-time data |
| **Distributed loading** | | Multi-GPU synchronized sampling |
| **Parquet export** | ✨ v1.3 | Configurable compression (snappy/gzip/brotli/zstd) |
| **LeRobot write-back** | ✨ v1.3 | Export fused data to LeRobot v3.0 format |
| **HuggingFace Hub loading** | ✨ v1.3 | Direct dataset import from HF Hub |
| **Memory-mapped Parquet** | ✨ v1.3 | Lazy loading for 100GB+ files |
| **Multi-camera sync** | ✨ v1.3 | Windowed frame alignment with jitter filtering |
| **Multi-rate sensor fusion** | ✨ v1.3 | Kalman filtering, weighted averaging |
| **Framework-agnostic tensors** | ✨ v1.3 | Auto-detect: CUDA → MLX → PyTorch → JAX |
| **On-device transforms (MLX/MPS)** | ✨ v1.3 | GPU-resident processing, zero-copy |

---

## Test Coverage: 245 Tests Passing 

```
Dataloader: 30 tests
Video decode: 25 tests
Proprioceptive: 16 tests
Augmentation: 15 tests
Temporal ops: 12 tests
Quality/scoring: 17 tests (+7 cross-dataset)
Validation: 13 tests
Caching: 5 tests
HDF5: 7 tests
NetCDF: 7 tests
Advanced I/O (P3-P6): 23 tests
  - Parquet I/O: 5 tests
  - Memory efficiency: 6 tests
  - Sensor fusion: 4 tests
  - Cross-platform: 7 tests
  - Integration: 1 test
Distributed: 8 tests
Streaming: 7 tests
Codecs: 7 tests (+3 round-trip)
GPU Acceleration: 10 tests (NEW v1.2)
Dataset Loaders: 16 tests (NEW v1.2)
Occupancy/3D: 32 tests (NEW v1.2)
Other: 6 tests
```

```bash
pytest tests/ -v
```

---

## How PyRoboFrames Compares

PyRoboFrames occupies a unique position in the robot learning dataloader ecosystem:

| Dimension | PyRoboFrames | torchcodec | Robo-DM | LeRobot |
|-----------|--------------|-----------|---------|----------|
| Multi-format support (LeRobot + RLDS + HDF5 + MCAP) | | | | |
| Apple Silicon native GPU | | | | |
| Waymo + nuScenes + KITTI loaders | | | | |
| 3D occupancy grids + sensor fusion | | | ? | |
| Unified GPU fallback chain | | | ? | |
| Production maturity | | | ? | |

**Best for:** Multi-lab robotics collaboration + autonomous driving integration + cross-platform training.

**See [COMPETITIVE_ANALYSIS.md](./docs/COMPETITIVE_ANALYSIS.md) for detailed comparison.**

---

## GPU Support

- **Apple Silicon**: VideoToolbox hardware decode, MLX zero-copy arrays
- **NVIDIA**: NVDEC hardware decode, PyTorch CUDA acceleration
- **CPU**: NumPy fallback (~10 slower than hardware)

```python
loader = ds.loader(device="auto", ...) # auto-detect
loader = ds.loader(device="mlx", ...) # Apple Silicon
loader = ds.loader(device="cuda", ...) # NVIDIA
loader = ds.loader(device="cpu", ...) # CPU
```

---

## Use Cases

- **LeRobot policy training** — Fast loading for imitation learning
- **Open X-Embodiment fine-tuning** — RLDS ingestion + LeRobot conversion
- **Large-scale cloud training** — S3/GCS streaming + Ray distribution
- **Multi-dataset curriculum** — Cross-dataset quality comparison + mixing ratios
- **Data quality auditing** — Validate integrity before long training runs
- **Legacy dataset migration** — HDF5/NetCDF  LeRobot conversion

---

## Performance

- **Video decode:** 100+ FPS (hardware-accelerated on macOS/CUDA)
- **Dataloader throughput:** 50–100 images/sec (PyTorch, Mac M3)
- **Proprioceptive loader:** 1,000+ batch/sec (no video decode)
- **Storage savings:** 40–50% with HEVC vs H.264 at equivalent quality

---

## Architecture

```
PyRoboFrames (Rust core + Python surface)

Input: LeRobot / HDF5 / NetCDF / RLDS / MCAP / ROS2 / S3 / GCS
 
Format Converters  LeRobot v3.0 (Parquet + MP4)
 
Rust Decoder (VideoToolbox / NVDEC / FFmpeg)
 
RoboFrameDataset (episode index, frame manifest)
 
Loader (temporal windows, augmentation, caching, batching)
 
Output: NumPy / MLX / PyTorch / JAX
 
Your training loop
```

### Module Organization

```
pyroboframes/
 RoboFrameDataset # Load LeRobot datasets
 ProprioceptiveLoader # State/action only (no video)
 DataLoader # Flexible batching + augmentation
 EpisodeCache # RAM-based episode LRU cache
 DatasetValidator # Deep data quality checks
 hdf5 # HDF5 reader + converter
 netcdf # NetCDF reader + converter
 rlds # RLDS / Open X-Embodiment reader
 distributed # RemoteDataset, RayDistributedLoader, shard_episodes
 quality # EpisodeScorer, CrossDatasetComparator
 backend/ # Device abstractions (MLX, PyTorch, JAX)
 transforms/ # Augmentation pipelines
 [streaming, sensor_fusion, depth_io, ...]
```

---

## Related Projects

- **[LeRobot](https://github.com/huggingface/lerobot)** — Robot learning datasets
- **[PyRoboVision](https://github.com/Mullassery/PyRoboVision)** — Autonomous driving perception + foundation models
- **[MLX](https://github.com/ml-explore/mlx)** — Apple Silicon ML framework
- **[Open X-Embodiment](https://robotics-transformer-x.github.io/)** — Cross-embodiment robotics datasets

---

## Documentation

- [ARCHITECTURE.md](./ARCHITECTURE.md) — Design and implementation
- [CONTRIBUTING.md](./CONTRIBUTING.md) — How to contribute
- [CHANGELOG.md](./CHANGELOG.md) — Version history

---

## Community

- **GitHub Issues** — [Ask questions, report bugs](https://github.com/Mullassery/PyRoboFrames/issues)
- **GitHub Discussions** — [Share ideas and best practices](https://github.com/Mullassery/PyRoboFrames/discussions)
- **Code of Conduct** — [Be respectful and constructive](./CODE_OF_CONDUCT.md)

---

## License

[MIT](./LICENSE)  Georgi Mammen Mullassery

---

## Citation

```bibtex
@software{mullassery2025pyroboframes,
 title={PyRoboFrames: Fast ML dataloader for robot learning},
 author={Mullassery, Georgi},
 url={https://github.com/Mullassery/PyRoboFrames},
 year={2025}
}
```

## Security & Error Handling

PyRoboFrames includes:

- **Secure Credential Handling**: IAM roles recommended over long-term credentials (see `DEPLOYMENT_SECURITY.md`)
- **Path Validation**: Prevents path traversal for S3/GCS access
- **Hardware Warnings**: Graceful degradation with fallback from GPU video decode
- **Detailed Error Messages**: See `python/pyroboframes/error_messages.py` for dataset recovery steps

## What's New in v1.3.0 (Q4 2026)

### Multi-Format Dataset Support 
Load datasets in multiple formats seamlessly:

```python
from pyroboframes import load_dataset, DatasetFormat

# Auto-detect format
loader = load_dataset('/path/to/dataset')

# Or hint the format explicitly
loader_rlds = load_dataset('/path/to/rlds_data', format_hint='RLDS')
loader_hdf5 = load_dataset('/path/to/hdf5_data', format_hint='HDF5')

# Load episodes and frames
episode = loader.load_episode(0)
frame = loader.load_frame(0, 42)
```

**Supported Formats:**

| Format | Source | Best For | Stream | Random |
|--------|--------|----------|--------|--------|
| LeRobot | HuggingFace | Modern datasets | | |
| RLDS | OpenX Embodiment | Multi-lab datasets | | |
| HDF5 | Traditional ML | Large hierarchical | | |
| Custom | Plugin system | Your format | | |

**Why This Matters:**
- Robot learning has 5+ competing dataset formats
- Teams locked into single format couldn't collaborate
- Multi-format support opens ecosystem collaboration
- Plugin system enables custom formats without forking

See `pyroboframes/_format_registry.py` for implementation.

