Metadata-Version: 2.4
Name: lerobot-doctor
Version: 0.2.0
Summary: Dataset quality diagnostics for LeRobot v2/v3 datasets
Author: Jash Shah
License: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pyarrow>=14.0
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: av>=11.0
Requires-Dist: rich>=13.0
Requires-Dist: numpy>=1.24
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-tmp-files>=0.0.2; extra == "dev"

# lerobot-doctor

Dataset quality diagnostics for [LeRobot](https://github.com/huggingface/lerobot) v2/v3 datasets.

Catches issues that waste debugging time: corrupted timestamps, dropped frames, frozen actions, clipped values, metadata inconsistencies, video problems, stuck actuators, and more.

Works on local datasets and HuggingFace Hub datasets. No dependency on the lerobot package.

**Live now** on the [LeRobot Dataset Visualizer](https://huggingface.co/spaces/lerobot/visualize_dataset) as the "Doctor" tab, and as a standalone [HF Space](https://huggingface.co/spaces/jashshah999/lerobot-doctor).

## Install

```bash
pip install lerobot-doctor
```

Or from source:

```bash
git clone https://github.com/jashshah999/lerobot-doctor.git
cd lerobot-doctor
pip install .
```

## Usage

```bash
# Check a local dataset directory
lerobot-doctor /path/to/dataset

# Check a local dataset .zip archive (format version is auto-detected)
lerobot-doctor /path/to/dataset.zip

# Check a HuggingFace dataset
lerobot-doctor lerobot/pusht

# Run specific checks only
lerobot-doctor /path/to/dataset --checks metadata,temporal,actions

# JSON output (for CI/CD integration)
lerobot-doctor /path/to/dataset --json

# Markdown report (paste into PRs or dataset cards)
lerobot-doctor /path/to/dataset --markdown report.md

# Limit episodes checked (recommended for huge HF datasets like lerobot/droid_1.0.1)
lerobot-doctor lerobot/droid_1.0.1 --max-episodes 10

# Verbose (show PASS details)
lerobot-doctor /path/to/dataset -v

# CI mode: JSON output, exits 1 on failures
lerobot-doctor /path/to/dataset --ci

# CI mode: exits 1 on warnings too
lerobot-doctor /path/to/dataset --ci --fail-on=warn
```

## Checks (11 total)

| Check | What it catches |
|-------|----------------|
| **metadata** | Missing/invalid info.json, wrong episode/frame counts, missing data files, tasks.parquet issues |
| **temporal** | Non-monotonic timestamps, dropped frames, inconsistent fps, broken frame/episode indices |
| **actions** | NaN/Inf values, clipped actions, frozen (stuck) actions, sudden action jumps |
| **videos** | Missing video files, decode errors, fps/resolution mismatches, frame count mismatches |
| **statistics** | NaN/Inf in observations, zero-variance features, extreme outliers, stats.json drift |
| **episodes** | Short/empty episodes, length distribution, policy window compatibility (ACT/Diffusion), metadata-data length mismatches, task imbalance |
| **consistency** | Cross-episode feature schema changes (missing columns, dtype/shape mismatches), within-episode shape inconsistencies |
| **training** | Policy compatibility (ACT/Diffusion/VLA), normalization readiness (zero-std dims), action space sanity, delta_timestamps compatibility |
| **anomalies** | Stuck actuators (>80% static), near-duplicate episodes, distribution shift across dataset, broken sensors (constant observations) |
| **portability** | Absolute paths, symlinks, large files, HF Hub compatibility, non-standard files |
| **per_episode** | Per-episode drilldown: flags specific bad episodes with reasons (short, frozen, NaN, timestamp gaps, action jumps) |

## Exit codes

- `0`: All checks PASS or WARN
- `1`: At least one check FAIL

## Example output

```
lerobot-doctor v0.1.0 -- Dataset Quality Report
Dataset: lerobot/pusht (v3.0)
Episodes: 206 | Frames: 25,650 | FPS: 10

[PASS] Metadata & Format Compliance
[PASS] Temporal Consistency
[WARN] Action Quality
  - action: Episode 2 has 1 sudden large action jumps (>5 std)
  - action: Episode 3 has 2 sudden large action jumps (>5 std)
[WARN] Video Integrity
  - Skipping video decode checks for remote dataset
[WARN] Data Distribution
  - next.success: zero variance (constant value 0.0000)
[WARN] Episode Health
  - 2/10 episodes shorter than chunk_size=100 (used by ACT/Diffusion policies)
[PASS] Feature Consistency
[PASS] Training Readiness
[WARN] Anomaly Detection
  - next.success: ALL 1 dimensions constant across ALL episodes
[WARN] Per-Episode Drilldown
  - Episode 2: 1 sudden action jumps
  - Episode 3: 2 sudden action jumps

Summary: 5 PASS | 6 WARN

Suggested fixes:
  Check sensor connections -- constant readings indicate hardware issues
  Filter episodes shorter than your policy's chunk_size before training
```

## CI / GitHub Actions

Use `--ci` for pipeline integration. Outputs JSON to stdout, one-line summary to stderr.

```bash
# Fail pipeline on any FAIL
lerobot-doctor lerobot/pusht --ci

# Fail pipeline on WARNs too (stricter)
lerobot-doctor lerobot/pusht --ci --fail-on=warn
```

Example GitHub Actions step:

```yaml
- name: Dataset quality gate
  run: lerobot-doctor my-org/my-dataset --ci --fail-on=warn
```

## JSON output

Use `--json` for JSON output without CI exit-code behavior.

```bash
lerobot-doctor /path/to/dataset --json | jq '.overall_severity'
```

## Huge datasets

For very large datasets (e.g. `lerobot/droid_1.0.1` at ~28M frames across 156 data parquets + videos), always pass `--max-episodes N` with a small N (10-100). Running without it attempts a full download, which:

- On the hosted [HF Space](https://huggingface.co/spaces/jashshah999/lerobot-doctor): **will fail** -- the Space has ~50GB ephemeral disk. The Space also blocks "all episodes" on datasets with >1M frames.
- Locally: works if you have the bandwidth and disk, but is slow.

When `--max-episodes` is set on a HF dataset, lerobot-doctor:

1. Fetches small meta files (info.json, tasks.parquet, stats.json).
2. Downloads episodes meta parquets one at a time until the first N episodes are covered.
3. Resolves `data/chunk_index` + `data/file_index` to pull only the data parquets that contain those N episodes.

Checks that rely on the full dataset (e.g. `total_episodes` count in metadata) are automatically skipped in partial mode instead of flagging false-positive failures.

**Future work:** sampled full-dataset scans (random subset across chunks), video sampling without full download.

## Fix (auto-repair)

```bash
lerobot-doctor fix /path/to/dataset              # fix all issues (creates backup)
lerobot-doctor fix /path/to/dataset --dry-run    # preview fixes
lerobot-doctor fix /path/to/dataset --fixes timestamps,metadata
```

Fixes: broken indices, timestamp drift, NaN values, metadata mismatches.

## Trim (remove idle frames)

```bash
lerobot-doctor trim /path/to/dataset             # remove start/end idle frames
lerobot-doctor trim /path/to/dataset --dry-run   # preview
```

Fixes the "robot does nothing at inference" problem caused by static training data.

## Score (find bad episodes)

```bash
lerobot-doctor score /path/to/dataset            # rank episodes by quality
lerobot-doctor score /path/to/dataset --json     # JSON for automation
```

Scores: smoothness, coverage, consistency, length. Recommends which episodes to drop.

## Gate (pre-training check)

```bash
lerobot-doctor gate /path/to/dataset --policy act        # ACT compatibility
lerobot-doctor gate /path/to/dataset --policy diffusion  # Diffusion Policy
lerobot-doctor gate /path/to/dataset --policy smolvla    # SmolVLA
lerobot-doctor gate /path/to/dataset --policy pi0        # Pi0
```

Exit code 1 if dataset is incompatible. Catches: wrong dims, short episodes, NaN normalization.

## Merge Check

```bash
lerobot-doctor merge-check ./dataset1 ./dataset2         # pre-merge compatibility
lerobot-doctor merge-check ./merged_dataset --post-merge # post-merge validation
```

## Development

```bash
git clone https://github.com/jashshah999/lerobot-doctor.git
cd lerobot-doctor
pip install -e ".[dev]"
PYTHONPATH=src pytest tests/ -v
```
