Metadata-Version: 2.4
Name: pera-sdk
Version: 0.3.0
Summary: pera SDK: the benchmarking platform for physical intelligence — remote environments, canonical protocols, verifiable evidence
Project-URL: Homepage, https://github.com/verapulse/pera-sdk
Project-URL: Repository, https://github.com/verapulse/pera-sdk
Project-URL: Documentation, https://github.com/verapulse/pera-sdk/blob/main/ONBOARDING.md
Project-URL: Changelog, https://github.com/verapulse/pera-sdk/blob/main/CHANGELOG.md
Author: verapulse
License: Apache-2.0
License-File: LICENSE
Keywords: benchmark,embodied-ai,evaluation,physical-intelligence,robotics,vla
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: numpy>=1.23
Requires-Dist: websockets>=12
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Description-Content-Type: text/markdown

![pera banner](https://raw.githubusercontent.com/verapulse/pera-sdk/main/banner_image.png)

# pera

**The benchmarking platform for physical intelligence. This is its open SDK.**

pera evaluates robot policies (VLAs) on canonical, versioned simulation
protocols: same tasks, same seeds, same rules for every model. Your model stays
on your GPU; the benchmark runs on pera's servers over a WebSocket Environment
API. Every published score is anchored to the model authors' own harness run on
the same machine, and links to per-episode videos, action traces, labeled
failures, and a checksummed evidence bundle anyone can verify.

Models evaluated so far include OpenVLA-OFT, GR00T N1.6, π0.5, X-VLA, SmolVLA,
Octo, MolmoAct2, and PulseVLA across the four LIBERO suites and SimplerEnv
WidowX.

## Install

```bash
pip install pera-sdk
```

Python ≥ 3.10. Runtime dependencies are only `numpy` and `websockets` — no
framework, validation stack, HTTP client, or computer-vision package to fight
your model's environment.

## Sixty seconds to a score

```python
import pera

MANIFEST = {
    "schema_version": "pera.policy.v1",
    "name": "my-policy",
    "observation": {
        "cameras": ({"role": "primary", "resolution": (256, 256), "required": True},),
        "language": True,
    },
    "control_frequency_hz": {"trained": 20.0, "accepts": (10.0, 30.0)},
    "action": {
        "kind": "chunk", "horizon": 8,
        "translation": {"format": "delta", "frame": "base"},
        "rotation": {"format": "euler_xyz"},
        "gripper": {"format": "continuous_neg1_1", "close": "high"},
    },
}

def act(obs, task):
    # obs["images"]["primary"]: HxWx3 uint8 (upright), obs["state"]: np.ndarray | None
    return my_model(obs, task.instruction)   # -> (horizon, 7) action chunk

out = pera.evaluate(act, "libero-spatial@1", url=PERA_URL, api_key=KEY,
                    policy_manifest=MANIFEST,
                    tasks=[0], episodes_per_task=2)   # smoke-test scale
print(out["episodes"], "->", out["run_id"])
```

Two episodes cost a minute and catch convention bugs before they cost an
afternoon; when the videos look right, widen to the full protocol
(`tasks=list(range(10)), episodes_per_task=50` for the LIBERO suites).

No GPU is needed on your side until a real model enters the loop — the
simulator runs server-side, so you can prove your whole integration from a
laptop.

Stateful policy (internal history, action queues, per-episode reset)? Drive the
loop yourself with `pera.make()` — see
[`docs/03-stateful-and-chunked-policies.md`](docs/03-stateful-and-chunked-policies.md).

## The manifest is the contract

You declare what your policy consumes and emits — cameras, state format and
rotation convention, action space, control rate. The platform resolves your
manifest against the benchmark's:

- **direct** — formats match, nothing touches your data;
- **adapted** — explicit, recorded transforms (e.g. a resize), listed in the
  result bundle;
- **refused** — incompatible, *before* any simulation runs or credits burn.

A mismatched pair can never silently produce garbage — that's the design goal.
The convention bugs that inspired it (quaternion double-cover at θ=π, a wrist
camera served in the wrong frame, upside-down training images) are documented in
[`docs/02-manifests-and-contracts.md`](docs/02-manifests-and-contracts.md),
along with a checklist for your own integration.

## Evidence you can check

Every evaluation exports a bundle: seeds, per-step action transcripts,
per-episode outcomes, failure videos, and per-file checksums under a root hash.
The bundle format is an open spec. Verify any published bundle:

```bash
pera verify path/to/bundle
```

See [`docs/04-verifying-bundles.md`](docs/04-verifying-bundles.md). The spec
itself is published in [`spec/`](spec) — bundle format, wire protocol, format
registry, manifest schemas and every launch protocol.

## Tutorials — runnable

[`tutorials/`](tutorials/) has three notebooks plus the actual clients behind
every published leaderboard row:

- **[01 · Quickstart](tutorials/01_quickstart.ipynb)** — manifest → evaluation → results
- **[02 · Explore & verify bundles](tutorials/02_explore_and_verify_bundles.ipynb)** —
  fully offline: dissect a *real* vendored evidence bundle, tamper with it,
  watch verification fail, play the episode video
- **[03 · Stateful policy loops](tutorials/03_stateful_policy_loop.ipynb)** — the
  `make()` pattern for models with history/action queues
- `tutorials/*_client.py` — the production integrations (OFT, GR00T, Octo,
  SmolVLA, π0.5, X-VLA, PulseVLA), each documenting the convention that mattered

New here? Start with **[ONBOARDING.md](ONBOARDING.md)** — the seven-step path
from install to a score you can trust.

## Reference docs

1. [Quickstart](docs/01-quickstart.md) — first evaluation, reading results
2. [Manifests & contracts](docs/02-manifests-and-contracts.md) — declaring your policy honestly
3. [Stateful & chunked policies](docs/03-stateful-and-chunked-policies.md) — `make()` loops, replanning
4. [Verifying bundles](docs/04-verifying-bundles.md) — the evidence format
5. [**Walkthrough: SmolVLA on LIBERO-Spatial**](docs/05-walkthrough-smolvla-libero.md) — a complete worked example, from `pip install` to a score you can compare against our published row
6. [**Walkthrough: π0-FAST on LIBERO-Spatial**](docs/06-walkthrough-pi0fast-libero.md) — a full 500-episode run, same-machine reference anchor, and evidence-led investigation

## Status & access

The hosted platform is in early access with waitlisted API keys. The SDK,
bundle spec, and these docs are Apache-2.0. The evaluation engine itself is not
open source — published scores are *verified* precisely because pera runs them;
the open spec plus `pera verify` is how you audit us.

---
pera is a verapulse project.
