Metadata-Version: 2.4
Name: dexos-lite
Version: 0.1.1
Summary: Python client for dexos lite model inference and robot control.
Author-email: dexmal <join-maas@dexmal.com>
Project-URL: Homepage, https://maas.dexmal.com
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: numpy>=1.23
Requires-Dist: opencv-python>=4.8
Requires-Dist: Pillow>=10
Requires-Dist: piper_sdk>=0.3.0
Requires-Dist: python-can>=3.3.4
Requires-Dist: pandas>=2.0
Requires-Dist: pyarrow>=14
Requires-Dist: pyrealsense2>=2.54
Requires-Dist: requests>=2.31
Requires-Dist: megfile>=5.0.17
Requires-Dist: botocore>=1.43.98
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytype; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-mock>3.6.0; extra == "dev"
Requires-Dist: pytest-socket; extra == "dev"
Requires-Dist: pytest-retry; extra == "dev"

# dexos_lite

Command-line client for Dexos lite dataset upload and robot inference.

Version 0.1.0 requires Python >= 3.10 (megfile 5.0.17 requirement).

## Installation

Install from PyPI:

```bash
pip3 install dexos-lite
```

Hardware-specific SDKs are loaded only when a robot controller starts. Install the SDK required by your robot before running robot inference.

## Common Options

`--api-key` can be passed as a command option or provided through an environment variable.

```bash
export DEXOS_API_KEY=ak_xxx
```

## Upload A Dataset

```bash
dexos-lite dataset upload /data/yellow_pepper \
  --api-key ak_xxx \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset
```

Dataset uploads should use the `lerobot_v3` format. For the concrete layout, see the data format section.

Optional flags:

```bash
dexos-lite dataset upload /data/yellow_pepper \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset \
  --dataset-format lerobot_v3 \
  --description "yellow pepper pick dataset"
```

The upload command validates an uncompressed LeRobot v3 directory and uses megfile `smart_sync` to upload its contents directly to OSS with temporary STS credentials. It preserves relative paths, displays progress, retries failed transfers, and automatically refreshes credentials during long multipart uploads. No archive is created. Symbolic links are rejected.

Uploads require dexos-lite >= 0.1.0. `--robot-type` accepts `cobot_magic`, `agibot_g1`, `galaxea_r1_lite`, `mirror`, `franka`, or `other`. For `other`, also pass `--custom-robot-type my_robot_v2` (1–128 lowercase letters, digits, or underscores). Python upload functions likewise require `robot_type` and optionally `custom_robot_type`.

The client includes `client_version` in preparation, credential renewal, and completion requests. All three require a stable version >= 0.1.0. The server validates the directory layout and size before completion returns `PROCESSING`; the conversion worker then validates and converts the data. Only `ACTIVE` datasets with robot metadata and a valid conversion can be selected for training.

Upload destinations are assigned by the server:
- Built-in robot: `datasets/{user_id}/{dataset_id}/{robot_type}/`
- Custom robot: `datasets/{user_id}/{dataset_id}/other/{custom_robot_type}/`

The directory root is uploaded without an extra local folder name. `file_size` means the total size of source files. The SDK uses a unique in-memory megfile profile and cleans it up after transfer; it does not overwrite your default AWS credentials. The server's long-term AK/SK never reaches the SDK. Credential renewal cannot change the assigned directory. Existing STS tokens remain valid until expiry even after completion.

The low-level `prepare_upload` method no longer accepts `filename` or returns a presigned PUT URL. Use the high-level `upload_dataset` function for validation, directory transfer, automatic renewal, and completion.

### Local LeRobot Validation

Upload validation uses the standalone [lerobot_validator.py](dexos_lite/lerobot_validator.py)
module. Client uploads use fast validation by default. Set `full_check=True` when every source frame and complete video decoding are required.
The conversion worker uses `episode_sample_size=10` as described below.

| Check | Fast (default) | Full |
| --- | --- | --- |
| Robot type, custom identifier, names and declared shapes | Yes | Yes |
| Tasks, episode metadata, minimum length and file references | Yes | Yes |
| Required parquet columns, episode/task identifiers and row totals | Yes | Yes |
| Every state/action value and actual vector dimension | No | Yes |
| Frame order and timestamps | No | Yes |
| Video dimensions, decoded frame count and corruption; FPS differences warn | No | Yes |

Fast mode reads only `episode_index` and `task_index` from data parquet files,
groups each file once, and caches episode summaries. Its cost still grows with the
number of rows and files, but it does not read vector columns or decode videos.
Existing but damaged videos or malformed vector contents may pass fast validation;
the worker therefore runs the documented episode sample plus a full output DataLoader scan.

Full mode requires `ffmpeg` and `ffprobe` on `PATH` (for example,
`brew install ffmpeg` on macOS or `apt-get install ffmpeg` on Debian/Ubuntu).
Fast mode does not require either tool.

The module can be copied to another project without the rest of this SDK. It requires
`pandas` and `pyarrow`; it does not import DexData, PyTorch, OSS, or robot hardware packages.

```python
# If lerobot_validator.py was copied into your project:
from lerobot_validator import validate_lerobot_dataset

report = validate_lerobot_dataset("/data/yellow_pepper")
print(report["mode"], report["episodes_checked"], report["total_frames"])

# After upload, explicitly enable all source checks on the server:
report = validate_lerobot_dataset("/data/yellow_pepper", full_check=True)
```

When using the SDK package, import from `dexos_lite.lerobot_validator` instead. The
existing `validate_lerobot_dataset_root(Path(...))` entry point imports this module and
returns `(total_frames, task_indices)`. It also accepts the keyword argument
`full_check=True`. The adapter maps errors to the SDK's `DatasetFormatError`,
`DatasetReadError`, and `DatasetDependencyError` types. `validate_dataset_path(...,
strict=True)` uses fast validation; its default non-strict mode only checks basic file
properties. The upload flow enables strict validation, which is independent of the
standalone validator's `full_check` option.

You can also run the copied file directly:

```bash
python -m pip install "pandas>=2.0" "pyarrow>=14"
python lerobot_validator.py /data/yellow_pepper
python lerobot_validator.py /data/yellow_pepper --full-check
```

It prints one JSON result and exits with `0` on success or `1` on validation failure
(invalid command-line arguments exit with `2`). The import API returns a report with
`status`, `profile`, `mode`, `skipped_checks`, `episodes_checked`, `total_frames`,
`videos_referenced`, `frames_checked`, `videos_checked`, `task_indices`, `fps`,
`robot_type`, `custom_robot_type`, and `cameras`.
Fast reports have zero `frames_checked` and `videos_checked`: these counters refer
to full content checks, not identifier checks. `total_frames` is the verified row total.
Fast reports list `frame_contents` and `video_contents` under `skipped_checks`;
full reports have an empty list. The report profile is `lerobot-v3-source-v2`. It raises `ValidationError`, `ValidationReadError`, or
`ValidationDependencyError` on failure; input files are never changed.

The full source rules come from the independent dataset validation service.
The table above identifies which checks fast mode defers:

- LeRobot v3 metadata, positive finite FPS, nonempty robot type, declared vector dimensions,
  and at least one RGB video camera.
- Tasks with nonempty text in the `task` column or DataFrame index, consistent task
  counts/indices, and one task per episode.
- Unique episode indices, exact episode/frame totals, and at least `max(3, int(fps))` frames
  per episode to match the current converter's supported input.
- Every frame in order from index zero, finite nonnegative strictly increasing timestamps,
  and finite state/action values matching each declared dimension. Timestamps need not
  equal `frame_index / fps`.
- Existing references inside the dataset root and consistent episode video intervals.
  Extra data Parquet files not referenced by metadata produce a warning and are not read
  or counted. Unregistered or duplicate rows inside referenced Parquet files still fail.
- Full source video decoding, matching dimensions, and enough actual frames for every
  referenced interval. Declared versus actual FPS differences only warn (both values are logged). Shared video shards may contain more frames than one episode.

#### Robot Type and Feature Names

The following is a **project naming convention**, not a mandatory LeRobot standard.
Upload parameters and `meta/info.json` use the same `DatasetRobotType` enum:
`cobot_magic`, `agibot_g1`, `galaxea_r1_lite`, `mirror`, `franka`, or `other`.
Values must match exactly. Missing/blank values, legacy aliases such as `aloha` or
`open_pdd_w1`, uppercase variants, hyphens, and surrounding spaces are rejected.

When `robot_type` is `other`, `custom_robot_type` is required in `meta/info.json` and
in the upload parameters. It is trimmed and must contain 1–128 lowercase letters,
digits, or underscores, matching the existing upload contract. Missing, null, blank,
and non-string values are rejected. Built-in robot types must not supply a non-null
`custom_robot_type`.

For example, custom robot metadata includes:

```json
{"robot_type": "other", "custom_robot_type": "my_robot_v2"}
```

The five built-in types skip only the `names` rules below. Shape, per-row dimensions,
finite numbers, file completeness, and all other validation remain mandatory.
`other` always requires names validation, even if `custom_robot_type` happens to be
`franka` or another built-in identifier. No legacy alias mapping or name inference is used.

For `other` robots, both `features["observation.state"]["names"]` and
`features["action"]["names"]` must be nonempty lists of nonempty strings, with no duplicates
and exactly `shape[0]` entries. Null, missing fields, and reference strings such as
`"observation.state"` are rejected. State and action names must be identical and exactly
match one of these ordered layouts:

1. Single-arm joints: `arm_joint_1` through `arm_joint_N`, then `gripper`, for `1 <= N <= 7`.
2. Dual-arm joints: `left_joint_1` through `left_joint_N`, `left_gripper`, then
   `right_joint_1` through `right_joint_N`, `right_gripper`. Both arms must have the same
   number of joints, from 1 to 7.

Names are case-sensitive and are not trimmed or normalized. Joint numbers must start at
1 without gaps; order is mandatory. Single- and dual-arm names cannot be mixed.
End-effector position and rotation names (including rotvec, Euler, and quaternion),
base/torso/hand joints, torque, velocity,
and any other extra vector dimensions are rejected. Each arm requires one scalar gripper.
Other LeRobot features, including images and timestamps, are not subject to these names
rules. This validator does not map slots, reorder vectors, pad to 32 dimensions, or generate
masks; it only checks the supplied names and data.

This replaces the old fixed 14-dimensional check and first-frame sampling. Pose fields,
`stats.json`, and the global `index` column are not required by these source rules.
Full mode keeps source parquet rows in memory; video decoding runs one file at a time.
Fast mode retains episode summaries after processing each file. Full decoding includes unused gaps in referenced source videos, so a
damaged gap also fails local validation. Unreferenced source videos are not inspected.

Task text can be stored in `tasks.parquet`'s `task` column or its text index. If both
contain text, they must agree exactly; an invalid `task` column is never silently ignored.
When present, `task_index` must still match row positions `0..n-1`. The SDK does not rewrite
input files. The server must normalize the validated texts into the temporary extracted
copy's task index before invoking the current converter, then reconcile every output
prompt with the original texts. Use the matching server update with this SDK change;
the older converter alone ignores the `task` column and would produce numeric prompts.

Local success is **source validation**, not server-side publication approval. Archive
safety, LeRobot-to-DexData conversion, conversion reconciliation, algorithm DataLoader
checks, OSS upload, and `_SUCCESS` publication remain server-side. It does not certify
annotation accuracy, joint semantics, physical safety, or training results.

## Generate Robot Config

Generate `config.py` in the current directory:

```bash
dexos-lite robot config --robot-id mirror
```

Generate a config at a specific path:

```bash
dexos-lite robot config \
  --robot-id mirror \
  --path ./configs/mirror_config.py
```

Overwrite an existing config:

```bash
dexos-lite robot config \
  --robot-id mirror \
  --path ./config.py \
  --force
```

Supported `robot_id` values:

- `mirror`
- `cobot_magic`

Edit the generated config before inference to set controller ports, sensor serial numbers, and robot-specific options.
Inference requires at least one camera sensor image. Each image must be a JPEG or PNG data URL. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent.

### Mirror Arm Service

For `mirror`, start the Mirror arm service before running inference.
Use the `mirror` documentation as the source of truth for exact service options.

```bash
conda activate airbot && sleep 1 && airbot_server -i can_left -p 50051
conda activate airbot && sleep 1 && airbot_server -i can_right -p 50053
```

## Run Robot Inference

```bash
dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./config.py \
  --prompt "pick up the red cube"
```

`--robot-config` accepts `path.py` or `path.py:attribute`. If no attribute is provided, `robot` is used.

`robot infer` runs for up to 600 seconds by default. Use `--time-limit` to set a different limit in seconds.

`--continue` starts inference from the robot's current pose by skipping the initial `robot.reset_to_home()` call. The CLI still calls `robot.start()` first to start hardware resources.

When inference ends because of `Ctrl+C` or the time limit, the CLI asks for a score from 0 to 10 and an optional feedback message, then submits it to service. When inference is interrupted with `Ctrl+C`, the CLI calls `robot.reset_to_home()` again before closing the robot.

Use `Ctrl+C` to stop the inference loop.

Joint-space controller commands use a fixed default interpolation step of `0.02`.
Set the uppercase `DEXOS_INTERPOLATION_STEP` environment variable to a positive
float to override it, or to `0` to disable interpolation:

```bash
export DEXOS_INTERPOLATION_STEP=0
```

## Use A Custom Robot

`--robot-config` can point to any Python file that exposes a `BaseRobot` instance or factory. If no attribute is provided, `robot` is used.

Custom robots must implement `action_type`, `start()`, `get()`, `set(actions)`, `reset_to_home()`, and `close()`.

`get()` must return a dict with:

- `state`: required. JSON-serializable current robot state sent to service as top-level `state`; for common arm models this is usually a flat list such as `[j1, j2, j3, j4, j5, j6, gripper]`. Refer to the `/api/v1/action` [API docs](https://maas.dexmal.com/documentation#api-state-ordering) for state element ordering.
- `sensors`: required. Dict of camera name to base64-encoded JPEG or PNG data URL. `infer_robot` sends this to the service as top-level `images`. Provide at least one image. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent. It supports at most three keys: `cam_high`, `cam_left_wrist`, and `cam_right_wrist`.

Each call to `set(actions)` receives and executes one model action step from an action horizon. The layout and element order of the action match the `state` returned by `get()`. The inference API may return multiple steps at once as an action horizon. The CLI passes each step to `set()` in order and requests the next inference result after the current action horizon has finished.

Continuous CLI inference does not impose an additional action rate limit. Therefore, `set()` must pace the hardware and return only after the current action has completed safely.

`reset_to_home()` is called once before inference starts and again when inference is interrupted with `Ctrl+C`. Custom robots should move the hardware to a safe home pose there; if there is no real home action, still return a JSON-serializable result such as `{"ok": True}`.

Create `custom_robot.py`:

```python
from __future__ import annotations

from typing import Any, Dict

from dexos_lite import ActionType
from dexos_lite.robot import BaseRobot


class CustomRobot(BaseRobot):
    @property
    def action_type(self) -> ActionType:
        return ActionType.JOINT

    def start(self) -> None:
        # Initialize SDKs, connect hardware, or start external services here.
        pass

    def get(self, *, sensor_timeout: float = 1.0) -> Dict[str, Any]:
        current_state = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        # Read cameras within sensor_timeout.
        # sensors requires at least one image and supports at most these three fields.
        sensors = {
            "cam_high": "data:image/jpeg;base64,...",
            # "cam_high": "data:image/png;base64,...",
            # "cam_left_wrist": "data:image/jpeg;base64,...",
            # "cam_right_wrist": "data:image/jpeg;base64,...",
        }
        return {"state": current_state, "sensors": sensors}

    def set(self, actions: Any) -> Dict[str, Any]:
        # Send actions to your robot SDK or hardware service here.
        return {"ok": True}

    def reset_to_home(self) -> Dict[str, Any]:
        # Move the robot to a safe home pose here.
        return {"ok": True}

    def close(self) -> None:
        pass


robot = CustomRobot()
```

Run inference with it:

```bash
dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./custom_robot.py:robot \
  --prompt "pick up the red cube"
```

You can also expose a factory, for example `def build_robot() -> CustomRobot: ...`, and pass `--robot-config ./custom_robot.py:build_robot`.

### Conversion worker validation (0.1.0)

The worker pins `dexos-lite==0.1.0`, matching the upload server's documented
minimum version. Publish the reviewed 0.1.0 package containing the sampling API
before building the worker image from the package index.

```python
from dexos_lite.lerobot_validator import validate_lerobot_source
source = validate_lerobot_source("/data/dataset", episode_sample_size=10)
report = source["report"]
```

All episode metadata, file references, identifier counts, and task assignments
are checked. The validator deterministically selects up to ten episode IDs,
evenly spaced over sorted IDs including both endpoints (all when there are at
most ten). Every frame and every camera segment in selected episodes is checked.
Only selected frame rows are retained in `source["episodes"]`; that list preserves
metadata traversal order so conversion output serials remain aligned. Shared
Parquet uses filtered batches and shared video uses segment seeking, which may
decode from a preceding keyframe. No complete frame-table cache is retained.

`validate_lerobot_dataset(..., episode_sample_size=10)` returns only the summary.
Existing default fast preflight and explicit `full_check=True` remain available
for callers that need those distinct scopes. The conversion worker compares only
the selected episodes, releases their source rows, then runs its DataLoader over
**every output episode and frame** in batches of 64. Video decoding is sampled;
DataLoader checks numeric values and media references, not image loading.


Video FPS mismatches do not reject readable data. Metadata interval duration differing
from `length / info.fps` also warns; intervals must still have finite nonnegative starts
and an end after the start. Sample and full validation seek to the actual referenced start and decode exactly
the required episode frame count. The optional end is informational and never truncates
the required frames or causes extra frames from the next episode to be read. Frame-boundary float rounding is tolerated;
frames are never duplicated to repair gaps. Corrupt video, an unreadable start or insufficient required frames remain errors.
An inaccurate end beyond EOF may warn and pass if every required frame is readable. Full mode additionally decodes the complete shared
video, including unreferenced gaps.
