Metadata-Version: 2.4
Name: dexos-lite
Version: 0.1.0
Summary: Python client for dexos lite model inference and robot control.
Author-email: dexmal <join-maas@dexmal.com>
Project-URL: Homepage, https://maas.dexmal.com
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: numpy>=1.23
Requires-Dist: opencv-python>=4.8
Requires-Dist: Pillow>=10
Requires-Dist: piper_sdk>=0.3.0
Requires-Dist: python-can>=3.3.4
Requires-Dist: pandas>=2.0
Requires-Dist: pyarrow>=14
Requires-Dist: pyrealsense2>=2.54
Requires-Dist: requests>=2.31
Requires-Dist: megfile>=5.0.17
Requires-Dist: botocore>=1.43.98
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytype; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-mock>3.6.0; extra == "dev"
Requires-Dist: pytest-socket; extra == "dev"
Requires-Dist: pytest-retry; extra == "dev"

# dexos_lite

Command-line client for Dexos lite dataset upload and robot inference.

Version 0.1.0 requires Python >= 3.10 (megfile 5.0.17 requirement).

## Installation

Install from PyPI:

```bash
pip3 install dexos-lite
```

Hardware-specific SDKs are loaded only when a robot controller starts. Install the SDK required by your robot before running robot inference.

## Common Options

`--api-key` can be passed as a command option or provided through an environment variable.

```bash
export DEXOS_API_KEY=ak_xxx
```

## Upload A Dataset

```bash
dexos-lite dataset upload /data/yellow_pepper \
  --api-key ak_xxx \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset
```

Dataset uploads should use the `lerobot_v3` format. For the concrete layout, see the data format section.

Optional flags:

```bash
dexos-lite dataset upload /data/yellow_pepper \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset \
  --dataset-format lerobot_v3 \
  --description "yellow pepper pick dataset"
```

The upload command validates an uncompressed LeRobot v3 directory and uses megfile `smart_sync` to upload its contents directly to OSS with temporary STS credentials. It preserves relative paths, displays progress, retries failed transfers, and automatically refreshes credentials during long multipart uploads. No archive is created. Symbolic links are rejected.

Uploads require dexos-lite >= 0.1.0. `--robot-type` accepts `cobot_magic`, `agibot_g1`, `galaxea_r1_lite`, `mirror`, `franka`, or `other`. For `other`, also pass `--custom-robot-type my_robot_v2` (1–128 lowercase letters, digits, or underscores). Python upload functions likewise require `robot_type` and optionally `custom_robot_type`.

The client includes `client_version` in preparation, credential renewal, and completion requests. All three require a stable version >= 0.1.0. The server validates the directory layout and size before completion returns `PROCESSING`; the conversion worker then validates and converts the data. Only `ACTIVE` datasets with robot metadata and a valid conversion can be selected for training.

Upload destinations are assigned by the server:
- Built-in robot: `datasets/{user_id}/{dataset_id}/{robot_type}/`
- Custom robot: `datasets/{user_id}/{dataset_id}/other/{custom_robot_type}/`

The directory root is uploaded without an extra local folder name. `file_size` means the total size of source files. The SDK uses a unique in-memory megfile profile and cleans it up after transfer; it does not overwrite your default AWS credentials. The server's long-term AK/SK never reaches the SDK. Credential renewal cannot change the assigned directory. Existing STS tokens remain valid until expiry even after completion.

The low-level `prepare_upload` method no longer accepts `filename` or returns a presigned PUT URL. Use the high-level `upload_dataset` function for validation, directory transfer, automatic renewal, and completion.

### Local LeRobot Validation

Upload validation uses the standalone [lerobot_validator.py](dexos_lite/lerobot_validator.py)
module. Client uploads use fast validation by default. Set `full_check=True` for
server-side source validation, including every frame and complete video decoding.

| Check | Fast (default) | Full |
| --- | --- | --- |
| Robot type, custom identifier, names and declared shapes | Yes | Yes |
| Tasks, episode metadata, minimum length and file references | Yes | Yes |
| Required parquet columns, episode/task identifiers and row totals | Yes | Yes |
| Every state/action value and actual vector dimension | No | Yes |
| Frame order and timestamps | No | Yes |
| Actual video dimensions, FPS, decoded frame count and corruption | No | Yes |

Fast mode reads only `episode_index` and `task_index` from data parquet files,
groups each file once, and caches episode summaries. Its cost still grows with the
number of rows and files, but it does not read vector columns or decode videos.
Existing but damaged videos or malformed vector contents may pass fast validation;
the server must explicitly run full validation before accepting the dataset.

Full mode requires `ffmpeg` and `ffprobe` on `PATH` (for example,
`brew install ffmpeg` on macOS or `apt-get install ffmpeg` on Debian/Ubuntu).
Fast mode does not require either tool.

The module can be copied to another project without the rest of this SDK. It requires
`pandas` and `pyarrow`; it does not import DexData, PyTorch, OSS, or robot hardware packages.

```python
# If lerobot_validator.py was copied into your project:
from lerobot_validator import validate_lerobot_dataset

report = validate_lerobot_dataset("/data/yellow_pepper")
print(report["mode"], report["episodes_checked"], report["total_frames"])

# After upload, explicitly enable all source checks on the server:
report = validate_lerobot_dataset("/data/yellow_pepper", full_check=True)
```

When using the SDK package, import from `dexos_lite.lerobot_validator` instead. The
existing `validate_lerobot_dataset_root(Path(...))` entry point imports this module and
returns `(total_frames, task_indices)`. It also accepts the keyword argument
`full_check=True`. The adapter maps errors to the SDK's `DatasetFormatError`,
`DatasetReadError`, and `DatasetDependencyError` types. `validate_dataset_path(...,
strict=True)` uses fast validation; its default non-strict mode only checks basic file
properties. The upload flow enables strict validation, which is independent of the
standalone validator's `full_check` option.

You can also run the copied file directly:

```bash
python -m pip install "pandas>=2.0" "pyarrow>=14"
python lerobot_validator.py /data/yellow_pepper
python lerobot_validator.py /data/yellow_pepper --full-check
```

It prints one JSON result and exits with `0` on success or `1` on validation failure
(invalid command-line arguments exit with `2`). The import API returns a report with
`status`, `profile`, `mode`, `skipped_checks`, `episodes_checked`, `total_frames`,
`videos_referenced`, `frames_checked`, `videos_checked`, `task_indices`, `fps`,
`robot_type`, `custom_robot_type`, and `cameras`.
Fast reports have zero `frames_checked` and `videos_checked`: these counters refer
to full content checks, not identifier checks. `total_frames` is the verified row total.
Fast reports list `frame_contents` and `video_contents` under `skipped_checks`;
full reports have an empty list. The report profile is `lerobot-v3-source-v2`. It raises `ValidationError`, `ValidationReadError`, or
`ValidationDependencyError` on failure; input files are never changed.

The full source rules come from the independent dataset validation service.
The table above identifies which checks fast mode defers:

- LeRobot v3 metadata, positive finite FPS, nonempty robot type, declared vector dimensions,
  and at least one RGB video camera.
- Tasks with nonempty text in the `task` column or DataFrame index, consistent task
  counts/indices, and one task per episode.
- Unique episode indices, exact episode/frame totals, and at least `max(3, int(fps))` frames
  per episode to match the current converter's supported input.
- Every frame in order from index zero, timestamps matching `frame_index / fps`, and finite
  state/action values matching each declared dimension.
- Existing references inside the dataset root, no unregistered data parquet files, and
  consistent episode video intervals.
- Full source video decoding, matching dimensions/FPS, and enough actual frames for every
  referenced interval. Shared video shards may contain more frames than one episode.

#### Robot Type and Feature Names

The following is a **project naming convention**, not a mandatory LeRobot standard.
Upload parameters and `meta/info.json` use the same `DatasetRobotType` enum:
`cobot_magic`, `agibot_g1`, `galaxea_r1_lite`, `mirror`, `franka`, or `other`.
Values must match exactly. Missing/blank values, legacy aliases such as `aloha` or
`open_pdd_w1`, uppercase variants, hyphens, and surrounding spaces are rejected.

When `robot_type` is `other`, `custom_robot_type` is required in `meta/info.json` and
in the upload parameters. It is trimmed and must contain 1–128 lowercase letters,
digits, or underscores, matching the existing upload contract. Missing, null, blank,
and non-string values are rejected. Built-in robot types must not supply a non-null
`custom_robot_type`.

For example, custom robot metadata includes:

```json
{"robot_type": "other", "custom_robot_type": "my_robot_v2"}
```

The five built-in types skip only the `names` rules below. Shape, per-row dimensions,
finite numbers, file completeness, and all other validation remain mandatory.
`other` always requires names validation, even if `custom_robot_type` happens to be
`franka` or another built-in identifier. No legacy alias mapping or name inference is used.

For `other` robots, both `features["observation.state"]["names"]` and
`features["action"]["names"]` must be nonempty lists of nonempty strings, with no duplicates
and exactly `shape[0]` entries. Null, missing fields, and reference strings such as
`"observation.state"` are rejected. State and action names must be identical and exactly
match one of these ordered layouts:

1. Single-arm joints: `arm_joint_1` through `arm_joint_N`, then `gripper`, for `1 <= N <= 7`.
2. Dual-arm joints: `left_joint_1` through `left_joint_N`, `left_gripper`, then
   `right_joint_1` through `right_joint_N`, `right_gripper`. Both arms must have the same
   number of joints, from 1 to 7.

Names are case-sensitive and are not trimmed or normalized. Joint numbers must start at
1 without gaps; order is mandatory. Single- and dual-arm names cannot be mixed.
End-effector position and rotation names (including rotvec, Euler, and quaternion),
base/torso/hand joints, torque, velocity,
and any other extra vector dimensions are rejected. Each arm requires one scalar gripper.
Other LeRobot features, including images and timestamps, are not subject to these names
rules. This validator does not map slots, reorder vectors, pad to 32 dimensions, or generate
masks; it only checks the supplied names and data.

This replaces the old fixed 14-dimensional check and first-frame sampling. Pose fields,
`stats.json`, and the global `index` column are not required by these source rules.
Full mode keeps source parquet rows in memory; video decoding runs one file at a time.
Fast mode retains episode summaries after processing each file. Full decoding includes unused gaps in referenced source videos, so a
damaged gap also fails local validation. Unreferenced source videos are not inspected.

Task text can be stored in `tasks.parquet`'s `task` column or its text index. If both
contain text, they must agree exactly; an invalid `task` column is never silently ignored.
When present, `task_index` must still match row positions `0..n-1`. The SDK does not rewrite
input files. The server must normalize the validated texts into the temporary extracted
copy's task index before invoking the current converter, then reconcile every output
prompt with the original texts. Use the matching server update with this SDK change;
the older converter alone ignores the `task` column and would produce numeric prompts.

Local success is **source validation**, not server-side publication approval. Archive
safety, LeRobot-to-DexData conversion, conversion reconciliation, algorithm DataLoader
checks, OSS upload, and `_SUCCESS` publication remain server-side. It does not certify
annotation accuracy, joint semantics, physical safety, or training results.

## Generate Robot Config

Generate `config.py` in the current directory:

```bash
dexos-lite robot config --robot-id mirror
```

Generate a config at a specific path:

```bash
dexos-lite robot config \
  --robot-id mirror \
  --path ./configs/mirror_config.py
```

Overwrite an existing config:

```bash
dexos-lite robot config \
  --robot-id mirror \
  --path ./config.py \
  --force
```

Supported `robot_id` values:

- `mirror`
- `cobot_magic`

Edit the generated config before inference to set controller ports, sensor serial numbers, and robot-specific options.
Inference requires at least one camera sensor image. Each image must be a JPEG or PNG data URL. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent.

### Mirror Arm Service

For `mirror`, start the Mirror arm service before running inference.
Use the `mirror` documentation as the source of truth for exact service options.

```bash
conda activate airbot && sleep 1 && airbot_server -i can_left -p 50051
conda activate airbot && sleep 1 && airbot_server -i can_right -p 50053
```

## Run Robot Inference

```bash
dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./config.py \
  --prompt "pick up the red cube"
```

`--robot-config` accepts `path.py` or `path.py:attribute`. If no attribute is provided, `robot` is used.

`robot infer` runs for up to 600 seconds by default. Use `--time-limit` to set a different limit in seconds.

`--continue` starts inference from the robot's current pose by skipping the initial `robot.reset_to_home()` call. The CLI still calls `robot.start()` first to start hardware resources.

When inference ends because of `Ctrl+C` or the time limit, the CLI asks for a score from 0 to 10 and an optional feedback message, then submits it to service. When inference is interrupted with `Ctrl+C`, the CLI calls `robot.reset_to_home()` again before closing the robot.

Use `Ctrl+C` to stop the inference loop.

Joint-space controller commands use a fixed default interpolation step of `0.02`.
Set the uppercase `DEXOS_INTERPOLATION_STEP` environment variable to a positive
float to override it, or to `0` to disable interpolation:

```bash
export DEXOS_INTERPOLATION_STEP=0
```

## Use A Custom Robot

`--robot-config` can point to any Python file that exposes a `BaseRobot` instance or factory. If no attribute is provided, `robot` is used.

Custom robots must implement `action_type`, `start()`, `get()`, `set(actions)`, `reset_to_home()`, and `close()`.

`get()` must return a dict with:

- `state`: required. JSON-serializable current robot state sent to service as top-level `state`; for common arm models this is usually a flat list such as `[j1, j2, j3, j4, j5, j6, gripper]`. Refer to the `/api/v1/action` [API docs](https://maas.dexmal.com/documentation#api-state-ordering) for state element ordering.
- `sensors`: required. Dict of camera name to base64-encoded JPEG or PNG data URL. `infer_robot` sends this to the service as top-level `images`. Provide at least one image. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent. It supports at most three keys: `cam_high`, `cam_left_wrist`, and `cam_right_wrist`.

Each call to `set(actions)` receives and executes one model action step from an action horizon. The layout and element order of the action match the `state` returned by `get()`. The inference API may return multiple steps at once as an action horizon. The CLI passes each step to `set()` in order and requests the next inference result after the current action horizon has finished.

Continuous CLI inference does not impose an additional action rate limit. Therefore, `set()` must pace the hardware and return only after the current action has completed safely.

`reset_to_home()` is called once before inference starts and again when inference is interrupted with `Ctrl+C`. Custom robots should move the hardware to a safe home pose there; if there is no real home action, still return a JSON-serializable result such as `{"ok": True}`.

Create `custom_robot.py`:

```python
from __future__ import annotations

from typing import Any, Dict

from dexos_lite import ActionType
from dexos_lite.robot import BaseRobot


class CustomRobot(BaseRobot):
    @property
    def action_type(self) -> ActionType:
        return ActionType.JOINT

    def start(self) -> None:
        # Initialize SDKs, connect hardware, or start external services here.
        pass

    def get(self, *, sensor_timeout: float = 1.0) -> Dict[str, Any]:
        current_state = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        # Read cameras within sensor_timeout.
        # sensors requires at least one image and supports at most these three fields.
        sensors = {
            "cam_high": "data:image/jpeg;base64,...",
            # "cam_high": "data:image/png;base64,...",
            # "cam_left_wrist": "data:image/jpeg;base64,...",
            # "cam_right_wrist": "data:image/jpeg;base64,...",
        }
        return {"state": current_state, "sensors": sensors}

    def set(self, actions: Any) -> Dict[str, Any]:
        # Send actions to your robot SDK or hardware service here.
        return {"ok": True}

    def reset_to_home(self) -> Dict[str, Any]:
        # Move the robot to a safe home pose here.
        return {"ok": True}

    def close(self) -> None:
        pass


robot = CustomRobot()
```

Run inference with it:

```bash
dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./custom_robot.py:robot \
  --prompt "pick up the red cube"
```

You can also expose a factory, for example `def build_robot() -> CustomRobot: ...`, and pass `--robot-config ./custom_robot.py:build_robot`.
