Metadata-Version: 2.3
Name: anyverse-adp
Version: 0.5.6
Summary: CLI and Python SDK for the Anyverse Data Platform Episode catalog
Requires-Dist: tomli>=2.0 ; python_full_version < '3.11'
Requires-Dist: httpx>=0.28,<1
Requires-Dist: prompt-toolkit>=3.0,<4
Requires-Dist: rich>=13.9,<14
Requires-Dist: questionary>=2,<3
Requires-Dist: typer>=0.15,<1
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# Anyverse ADP CLI and Python SDK

`anyverse-adp` provides the `adp-cli` command and a Python SDK for reading the Anyverse Data Platform's registered LeRobot catalog, managing Episode and Clip tags, managing Episode annotations and clips (including atomic Annotation/Clip writes), and exporting Episode metadata CSV files.

Install it with `pip install anyverse-adp`. Create a Platform API Key with `episodes:read`; Annotation writes, Episode/Clip tag updates, and Clip writes also require `episodes:write`. Configure the key with an environment variable or reuse the CLI's secure `~/.adp-cli/config.toml`.

```text
adp-cli list
adp-cli --version
adp-cli browse
adp-cli info <dataset-slug>
adp-cli export <dataset-slug> <collection-key> --region <region> --output <directory-or-file.csv>
```

`adp-cli browse` is an inline terminal selector. It redraws a single `Anyverse Data Platform CLI` panel in the current terminal so the selected Dataset, Collection, region, and latest result stay together. Use Up/Down and Enter to select a Dataset, Collection, region, and export mode; type in Dataset and Collection menus to filter choices, and use Backspace to clear the filter. Press Esc to return to the previous level (or exit from Dataset). After a dry-run or completed export, its result remains visible and the selector offers continue or exit. It does not open a full-screen UI or switch to an alternate terminal screen.

The command reads catalog metadata, including the selected Artifact `oss://bucket/prefix/` root in the `oss_location` CSV column, the Episode UUID, and `video_refs` keyed by the original camera/view feature name. Each video reference contains its stable OSS path and indexed start/end offsets in nanoseconds. It does not read or download dataset objects, expose raw annotations, or grant object-storage access.

## Parquet references (0.4.0)

Version 0.4.0 appends `parquet_filepath`, `parquet_row_start`, and `parquet_row_end` to CSV exports. The filepath is a stable `oss://` URI resolved from the selected region's unique Artifact location and the API's relative file path. The row range is zero-based `[start, end)` inside that file, with `end - start == frame_count`; it is not the Episode index. Unindexed references produce three empty cells. Older APIs without all three metadata fields also produce empty cells, so deploy the updated API before using 0.4.0 for Parquet exports. Partial or invalid references fail the export instead of publishing an incomplete CSV.

SDK Episodes expose `parquet_relative_path`, `parquet_filepath`, `parquet_row_start`, and `parquet_row_end`. These are `None` when the reference is unavailable. Resolving a complete path requires exactly one Artifact location in the client's selected region; ambiguous locations raise `AdpResponseError`.

## Python SDK (0.5.6)

The SDK uses a synchronous `Client` and resource handles organized as `Dataset → Collection → LeRobot → Episode`. Client and Dataset lists are materialized for indexing; LeRobot and Episode collections remain lazy and page as you iterate.

Read the installed package version from Python with `import anyverse_adp; print(anyverse_adp.__version__)`. It reports `"unknown"` when package metadata is unavailable, such as when importing an unpackaged source checkout.

```python
from anyverse_adp import AdpAuthorizationError, AdpError, Client

client = Client()  # explicit api_key, ADP_API_KEY, ADP_CLI_API_KEY, then ~/.adp-cli/config.toml
try:
    datasets = client.get_datasets()
    dataset = client.get_dataset("agibot_world")
    collections = dataset.get_collections()
    collection = dataset.get_collection("dexhand_auto_views")
    lerobot = next(iter(collection.lerobots))
    episode = lerobot.get_episode(index=100)  # one direct server lookup by (LeRobot, index)
    print(episode.episode_uuid, episode.index)
    episode = client.get_episode(episode_uuid=episode.episode_uuid)  # direct lookup by Episode UUID
    episode = lerobot.get_episode(episode_uuid=episode.episode_uuid)  # scoped to this LeRobot
    episode = collection.lerobots.get_episode(episode_uuid=episode.episode_uuid)  # scoped to this Collection

    # Bulk traversal remains lazy and cursor paged.
    for item in lerobot.get_episodes():
        print(item.index)

    # These are complete lists scoped to this Episode. List entries contain summaries;
    # accessing .annotation fetches the individual JSON body.
    annotations = episode.annotations
    annotation = episode.get_annotation("scene", "prompt-v3")
    print(annotations[0].annotation_type, annotation.annotation)

    clips = episode.get_clips("scene", "prompt-v3")
    if clips:
        print(clips[0].label, clips[0].annotation)

    # Writes use episodes:write and are keyed upserts; repeated keys update the same record.
    episode.add_clip(
        annotation_type="scene",
        model_prompt_version="prompt-v4",
        segment_id=1,
        frame_start=12,
        frame_end=48,
        label="pick up object",
        annotation={"object": "cup"},
    )
    episode.add_annotation(
        annotation_type="scene",
        model_prompt_version="prompt-v4",
        artifact_uri="oss://annotations/episode-100.json",
        result_flag=True,
        annotation={"summary": "The robot picks up the object."},
    )
except AdpAuthorizationError as exc:
    print(f"This operation requires {exc.required_scope}: {exc.operation}")
except AdpError as exc:
    print(f"ADP request failed: {exc}")
finally:
    client.close()
```

The default endpoint is `https://data-platform.cloud.anyverse.work` and the default region is `cn-shanghai`. Pass `endpoint=`, `region=`, or `api_key=` to override them. API Key precedence is explicit `api_key`, `ADP_API_KEY`, `ADP_CLI_API_KEY`, then the CLI config file `~/.adp-cli/config.toml`. The SDK reads that file only, preserves its permission and symlink checks, never prompts, and never writes it.

Use `client.get_episode(artifact_id, episode_index)` or `lerobot.get_episode(index)` for one Episode. The Platform API looks up that `(artifact_id, episode_index)` key directly. `iter_episodes()` and `get_episodes()` continue to use lazy cursor pagination.

`get_episode()` accepts one selector: `client.get_episode(artifact_id, episode_index)` (or `artifact_id=` and `episode_index=`) for index lookup, or `client.get_episode(episode_uuid=...)` for UUID lookup. The same UUID selector is available on `lerobot.get_episode()` and `collection.lerobots.get_episode()`; those resource methods constrain the result to the current LeRobot Artifact or Dataset Collection. `client.get_episode_by_uuid(episode_uuid)` remains as a compatibility alias. The Platform API looks up the Episode UUID primary key and returns the same metadata and region-scoped Artifact locations as index lookup. Direct lookups require `episodes:read`; a missing UUID or a UUID outside the selected resource scope raises `AdpNotFoundError`. No database migration is required.

`episode.video_refs` maps each camera feature key to a tuple of `VideoRef` values. Each reference exposes `relative_path`, `oss_uri`, `time_start_ns`, and `time_end_ns`. `oss_uri` combines the reference with the unique Artifact location for the Client's selected region; it is `None` if that location is ambiguous. Direct Episode lookups and paginated Episode reads populate the same fields. The URI identifies an OSS object but is not a signed HTTP URL; the SDK does not download video files.

`episode.get_tags()` / `clip.get_tags()` read the current tags from the Platform API. `episode.get_tag_confidence()` reads the Episode's `tag_code -> confidence` mapping; `episode.tag_confidence` and `episode.tags` contain the values returned with a loaded Episode. Episode writes use three methods: `update_tags(tags)` replaces the full tag list while retaining confidence for codes that remain; `update_tags_with_confidence(mapping)` replaces tags and confidence together, deriving the complete tag list from mapping keys in insertion order; and `add_tag(name=..., confidence=None)` appends one tag, optionally setting its confidence. An empty mapping to `update_tags_with_confidence` clears both tags and confidence. `confidence=None` leaves existing confidence unchanged (a newly added tag has no confidence); a number sets that tag's confidence without reading Episode metadata or replacing other confidence values. Confidence values must be finite numbers from 0 through 1. The SDK intentionally has no standalone confidence write method. Tag reads require `episodes:read`; writes require `episodes:write`.

The corresponding `Client` methods write directly by Episode UUID and do not fetch the full Episode first:

```python
client.update_episode_tags(episode_uuid, ["scene:private_living:kitchen"])
client.update_episode_tags_with_confidence(
    episode_uuid,
    {"scene:private_living:kitchen": 0.93},
)
client.add_episode_tag(
    episode_uuid,
    name="object:cup",
    confidence=0.87,
)
```

```python
episode = client.get_episode(episode_uuid="<episode_uuid>")
print(episode.get_tag_confidence())

# Replace tags and confidence together; the mapping keys become the complete tag list.
episode.update_tags_with_confidence({"scene:private_living:kitchen": 0.93})

# Add a tag and confidence atomically, preserving other confidence entries.
episode.add_tag(name="object:cup", confidence=0.87)
```

`episode.get_annotations()` / `episode.annotations` and `episode.get_clips()` / `episode.clips` return complete lists for one Episode. The server returns summary fields for lists, avoiding transfer of every JSON body. Use `episode.get_annotation(annotation_type, model_prompt_version)` for an Annotation body; Clip list items load their body through `clip.annotation`. Annotation and Clip writes are keyed upserts using the existing `episodes:write` scope. Use `episode.add_annotation_with_clips(...)` when one Annotation and its Clips must be written together: the SDK sends one request, the server stores them in one transaction, and each Clip is linked to the resulting Annotation automatically. Read calls require `episodes:read`; writes without `episodes:write` raise `AdpAuthorizationError` with `required_scope` and `operation`. Invalid request data raises `AdpValidationError` with the safe server code and message. Other typed failures include authentication, not-found, region, timeout, network, response-shape, and unhandled HTTP errors. Exceptions never include the API Key or raw response body.

The SDK reads metadata and annotation/clip records through Platform API. It does not download OSS objects, sign URLs, or use OSS credentials.

可直接运行的完整示例位于 `examples/sdk_catalog.py`：

```bash
ADP_API_KEY='<完整 API Key>' \
python packages/anyverse-adp/examples/sdk_catalog.py agibot_world --collection dexhand_auto_views
```

Jupyter 代码示例位于 [`examples/sdk_catalog.ipynb`](examples/sdk_catalog.ipynb)，按单元依次演示 Dataset、Collection、LeRobot 和 Episode 的读取。它只读取目录元数据，不下载 OSS 对象。
