Metadata-Version: 2.4
Name: benchloop_client
Version: 0.1.7
Summary: bmcli — command line tool for benchmark analysis data (metrics, trajectory, reports, artifacts).
Requires-Python: >=3.10
Requires-Dist: httpx>=0.25.2
Requires-Dist: tomli>=1.1.0; python_version < '3.13'
Requires-Dist: typer>=0.9.4
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# benchloop-client (`bmcli`)

Command line tool for fetching and downloading benchmark analysis data
(run metrics, trajectory, pre-analysis reports, task/job lists, and artifacts)
by `group_id` / `job_id`.

> PyPI package name is `benchloop_client`; the installed command is `bmcli`.

## Install package

```
pip install benchloop_client

bmcli --api-key $BENCH_LOOP_API_KEY overview get -j $JOB_ID
bmcli --api-key $BENCH_LOOP_API_KEY metrics get -j $JOB_ID

# Option 2: config ~/.benchloop/config.toml by cli, and run bmcli without --api-key option:
bmcli config set --api-key $BENCHLOOP_API_KEY --host https://benchmark-analysis.alibaba-inc.com

bmcli overview get -j $JOB_ID
bmcli metrics get -j $JOB_ID

```

## Install and run from source

```bash
# with uv (recommended)
uv sync
uv run bmcli --help

# or editable install with pip
python -m venv .venv && source .venv/bin/activate
pip install -e .
bmcli --help
```

## Auth & config

Authentication uses a personal api key sent as the `X-API-Key` header.

Precedence: CLI flag > env var > `~/.benchloop/config.toml` > default.

- Env: `BENCHLOOP_API_KEY` (or `BENCHMARK_API_KEY`), `BENCHLOOP_HOST`, `BENCHLOOP_OUTPUT_DIR`
- Default host: `https://benchmark-analysis.alibaba-inc.com`

```bash
bmcli config set --api-key $BENCHLOOP_API_KEY --host https://benchmark-analysis.alibaba-inc.com
```

## Usage

Two-segment `bmcli <resource> <action>` form. `get` prints a JSON array to
stdout (pipeable to `jq`); `download` writes the standard file layout.

```bash
bmcli metrics get -j J123 | jq '.[0]'
bmcli metrics download -g G1 --scope all -o ./out
bmcli trajectory download -g G1,G2 -o ./out
bmcli report download -j J123 --agent all
bmcli jobs get -g G1 --all          # get supports --all (auto-paginate)
bmcli jobs download -g G1           # download is always full
bmcli artifacts download -g G1 -o ./out
bmcli pull -g G1 --with metrics,trajectory,jobs,artifacts -o ./out
```

IDs accept repeated flags, comma-separated values, and `-` to read from stdin.
`-g` and `-j` are mutually exclusive in a single command.

## Exit codes

| code | meaning |
| ---- | ------- |
| 0 | all success |
| 1 | partial failure (some ids failed) |
| 2 | usage / parameter error |
| 3 | auth failure (HTTP 401) |
| 4 | network / timeout |

stdout carries data only; logs and progress go to stderr. `--quiet` suppresses
non-error logs, `--verbose` adds debug logs.


## Release new package

```
uv build
twine check dist/benchloop_client-0.1.7-py3-none-any.whl
twine upload dist/benchloop_client-0.1.7-py3-none-any.whl --verbose
```
