Metadata-Version: 2.4
Name: ydderd-momentum-cli
Version: 0.5.0
Summary: Momentum customer CLI + experiment-logging SDK — authenticate, upload field data, and report training runs to your workspace.
Project-URL: Homepage, https://withflywheel.com
Project-URL: Repository, https://github.com/ydderd/flywheel
Author: Momentum
License: Apache-2.0
Keywords: cli,ingest,momentum,upload
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Requires-Dist: boto3>=1.34
Requires-Dist: httpx>=0.27
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Description-Content-Type: text/markdown

# flywheel-cli

The Momentum customer CLI + experiment-logging SDK: authenticate and bulk-upload field data
straight to your workspace's storage bucket, and report training/eval runs from your own compute
into your workspace's experiment tracker.

PyPI distribution: `ydderd-momentum-cli` · Homebrew formula: `momentum-cli` · command: `momentum`.
(The clean `momentum-cli` PyPI name was taken, so the distribution carries the `ydderd-` prefix;
the import package `momentum_cli`, the `momentum` command, and the brew name are unaffected.)

## Why this is a separate package

The CLI talks to the Momentum API purely over HTTP (and to R2 over S3). It shares **no Python
code** with the backend, so it ships with a tiny dependency set — `httpx` + `boto3` — instead of
the full server stack (torch, opencv, fastapi, …). That keeps the install small and avoids
shipping the backend's AGPL detector to customers.

## Install

```bash
brew install ydderd/flywheel/flywheel-cli
# or:
pipx install ydderd-momentum-cli
flywheel --help
```

## Usage

```bash
momentum auth login                       # opens a browser; a workspace admin approves
momentum auth whoami                      # confirm tenant
momentum upload ./your-data --scan        # bulk upload + trigger ingest
flywheel ingest status                    # ingest ledger stats
momentum eval submit --model hf://lab/pi05-fast --benchmark bench_roboarena@3  # open an eval run, print URL
flywheel trials log --policy hf://lab/pi05-fast --n 20 --successes 14 --calibration-set pcsk-…
flywheel trials log --csv trials.csv      # bulk floor tallies (per-row partial success)
flywheel secrets set lab-bucket           # store a secret (value from stdin); prints its creds_ref
flywheel secrets list                     # secret names + configured (never values)
```

`upload` always writes to your workspace's one raw prefix — there's no target to choose. Whether
what you uploaded is raw drone video (needs extraction) or already-extracted frames is classified
server-side once it lands, not by the client beforehand.

For headless/CI use, skip the browser with a token minted by a workspace admin:
`momentum auth login --token <fw_cli_…>`.

Config is stored at `~/.flywheel/config.json`. Auth precedence: `MOMENTUM_CLI_TOKEN` env >
config file.

## Experiment-logging SDK

Training and eval runs executed on your own compute (Modal, Brev, a lab box) report themselves
into your workspace's experiment tracker — W&B-style, and safe to leave in production training
code (a logging failure never raises into the train):

```python
import momentum_cli as momentum

run = momentum.init(name="my_sft_run", tags=["sft"], config={"iters": 800, "lr": 2e-4},
                    provider="modal")
run.log({"train/loss": 0.42}, step=100)
run.finish(status="succeeded", checkpoint_ref="r2://bucket/ckpt", cost_usd=295.26)

# later — scoring results and billed cost arrive after the train, so annotation
# works on finished runs:
momentum.annotate(run.id, results={"auroc": {"value": 0.61, "ci": [0.55, 0.67]}})
```

### Eval runs (policy context — the CI-integration path)

An eval process (a lab rig, Modal, the robot) evaluates `model × benchmark@version` and streams its
rollouts back. Same never-raise/heartbeat/reattach posture as training runs; rollouts buffer and flush
in batches, each with a client-generated id so a re-sent batch is idempotent:

```python
ev = momentum.eval_run(benchmark="bench_roboarena@3", model="hf://lab/pi05-fast", seeds=3)
ev.log_rollout(scenario="scn_pick", seed=0, status="success",
               scorer={"success": True, "task_progress": 1.0}, latency_p50=61.0)
ev.log_rollout(scenario="scn_pick", seed=1, status="fail", scorer={"success": False})
ev.finish()                                   # flushes any buffered rollouts first
ev.annotate(results={"headline": {"value": 0.5, "ci": [0.31, 0.69]}})   # post-hoc scoring
```

`eval_run()` prints the run URL on create; `eval_run(run_id=…)` (or `MOMENTUM_EVAL_RUN_ID`) reattaches
after a preemption. Runs land in the UI under **Eval runs**.

### Real trials (floor tallies → calibration audit)

Report real-robot trials of a policy; landing trials that ground a calibration set recomputes that world
model's τ/ρ trust:

```python
momentum.real_trials.log(policy="hf://lab/pi05-fast", scenario="scn_pick",
                         n=20, successes=14, operator="alice", calibration_set="pcsk-…")

report = momentum.real_trials.log_csv("trials.csv")   # a path or raw CSV text; per-row partial success
print(report["accepted"], report["rejected"])
```

### Secrets & referenced episodes

Register an episode that lives in your own bucket by first storing its credentials in the tenant secret
store (Fernet-encrypted at rest; the value is never returned by a read), then passing the returned
`creds_ref`:

```python
ref = momentum.secrets.set("lab-bucket", '{"access_key": "…", "secret_key": "…"}')  # → "secret://lab-bucket"
momentum.episodes.register("s3://lab-corpus/session_042", creds_ref=ref)
momentum.secrets.list()      # {name: {configured: bool}}, incl. provider keys under provider:<name>
```

Auth: `MOMENTUM_API_KEY` env (a `fw_cli_…` token — inject it as a secret in your training
environment), falling back to the token saved by `momentum auth login`. `MOMENTUM_API_URL`
overrides the API endpoint. `with momentum.init(...) as run:` (and `momentum.eval_run(...)`) marks the
run failed (with the exception) if the block raises. Runs land in the workspace UI under
**Experiments** / **Eval runs**.

Release/consumption mechanics (PyPI, git-ref installs, versioning): see `PUBLISHING.md`.

## Developer notes

These knobs exist for Momentum developers and are intentionally hidden from customer-facing
help and docs:

- **`--api-url <url>` on `momentum auth login`** — persist a non-production API base URL to the
  config (e.g. a local API). Hidden via `argparse.SUPPRESS`.
- **`MOMENTUM_API_URL` env** — override the API base per-invocation. Takes precedence over the
  config file.

Precedence for the API base URL: `MOMENTUM_API_URL` env > `api_url` in config > default
(`https://flywheeling.fly.dev/api` — swap to a custom domain once one is live).

Point the CLI at a local backend during development:

```bash
MOMENTUM_API_URL=http://localhost:8000 momentum auth whoami
# or persist it:
momentum auth login --token <fw_cli_…> --api-url http://localhost:8000
```

### Local development

```bash
cd cli
uv sync
uv run flywheel --help
uv run pytest
```

## Releasing (PyPI + Homebrew)

PyPI is the source of truth; the Homebrew formula wraps the published PyPI sdist.

### 1. Publish to PyPI — via GitHub Actions (Trusted Publishing, no token)

The `.github/workflows/publish-cli.yml` workflow builds and publishes over OIDC. Cut a release
by pushing a namespaced tag from the monorepo default branch:

```bash
git tag cli-v0.1.0 && git push origin cli-v0.1.0
```

The PyPI project is `ydderd-momentum-cli`, published from `ydderd/flywheel` via the `pypi`
environment. (First publish activates the "pending" Trusted Publisher and creates the project.)

### 2. Update the Homebrew tap formula

After the PyPI release exists, point `release.sh` at your tap checkout — with `SKIP_PUBLISH=1`
it skips the upload and only fetches the published sdist's `url`/`sha256`, rewrites the formula,
and regenerates its Python `resource` blocks:

```bash
SKIP_PUBLISH=1 \
FORMULA_PATH=/path/to/homebrew-momentum/Formula/flywheel-cli.rb \
  cli/scripts/release.sh
```

Then commit + push the tap. Customers install with:

```bash
brew install ydderd/flywheel/flywheel-cli
```

> `release.sh` can also publish to PyPI itself (`UV_PUBLISH_TOKEN=pypi-… cli/scripts/release.sh`)
> if you prefer a token-based local release over the GitHub Action.

Bumping a release: change `version` in `pyproject.toml`, push a new `cli-v*` tag, then re-run
step 2.

