# CHANGELOG


## v0.13.0 (2026-03-03)

### Features

- Add docs sync trigger ([#52](https://github.com/OpenAdaptAI/openadapt-ml/pull/52),
  [`dff678a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/dff678ac7a217fb76e2a17eadd9d946d40fde59a))


## v0.12.0 (2026-03-03)

### Features

- Add GRPO training module with minimal TRL bridge
  ([#34](https://github.com/OpenAdaptAI/openadapt-ml/pull/34),
  [`339e5d3`](https://github.com/OpenAdaptAI/openadapt-ml/commit/339e5d35f8c7d0c9880ad3bed9cc748ee7e77945))

* docs: add experimental roadmap and evidence context to vision

- Add 2x2 experimental matrix (retrieval × fine-tuning) to Core Thesis - Add evidence context to
  benchmark table: note it's an internal synthetic benchmark (~3 UI elements) that validates the
  pipeline, not real-world performance. Link to openadapt-evals for ongoing WAA/OSWorld evaluation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use 46.7% consistently in 2x2 matrix

Was showing 33-47% range which conflated preliminary (n=3) and full (n=45) results. The validated
  number is 46.7%.

* feat: add GRPO training module for online RL

Add openadapt_ml/training/grpo/ package with: - GRPOConfig for training hyperparameters -
  GRPORolloutCollector connecting to openadapt-evals RLEnvironment - GRPOTrainer implementing custom
  GRPO loop for multimodal VLMs - Binary reward function and group-relative advantage computation -
  Chain-of-thought warm-up pipeline for SFT pre-training - 20 unit tests passing without GPU

* fix: address review findings in GRPO module

- Replace copy.deepcopy(model) with LoRA state dict snapshot (prevents OOM) - Mark
  _compute_rollout_loss as scaffold with dummy forward pass for grad flow - Fix collect_rollout call
  to match RLEnvironment API (task_id in signature) - Add model.eval()/model.train() toggling around
  rollout/training phases - Remove unused gradient_accumulation_steps config field - Use actual
  screen_size from RLEnvironment instead of hardcoded 1920x1200 - Clamp CLICK coordinates to [0.0,
  1.0] to prevent invalid pixel values - Validate task_ids non-empty at start of train() - Export
  CoT warmup functions from package __init__ - Add BenchmarkAction fallback when openadapt-evals not
  installed - Add 9 new tests: action parser (8) + empty task_ids validation (1) - All 29 tests
  passing

* feat: implement GRPO loss computation and fix cot_warmup dependency

Implement the core _compute_rollout_loss method that was previously a NotImplementedError scaffold.
  The implementation:

- Reconstructs VLM prompts from rollout observations - Formats actions back to DSL text via new
  _format_action_as_text helper - Computes log-probabilities of action tokens under current policy -
  Computes reference policy log-probs via PEFT disable_adapter() with fallback to manual LoRA weight
  swapping - Returns GRPO loss: -advantage * log_prob + kl_coef * KL penalty

Also adds get_api_adapter() factory function to api_adapter.py, fixing the broken import in
  cot_warmup.py's generate_cot_annotations().

Additional review fixes from prior session: - Initialize _is_unsloth and _ref_lora_state in __init__
  - Remove dead else branch for task_id selection - Fix total_loss device placement - LoRA-only
  fallback save in checkpoint - TYPE regex accepts single quotes - Coordinate clamping in
  _parse_vlm_output_to_action

40 tests passing (10 new: 8 format_action + 1 roundtrip + 1 api_adapter).

* refactor: deduplicate GRPO prompts via shared _build_agent_messages

Extract prompt construction into _build_agent_messages() which imports SYSTEM_PROMPT from
  next_action.py (the SFT training prompt). This ensures the GRPO agent uses the same prompt
  distribution the model was warm-started on, and guarantees _make_agent_fn and
  _compute_rollout_loss use identical prompts (critical for correct log-prob computation).

* fix(grpo): address critical review findings in GRPO loss computation

- C-01: Store raw model output on action._grpo_raw_text for accurate loss - C-02: Separate
  tokenization of prompt/action with concatenation to fix BPE boundary alignment - I-01: Prefer LoRA
  weight swapping over disable_adapter() for reference policy (captures initial LoRA state after SFT
  warm-start) - I-03: Per-step gradient accumulation via immediate backward() to prevent OOM from
  building computation graph over all rollout steps - I-04: Fix unescape order in TYPE parser
  (backslash before quotes) - M-03: Pass model_name through get_api_adapter to ApiVLMAdapter - M-07:
  Case-insensitive CLICK/TYPE regex in _parse_vlm_output_to_action - L-01: Extract
  DEFAULT_SCREEN_SIZE constant, replace all hardcoded values

* fix(grpo): fix instruction propagation, screen size, weight swap safety

- CR-01: Task instruction was never populated during GRPO rollouts.
  WAALiveAdapter._get_observation() does not populate raw_observation, so the agent prompt said
  "Goal: " with nothing after it. Fix: store instruction on Rollout dataclass (populated from
  env._current_task in collector), use it in both agent_fn and _compute_rollout_loss. - IM-01:
  Change DEFAULT_SCREEN_SIZE from 1920x1200 to 1920x1080 for consistency with baselines module and
  standard VM configurations. Add screen_size field to GRPOConfig so it is configurable. - IM-02:
  Add try/finally around LoRA weight swap in _compute_ref_log_probs. Without this, an exception
  during the reference forward pass permanently corrupts the model state.

* fix(grpo): remove unused torch import in _setup_model

The import torch at line 121 was flagged by ruff (F401) as unused. The surrounding code only calls
  .detach().clone() on tensor objects, which does not require the torch module directly.

* style(grpo): apply ruff formatting to GRPO module files

Run ruff format on cot_warmup.py, rollout_collector.py, and trainer.py to satisfy the CI ruff
  formatter check.

* refactor(grpo): replace custom trainer with minimal TRL bridge

Replace 809-line custom GRPO trainer with ~280 lines that: - Use standard HuggingFace
  AutoModelForVision2Seq + AutoProcessor + PEFT LoraConfig instead of Unsloth monkey-patching -
  Implement standalone GRPO loss in ~15 lines of PyTorch (clipped surrogate) instead of custom
  policy gradient + KL penalty - Use beta=0.0 (no KL penalty, no reference model) per DAPO/Open-
  Reasoner-Zero literature, eliminating weight-swap complexity - Keep per-step backward to avoid OOM
  on long trajectories - Use standard model.save_pretrained() for checkpointing - Document WHY
  standalone GRPO math vs TRL GRPOTrainer (VLM multi-turn image pixel_values not stored in token
  IDs) and WHEN to switch

Preserves all public API: GRPOTrainer, _parse_vlm_output_to_action, _format_action_as_text,
  _build_agent_messages, DEFAULT_SCREEN_SIZE. All 50 tests pass (44 existing + 6 new for grpo_loss
  and trainer internals).

* feat(grpo): add E2E tests with artifact generation and architecture docs

- tests/test_grpo_e2e.py: 5 E2E tests (training loop, rollout collection, loss convergence, weight
  diff, mathematical properties) using tiny mock VLM. Produces 65+ artifacts (JSON traces, PNGs,
  checkpoints, summaries). - scripts/grpo_e2e_report.py: CLI report generator for test artifacts
  (text + optional HTML output). - docs/grpo_e2e_test_design.md: design rationale for E2E test
  approach - docs/grpo_architecture_analysis.md: analysis of custom vs TRL-based GRPO -
  docs/grpo_trl_rewrite_draft.py: TRL v0.29.0 integration research -
  docs/strategic_analysis_evals_ml_synergy.md: business/economics analysis

* fix(grpo): address self-review findings (BUG-01, CLEAN-01 through -05)

- Rename grpo_loss to policy_gradient_loss with honest docstring: single-epoch on-policy means
  ratio=1.0, clipping never fires, this is REINFORCE with group-relative advantages. Keep grpo_loss
  as backwards-compatible alias. - Add public aliases: parse_vlm_output_to_action,
  format_action_as_text (drop underscore prefix for public API) - Export policy_gradient_loss and
  public functions from __init__.py - Remove unused config fields: kl_coef (was 0.01 but never used
  with beta=0), max_seq_length (never referenced) - Fix model_name default:
  Qwen/Qwen2.5-VL-7B-Instruct (not unsloth variant) - Fix trivial test assertion: grad_norm > 0 (was
  >= 0, always true) - Update loss tests to verify gradient direction, not just loss sign - Add
  test_public_api_exports for new public names

56 tests pass (51 unit + 5 E2E).

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.11.2 (2026-02-25)

### Bug Fixes

- **docs**: Require conventional commit format for PR titles
  ([#32](https://github.com/OpenAdaptAI/openadapt-ml/pull/32),
  [`303f54f`](https://github.com/OpenAdaptAI/openadapt-ml/commit/303f54f64a342044fddd26ffdd835b006c2a48a5))

PR titles become squash merge commit messages. Without the fix:/feat: prefix,
  python-semantic-release skips the release. Document this requirement prominently in CLAUDE.md.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

### Documentation

- Enforce branch protection rules ([#30](https://github.com/OpenAdaptAI/openadapt-ml/pull/30),
  [`afad981`](https://github.com/OpenAdaptAI/openadapt-ml/commit/afad9810bc51909bc86237378b2fb5356c569790))

* docs: add mandatory branch/PR rule to CLAUDE.md

Adds explicit instruction that all changes must go through feature branches and pull requests.
  enforce_admins has been enabled on GitHub to prevent admin bypass of branch protection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(modal): remove unused os import

Fixes ruff F401 lint error on modal_cloud.py.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.11.1 (2026-02-24)

### Bug Fixes

- **modal**: Fix inference container image and multi-modal message handling
  ([`88e4c09`](https://github.com/OpenAdaptAI/openadapt-ml/commit/88e4c09dcfdf26acda0b4b35d064c68aa297c895))

- Pin transformers==4.57.3 (matches local, has Qwen3-VL support) - Add torchvision dependency
  (required by AutoVideoProcessor) - Add fallback: AutoModelForVision2Seq ->
  Qwen2_5_VLForConditionalGeneration - Add fallback: AutoProcessor -> Qwen2_5_VLProcessor -
  Reconstruct multi-modal messages with {"type": "image"} placeholders for proper vision token
  generation in apply_chat_template - Rename container_idle_timeout -> scaledown_window (Modal API
  update)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.11.0 (2026-02-24)

### Features

- **modal**: Add inference serving with call_inference API
  ([`57e5c5f`](https://github.com/OpenAdaptAI/openadapt-ml/commit/57e5c5ff9bfefc69003c608981f2e702b0507e65))

- Add _build_inference_app() for Modal GPU inference with PEFT adapter - Add
  upload_adapter_to_volume() for uploading adapters to Modal volume - Add call_inference() as the
  primary API for remote inference - Add 'serve' CLI command for interactive model serving -
  Container caches model in memory across calls (container_idle_timeout=600) - Support --no-adapter
  for zero-shot base model serving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.10.1 (2026-02-24)

### Bug Fixes

- **modal**: Apply fixes from first successful Modal training run
  ([`120c903`](https://github.com/OpenAdaptAI/openadapt-ml/commit/120c903065b0dc15936bc1a58da7ec2e1b84e6af))

- Add `serialized=True` to @app.function for non-global-scope support - Auto-create volume before
  upload, add `--force` for overwrites - Fix variable scoping (`vol = training_volume`) inside
  remote function - Add `openadapt-ml[training]` to container image dependencies - Use `--jsonl`
  flag in train subprocess for correct data path - Add `modal` to project dependencies - Update test
  to verify create+put two-call pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.10.0 (2026-02-24)

### Features

- **cloud**: Add Vast.ai and Modal GPU providers
  ([`5812f89`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5812f89db83f6be71202b0cc6fec587554b11ec7))

Vast.ai (~$0.17/hr A10): SSH+rsync marketplace model with full CLI (list, launch, terminate, train)
  matching lambda_labs.py pattern. Includes GPU search, --gpu-wait retry, auto-convert --demo-dir
  flow.

Modal ($30/mo free, $1.10/hr A10G): Python-native cloud with zero-ops training via decorated
  functions and Modal Volumes for data transfer. CLI: train, status, download, list-volumes.

Both support the same --demo-dir end-to-end pipeline as Lambda Labs.

53 new tests (34 Vast.ai + 19 Modal), all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.9.0 (2026-02-24)

### Features

- **train**: Add end-to-end pipeline automation with --demo-dir flag
  ([`b874018`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b874018d45b174f4f9c6bacb855f571f91612cf0))

Add prepare_bundle() and generate_screenshot_mapping() to convert_demos.py for single-call demo
  conversion. Extend both train.py and lambda_labs.py train commands with --demo-dir,
  --captures-dir, --mapping flags so the full pipeline (mapping → conversion → bundle → upload →
  train) runs as one command. Add --gpu-wait for Lambda GPU availability retry loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.8.0 (2026-02-24)

### Features

- Sft training pipeline with demo conversion, Lambda Labs integration, and data persistence
  ([#29](https://github.com/OpenAdaptAI/openadapt-ml/pull/29),
  [`e8baa69`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e8baa692a30c19c24981ac5c2c25e7e1462e26cc))

* feat(training): add demo conversion pipeline for ms-swift SFT format

Convert annotated demo JSON files to JSONL training data compatible with ms-swift for Qwen3-VL
  fine-tuning. Handles coordinate conversion from [0,1] to [0,1000] range, generates <think> blocks
  from observation and intent fields, and accumulates action history across steps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(training): add screenshot linking via mapping file

Support --mapping flag for pre-computed screenshot mapping JSON that maps task_id -> {step_index ->
  screenshot_path}. This correctly handles the coalesced step-to-raw-capture mapping (where step
  indices skip due to merging). Also adds --captures-dir with DB-based fallback and
  DOUBLE_CLICK/RIGHT_CLICK parsing.

* fix(training): align SFT format with inference prompt and add validation

- Add system role message to training conversations matching qwen3vl_agent.SYSTEM_PROMPT - Add
  "Output exactly one action" / thinking instruction to user message matching _build_prompt() output
  - Add coordinate range validation warning for values outside [0, 1] - Add input schema validation
  for required demo/step fields - Remove broken _resolve_screenshots_from_db() and
  _resolve_screenshots_direct() fallbacks that produced silently wrong mappings for coalesced demos
  - Remove --screenshot-dir CLI arg (unreliable for coalesced demos) - Keep --mapping (recommended)
  and capture API as screenshot resolution

* feat(training): add JSONL training pipeline with bundle support

Align convert_demos output with internal SFT format (images + messages), add train_from_jsonl()
  loader, --jsonl flag to train.py, --bundle flag to convert_demos and Lambda Labs train command.
  Enables training on annotated demo data without Episode objects.

* fix(training): add TRL callback, 4-bit quantization, early stopping

- Add OpenAdaptCallback for training_log.json output + early stop on loss - Fix _load_standard_model
  to use BitsAndBytesConfig for 4-bit quantization - Use AutoModelForImageTextToText (supports
  Qwen3-VL) instead of Qwen2VL class - Switch demo config to 2B model for fast iteration on A10 -
  Hide Azure ML Jobs panel when cloud_provider is not azure - Fix Lambda setup: remove uv.sources
  before uv sync on remote

* fix(cloud): use git archive for code sync, fix callback MRO

Replace rsync with `git archive HEAD | ssh tar` in sync_local_code() to send only committed tracked
  files (~10MB vs ~1.8GB with binary artifacts).

Fix callback class MRO: _OpenAdaptCallback must precede TrainerCallback so our on_log/on_train_begin
  override the no-op base implementations.

* refactor(training): consolidate duplicated SFTTrainer setup

Extract _run_sft_training() shared by train_with_trl() and train_from_jsonl(), eliminating ~80 lines
  of duplicated SFTConfig, SFTTrainer instantiation, and training loop code.

* feat(training): add plateau-based early stopping

Add early_stop_min_delta and early_stop_plateau_patience to stop training when loss stops improving
  by at least min_delta for N consecutive steps. Works alongside the existing absolute threshold.

* docs: add GPU hosting options and training pipeline gap analysis

GPU hosting options covers 24 platforms ranked by value for open-source projects needing
  free/credited GPU compute for VLM fine-tuning.

* docs: add Qwen3-VL-2B training results analysis

Detailed analysis of first fine-tuning run: 27.24 → 9.77 loss (64% reduction) over 50 steps on 20
  annotated WAA demo samples. Includes per-epoch breakdown, compute efficiency metrics, and
  recommendations for future training runs.

* fix: remove absolute paths from repo, fix LoRA task_type, add tests

- Remove screenshot_mapping.json (has absolute local paths), add to .gitignore, add
  screenshot_mapping.example.json instead - Fix LoRA task_type: always use CAUSAL_LM (Qwen-VL is
  decoder-only, not encoder-decoder like T5/BART that needs SEQ_2_SEQ_LM) - Add 57 tests for
  convert_demos (action parsing, coordinate conversion, step conversion, validation, bundle
  creation) and training callback (log writing, threshold early stopping, plateau detection)

* fix: address self-review issues (config wiring, security, tests)

- Wire lr_scheduler_type, weight_decay, max_grad_norm, target_modules from YAML config through to
  SFTConfig (were silently ignored) - Fix command injection in lambda_labs.py via shlex.quote() -
  Fix callback writing loss=0 on non-loss log events (track _last_loss) - Fix WAIT() mapping to
  wait() instead of finished() in convert_demos - Fix CI: add --no-sources to uv sync for uv.sources
  compatibility - Add test for non-loss log event callback behavior - Update SYSTEM_PROMPT comment
  (remove stale cross-reference)

* ci: fix uv.sources with UV_NO_SOURCES env var, skip integration tests

UV_NO_SOURCES=1 covers uv sync, uv run ruff, and uv run pytest. Integration tests require
  openadapt_evals which is not a dependency.

* style: format annotate.py with ruff

* feat: auto-generate training plots, persist data with checkpoint

- Add plot_training.py: generates loss curve, LR schedule, and combined plots from training_log.json
  using matplotlib - Copy training_log.json + plots into checkpoint directory after training so
  artifacts are self-contained and never lost - Add periodic rsync of training_log.json during
  Lambda training (every 5 min) so data survives instance interruption - Replace ASCII loss curve in
  training results doc with real PNG plots - Add reconstructed training_log.json from Qwen3-VL-2B
  demo run

* test: add tests for plot generation and checkpoint co-location

11 tests covering: - Loss plot generation (with/without LR data) - Output directory creation and
  defaults - Empty data handling - Epoch boundary rendering - Real training log validation -
  Checkpoint co-location of log + plots

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.7.1 (2026-02-18)

### Bug Fixes

- **ci**: Use v9 branch config for python-semantic-release
  ([#28](https://github.com/OpenAdaptAI/openadapt-ml/pull/28),
  [`2a79519`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2a7951959984ee49082bdbbe7700cc873385caac))

Replace `branch = "main"` (v7/v8 key) with `[tool.semantic_release.branches.main]` table (v9 key).
  The old key is silently ignored by v9, causing releases to never trigger on the main branch.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.7.0 (2026-02-18)

### Features

- **demo-prompt**: Add VLM-annotated traces for 3 recorded demos
  ([`5462725`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5462725bbb753979072398cca2b3619fa640987a))

Ran annotation pipeline (GPT-4o) on all 3 recorded captures: - 37e10fc4 (notifications): 5 steps —
  turn off system notifications - 0c9dda13 (archive): 9 steps — create Archive folder, move .docx
  files - 366de66e (notepad): 6 steps — open Notepad, create draft.txt

These grounded traces replace fabricated hand-written demos for demo-conditioned evaluation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.6.0 (2026-02-17)

### Bug Fixes

- **deps**: Bump openadapt-capture to >=0.3.0, add uv.sources
  ([`26b10d0`](https://github.com/OpenAdaptAI/openadapt-ml/commit/26b10d0014b3145c757d7b86fe414690835b5ae6))

The new recording format uses recording.db (not capture.db). Local editable source ensures lockfile
  resolves correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

### Code Style

- Fix ruff formatting for migrated import references
  ([#27](https://github.com/OpenAdaptAI/openadapt-ml/pull/27),
  [`a4628f8`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a4628f8482a50d1b9a7c1a08f7eb86e87a107a80))

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

### Documentation

- Add demo GIFs back to README
  ([`f23daa6`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f23daa63fd3f4d0a3cd8900a4815691843b1c093))

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Rewrite CLAUDE.md — remove migration guide, match pure ML scope
  ([`c9be969`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c9be9690871481f437020a27792bdd76a41f0ea5))

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Rewrite README for professional open-source style
  ([`e393edb`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e393edb42a71821f0c2cbaf16cb84ddd4804c216))

Replace 1100-line README containing stale VM/pool references with clean 220-line README reflecting
  what the package actually contains post-migration. Use test.yml badge instead of release.yml for
  accurate build status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Update README references to migrated CLI
  ([#26](https://github.com/OpenAdaptAI/openadapt-ml/pull/26),
  [`71ffaad`](https://github.com/OpenAdaptAI/openadapt-ml/commit/71ffaad20bb6ecac0f40ec2dff82c69caa3a3e0d))

* feat: remove evaluation infrastructure (moved to openadapt-evals)

All evaluation infrastructure (~13,000 lines) has been migrated to openadapt-evals (PR #29). This PR
  removes the now-redundant code from openadapt-ml, making it a pure ML package.

Deleted files: - benchmarks/cli.py (8,503 lines - VM/pool CLI) - benchmarks/azure_vm.py
  (AzureVMManager) - benchmarks/pool.py (PoolManager) - benchmarks/vm_monitor.py,
  azure_ops_tracker.py, resource_tracker.py - benchmarks/azure.py, viewer.py, pool_viewer.py,
  trace_export.py - benchmarks/waa_deploy/ (Docker agent deployment) -
  tests/test_quota_auto_detection.py, test_demo_persistence.py - tests/benchmarks/test_api_agent.py,
  test_waa.py

Updated: - benchmarks/__init__.py: Only exports ML agents (PolicyAgent, etc.) - pyproject.toml:
  Removed azure-ai-ml, azureml-core, azure-mgmt-* - CLAUDE.md: Removed CLI/VM/pool docs, added
  migration guide

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale references to migrated benchmark modules

Update all remaining references to deleted benchmark modules across source code, scripts, and tests:

- cloud/local.py: azure_ops_tracker, session_tracker, CLI subprocess calls - scripts/: p0/p1
  validation scripts, screenshot generators, quota checker - training/benchmark_viewer.py: HTML
  template CLI references - experiments/waa_demo/runner.py: docstring and print references -
  deprecated/waa_deploy/__init__.py: import path

All now point to openadapt_evals equivalents.

* docs: update README references to migrated CLI

All VM/pool CLI commands moved from openadapt_ml.benchmarks.cli to openadapt-evals (oa-vm). Update
  all README references.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

### Features

- **demo-prompt**: Add VLM annotation pipeline for recorded demos
  ([`2ef7f64`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2ef7f64644ea3f8aae21c620d0558bd2a4acef84))

Converts raw recordings (coordinates + screenshots) into structured text traces matching the
  hand-written demo format. Uses VLM to annotate each step with screen observation, intent, semantic
  action, and result.

Pipeline: capture → episode → coalesce → annotate (VLM) → validate → format

Key components: - Step coalescing (500 raw actions → 5-30 meaningful steps) - Click marker rendering
  on screenshots for VLM - Before+after frame pairs for grounded result descriptions - Sequential
  context (previous step annotation feeds into next) - Compact formatting matching hand-written demo
  shape - Runner integration with annotated > hand-written priority

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


## v0.5.0 (2026-02-13)

### Features

- Remove evaluation infrastructure (moved to openadapt-evals)
  ([#25](https://github.com/OpenAdaptAI/openadapt-ml/pull/25),
  [`2d57d02`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2d57d028411cc591eb9459ef25fc91b8c00839ca))

* feat: remove evaluation infrastructure (moved to openadapt-evals)

All evaluation infrastructure (~13,000 lines) has been migrated to openadapt-evals (PR #29). This PR
  removes the now-redundant code from openadapt-ml, making it a pure ML package.

Deleted files: - benchmarks/cli.py (8,503 lines - VM/pool CLI) - benchmarks/azure_vm.py
  (AzureVMManager) - benchmarks/pool.py (PoolManager) - benchmarks/vm_monitor.py,
  azure_ops_tracker.py, resource_tracker.py - benchmarks/azure.py, viewer.py, pool_viewer.py,
  trace_export.py - benchmarks/waa_deploy/ (Docker agent deployment) -
  tests/test_quota_auto_detection.py, test_demo_persistence.py - tests/benchmarks/test_api_agent.py,
  test_waa.py

Updated: - benchmarks/__init__.py: Only exports ML agents (PolicyAgent, etc.) - pyproject.toml:
  Removed azure-ai-ml, azureml-core, azure-mgmt-* - CLAUDE.md: Removed CLI/VM/pool docs, added
  migration guide

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale references to migrated benchmark modules

Update all remaining references to deleted benchmark modules across source code, scripts, and tests:

- cloud/local.py: azure_ops_tracker, session_tracker, CLI subprocess calls - scripts/: p0/p1
  validation scripts, screenshot generators, quota checker - training/benchmark_viewer.py: HTML
  template CLI references - experiments/waa_demo/runner.py: docstring and print references -
  deprecated/waa_deploy/__init__.py: import path

All now point to openadapt_evals equivalents.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.4.2 (2026-02-13)

### Bug Fixes

- **cli**: Write correct dict format for --task in run command
  ([#24](https://github.com/OpenAdaptAI/openadapt-ml/pull/24),
  [`e2bba55`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e2bba555bf04941a2bf50fa4ca95847868e66369))

WAA's run.py expects test config as {domain: [task_ids...]} dict, but --task wrote a bare JSON array
  [task_id] causing TypeError when run.py indexes by domain string key.

Now looks up the task's domain from test_all.json inside the container and writes the correct
  {domain: [task_id]} format.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.4.1 (2026-02-13)

### Bug Fixes

- Remove broken WAA submodule, embed startup script
  ([#22](https://github.com/OpenAdaptAI/openadapt-ml/pull/22),
  [`075592b`](https://github.com/OpenAdaptAI/openadapt-ml/commit/075592b5217ea836514ead062f3875e444f0deff))

The vendor/WindowsAgentArena submodule pointed to unpushed local commits (a956c5b) that don't exist
  upstream, breaking git-based pip installs.

- Remove submodule entirely (not a runtime dependency) - Embed the 9-line
  compute-instance-startup.sh as a constant in cli.py - Update path references in Azure ML commands
  to not depend on vendor/

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

- **ci**: Use ADMIN_TOKEN for release automation on protected branches
  ([#23](https://github.com/OpenAdaptAI/openadapt-ml/pull/23),
  [`177c1e4`](https://github.com/OpenAdaptAI/openadapt-ml/commit/177c1e4a95984dfd6743af152acc1041a5ae8618))

GITHUB_TOKEN cannot push version-bump commits to branches with PR protection. Use org-level
  ADMIN_TOKEN instead, with skip-check to prevent infinite loops on release commits.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

### Refactoring

- **benchmarks**: Extract library API from CLI
  ([#21](https://github.com/OpenAdaptAI/openadapt-ml/pull/21),
  [`f4be903`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f4be903d74a97267ac68d7efd292dcc85ab36ff6))

* refactor(benchmarks): extract library API from CLI for programmatic usage

Extract core VM management and pool lifecycle logic from cli.py into importable modules
  (azure_vm.py, pool.py) with clean Python APIs.

- Add AzureVMManager class with Azure SDK primary path + az CLI fallback - Add PoolManager class for
  pool create/wait/run/cleanup lifecycle - Add configurable resource_group via Settings, env var, or
  --resource-group flag - Support DefaultAzureCredential for enterprise SSO/service principals - CLI
  handlers become thin wrappers delegating to library classes - Add agent_factory parameter stub on
  PoolManager.run() for pluggable agents

All 327 tests pass, CLI surface unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix pre-existing ruff lint errors in pool_viewer and resource_tracker

Remove unused import `json` and unused variable `worker_re` in pool_viewer.py, and unused import
  `Optional` in resource_tracker.py.

* style: run ruff formatter on benchmarks modules

* fix(azure_vm): add SDK path for set_auto_shutdown via generic resource API

Auto-shutdown schedules are Microsoft.DevTestLab/schedules resources. Use azure-mgmt-resource
  (already a dependency) to create them via the generic resource client, with az CLI fallback if SDK
  fails.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>


## v0.4.0 (2026-02-06)

### Code Style

- **cli**: Run ruff formatter
  ([`84fb35a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/84fb35a73c68bf7b0825cc2af6bbdf06f56e79ad))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Features

- **benchmarks**: Add pool viewer and auto-shutdown
  ([#20](https://github.com/OpenAdaptAI/openadapt-ml/pull/20),
  [`2aeb81c`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2aeb81c7b732d6adb0daa5839933afca0cb5cf3e))

* feat(benchmarks): add HTML viewer for WAA pool benchmark results

Add pool_viewer.py module and CLI command for generating interactive HTML viewers from WAA parallel
  benchmark runs.

Features: - Parse waa-pool-*.log files to extract task results - Summary stats (total tasks, success
  rate, avg time per task) - Per-worker breakdown showing tasks per worker - Task list with
  pass/fail status and step counts - Domain breakdown with per-domain success rates - Interactive
  filters for domain and status

Usage: uv run python -m openadapt_ml.benchmarks.cli view-pool uv run python -m
  openadapt_ml.benchmarks.cli view-pool --run-name pool_run_20260204 uv run python -m
  openadapt_ml.benchmarks.cli view-pool --no-open

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs(claude): document VM auto-shutdown and orphan prevention

Add documentation for the auto-shutdown feature: - Explain auto-shutdown policy (default 4 hours) -
  Document --auto-shutdown-hours flag for pool-create and create - Document -y flag for pool-cleanup
  (skip confirmation) - Document test VM cleanup via try/finally

* docs(readme): update CLI commands to use pool-* workflow

Update documentation to reflect the current working CLI: - Replace outdated `vm monitor` with
  `pool-status/pool-vnc/pool-logs` - Update single VM workflow to use `pool-create --workers 1` -
  Add analyze_pool_logs.py script for parsing benchmark results

* fix(cli): prevent orphaned test VMs during pool-create

Remove --no-wait flag from test VM creation so the VM fully exists before we attempt to delete it.
  Previously, the test VM would still be provisioning when delete was called, causing delete to fail
  silently and leave orphaned VMs consuming quota.

* fix(cli): use waa-auto image in pool-wait, wait for apt lock

Critical fixes for end-to-end pool workflow:

1. Use waa-auto:latest in pool-wait (not windowsarena/winarena) - pool-create builds waa-auto with
  modern dockurr/windows v5.14 - pool-wait was incorrectly using vanilla windowsarena/winarena
  (v0.00) - v0.00 doesn't support VERSION=11e auto-download - This caused "ISO file not found"
  errors

2. Wait for apt lock before Docker install - Fresh Azure VMs run unattended-upgrades - apt-get
  install failed with "unable to locate package" - Added wait loop for /var/lib/apt/lists/lock

* fix(pool): match working waa command parameters exactly

- Use vanilla windowsarena/winarena:latest with --entrypoint /bin/bash - Add --prepare-image false
  --start-client false flags (skips ISO download) - Use 172.30.0.2 for probe and emulator_ip
  (matching working waa command)

The pool-wait command was broken because it used waa-auto:latest without the proper entrypoint and
  flags. The working 'waa' command (line 5404-5454) uses these exact parameters successfully.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>


## v0.3.1 (2026-02-05)

### Bug Fixes

- **cli**: Resolve ruff linter errors
  ([`210a31f`](https://github.com/OpenAdaptAI/openadapt-ml/commit/210a31fcc054238a08e609520bdf57c312600d72))

- Replace bare `except:` with `except Exception:` - Remove unused f-string prefixes - Remove unused
  variable assignments - Remove unused imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Documentation

- **readme**: Add parallel WAA evaluation, fix build badge
  ([#19](https://github.com/OpenAdaptAI/openadapt-ml/pull/19),
  [`fea0a10`](https://github.com/OpenAdaptAI/openadapt-ml/commit/fea0a10c514b87a8a73310a142acb73a6b31146e))

* docs(readme): add parallel WAA evaluation section, fix build badge

- Fix broken build badge (publish.yml → release.yml) - Add prominent "Parallel WAA Benchmark
  Evaluation" section near top - Add detailed "WAA Benchmark Workflow" section (#14) with: - Single
  VM and parallel pool workflows - VNC access instructions - Architecture diagram - Cost estimates -
  Update section numbering (Limitations → 15, Roadmap → 16)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(readme): address self-review feedback

- Fix anchor placement (move before heading for proper navigation) - Correct pool-delete →
  pool-cleanup (actual command name) - Add pool-status example for getting worker IPs - Add "prices
  vary by region" caveat

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>


## v0.3.0 (2026-02-05)

### Bug Fixes

- **cli**: Improve pool-create reliability and error handling
  ([`f23bd57`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f23bd571a76c361d9e46d99820728ffdedb5cef5))

- Properly clean up test VM and associated resources during quota check - Use sudo for docker pull
  (usermod not effective in same session) - Add pool-cleanup command for orphaned resources - Show
  full error messages in pool creation failures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **pool**: Use WAA native task distribution with --worker_id/--num_workers
  ([`ef0d8c7`](https://github.com/OpenAdaptAI/openadapt-ml/commit/ef0d8c7ecf60b1644dbc5a40ed0a05b1b4c2f597))

- Fixed task distribution: WAA ignores --start_idx/--num_tasks, use native --worker_id and
  --num_workers parameters instead - Worker 0 gets tasks 0, N, 2N... Worker 1 gets tasks 1, N+1,
  2N+1... - Use vanilla windowsarena/winarena image with correct IP (20.20.20.21) - Add container
  reuse check (skip restart if already running) - Pass API key via env var instead of config file -
  Fix QMP port exposure (7200) for QEMU control - Store Windows disk on /mnt for 300GB temp storage
  (D8ds_v5)

Tested: 2-worker pool running 4 tasks in parallel successfully

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **waa**: Use D4ds_v4 VM size for quota compatibility
  ([`2a51a97`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2a51a976f10db135ed79971162f5605de944dd6e))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **waa**: Use D8ds_v5 VM size for Azure ML workers
  ([`71a0fdd`](https://github.com/OpenAdaptAI/openadapt-ml/commit/71a0fddfa4216a49b3a37c1ff1cc2d98d1f605a1))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Documentation

- Add Azure ML log streaming and cost tracking guides
  ([`59c3a3e`](https://github.com/OpenAdaptAI/openadapt-ml/commit/59c3a3ef852d747599de53fe74f660aea6d5b033))

Document the new CLI commands for: - Live log streaming from Azure ML jobs - Cost tracking for
  compute instances - Teardown procedures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Features

- **cli**: Add Azure ML log streaming, cost tracking, and teardown
  ([`59e3cf7`](https://github.com/OpenAdaptAI/openadapt-ml/commit/59e3cf7fbdb53f07866709efd60b05a1aa511ed5))

Add comprehensive Azure ML management commands: - azure-ml-stream: Stream logs from running jobs
  using Python SDK with account key auth (works around DefaultAzureCredential permission issues) -
  azure-ml-cost: Track compute instance uptime and estimated costs - azure-ml-teardown: Cancel jobs
  and delete compute instances

Also improves: - azure-ml-quota: Shows both ML Dedicated quota (what Azure ML actually uses) and
  regular VM quota - Better error handling and logging throughout

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **cli**: Add Azure ML status, VNC, and monitor commands
  ([`7985cff`](https://github.com/OpenAdaptAI/openadapt-ml/commit/7985cff95cc50d0313b3f3cb8ff5a1a1de039a71))

New commands for end-to-end Azure ML automation: - azure-ml-status: Show jobs and compute instances
  - azure-ml-vnc: Set up VNC tunnel to compute instance - azure-ml-monitor: Monitor jobs with auto
  VNC setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **cli**: Add azure-ml-quota command for quota management
  ([`eecb3a4`](https://github.com/OpenAdaptAI/openadapt-ml/commit/eecb3a461c2b5f6882755dab52f2cfb0c9a9616a))

Semi-automated quota increase workflow: - Checks current quota for WAA-compatible VM families -
  Shows which families have sufficient quota - Opens Azure Portal quota page with instructions -
  Guides user through the request process

Usage: uv run python -m openadapt_ml.benchmarks.cli azure-ml-quota

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **cli**: Add multi-VM pool commands for parallel WAA evaluation
  ([`005664a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/005664ab0ec6c425ac9ebe1af19b23e552b9bf90))

Add pool-create, pool-wait, and pool-run commands for running WAA benchmarks across multiple VMs in
  parallel:

- pool-create --workers N: Create N VMs with Docker and WAA image - Parallel VM creation using
  ThreadPoolExecutor - Auto-selects available region and VM size - Configures Docker with /mnt
  storage - Registers pool for tracking

- pool-wait: Wait for WAA to be ready on all workers - Starts WAA containers on each worker - Polls
  /probe endpoint until ready - Configurable timeout

- pool-run --tasks N: Distribute tasks across pool - Round-robin task distribution - Parallel
  execution on all workers - Progress tracking in registry

This enables ~5x faster benchmark completion with 5 workers, or full 154-task evaluation in ~10min
  with 10+ workers.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Refactoring

- **waa**: Update submodule with SDK v2 migration
  ([`5080ad6`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5080ad697e88ff297dfbb14f2c0756f53ebfd496))

Updates WindowsAgentArena submodule to include Azure ML SDK v2 migration that enables job submission
  from macOS ARM64.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>


## v0.2.2 (2026-01-29)

### Bug Fixes

- **ci**: Remove build_command from semantic-release config
  ([`c0d455a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c0d455a395ec27f9705b15661cf978b092772a35))

The python-semantic-release action runs in a Docker container where uv is not available. Let the
  workflow handle building instead.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Continuous Integration

- Add auto-release workflow
  ([`46c91fe`](https://github.com/OpenAdaptAI/openadapt-ml/commit/46c91fe137d807a738a122a60c512470612ea708))

Automatically bumps version and creates tags on PR merge: - feat: minor version bump - fix/perf:
  patch version bump - docs/style/refactor/test/chore/ci/build: patch version bump

Triggers publish.yml which deploys to PyPI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Switch to python-semantic-release for automated versioning
  ([`4b5ab9a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/4b5ab9aaeb970b0ef4798fed9aa0a7d7d7854e01))

Replaces manual commit parsing with python-semantic-release: - Automatic version bumping based on
  conventional commits - feat: -> minor, fix:/perf: -> patch - Creates GitHub releases automatically
  - Publishes to PyPI on release

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>


## v0.2.1 (2026-01-29)

### Bug Fixes

- **ci**: Resolve ruff linter and format errors
  ([#15](https://github.com/OpenAdaptAI/openadapt-ml/pull/15),
  [`af64fe3`](https://github.com/OpenAdaptAI/openadapt-ml/commit/af64fe3c710b747baab7866a21b1bd87993ab426))

- Move warnings.warn() after imports to fix E402 in viewer files - Remove unused imports (Any,
  base64, os, Service) to fix F401 - Remove f-string without placeholders to fix F541 - Apply ruff
  formatting to 5 files

Files changed (7): - benchmarks/viewer.py - E402 fix - benchmarks/waa_deploy/api_agent.py - F401 +
  format - benchmarks/azure_ops_tracker.py - format only - benchmarks/vm_monitor.py - format only -
  cloud/local.py - format only - scripts/capture_screenshots.py - F401, F541 + format -
  training/viewer.py - E402 fix

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

- **training**: Support VL models in standard transformers fallback
  ([#18](https://github.com/OpenAdaptAI/openadapt-ml/pull/18),
  [`2b2c1df`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2b2c1df49753f9498b793925772f349f4a66c00a))

* fix(training): support VL models in standard transformers fallback

Auto-detect vision-language models (Qwen2-VL, Qwen2.5-VL) and use the appropriate model class
  instead of always using AutoModelForCausalLM.

Detection criteria: - "VL" in model name (case-insensitive) - "vision" in model name - vision_config
  attribute in model config

Model class selection: - VL models: Qwen2VLForConditionalGeneration (with AutoModelForVision2Seq
  fallback) - Text-only models: AutoModelForCausalLM

Also sets task_type to SEQ_2_SEQ_LM for VL models in LoRA config.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(training): simplify VL tests to avoid model downloads

* fix(training): improve VL model support - catch RuntimeError, disable assistant_only_loss

- Add RuntimeError and TypeError to exception handling in _load_standard_model() to catch errors
  when loading Qwen2.5-VL with Qwen2VLForConditionalGeneration - Disable assistant_only_loss in
  standard TRL config as it's not supported for VL models yet

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

### Chores

- Bump version to 0.2.1
  ([`cd969f8`](https://github.com/OpenAdaptAI/openadapt-ml/commit/cd969f87a66379bad37c14e15fdd39386eb8613b))

Includes VL model support fix (PR #18): - Auto-detect VL models and use correct model class - Handle
  Qwen2VLForConditionalGeneration properly - Set assistant_only_loss=False for VL models

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove dead code and legacy fix scripts
  ([#16](https://github.com/OpenAdaptAI/openadapt-ml/pull/16),
  [`6d808ea`](https://github.com/OpenAdaptAI/openadapt-ml/commit/6d808ea481a327027e319112359ece07fc5012b0))

Delete unused files: - training/viewer_migration_example.py (72 lines) - only self-referential -
  scripts/fix_acr_auth.py (212 lines) - one-time fix now baked into setup_azure.py -
  docs/azure_acr_authentication.md - docs for removed script

Update CLAUDE.md to remove references to deleted fix script.

Verified safe to delete: - None of these files are imported by cli.py - fix_acr_auth.py
  functionality is now in setup_azure.py (steps 10-12)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

- Update gitignore and module exports
  ([`81f19d7`](https://github.com/OpenAdaptAI/openadapt-ml/commit/81f19d70af9bf52df566ebe60e9537f4f960404a))

- Add patterns for training output, synthetic data, experiment results - Add .jsonl,
  benchmark_live.json, external/, demos/ to gitignore - Export new runtime and schema types in
  module __init__.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Documentation

- Add architecture decisions, analysis, and design documentation
  ([`178e6e5`](https://github.com/OpenAdaptAI/openadapt-ml/commit/178e6e534674e281a2ab4590e7fd69b0813cdc13))

Key documents: - ARCHITECTURE_DECISIONS.md: Technical direction and decision records -
  analysis_jan2026.md: Comprehensive analysis and strategic options - enterprise/: SAC, Design
  Roadmap, Coords vs Marks ablation research

Design docs: - safety_gate_design.md: Safety gate architecture - perception_integration.md:
  Grounding integration design - representation_shootout_design.md: Coords vs Marks experiment
  design - viewer_consolidation_design.md, viewer_redesign_proposal.md

Experiment results: - waa_benchmark_results_jan2026.md: WAA benchmark analysis -
  grpo_training_report.md: GRPO training experiments - trl_unsloth_integration_analysis.md: Training
  integration analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add ecosystem planning documents
  ([`897dedd`](https://github.com/OpenAdaptAI/openadapt-ml/commit/897dedd46cef94a2e8500850337e7187bdc61bf3))

- github_org_update_plan.md: GitHub org profile update strategy - desktop_app_plan.md: Desktop app
  distribution (pywebview + PyInstaller) - openadapt_integration_plan.md: Core openadapt integration
  roadmap

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add GitHub organization profile content recommendations
  ([`fac58bb`](https://github.com/OpenAdaptAI/openadapt-ml/commit/fac58bb0ab0c6f2b463d86d5139bbbd2f90d84ce))

Add comprehensive recommendations for updating the OpenAdaptAI GitHub organization profile
  including:

- Organization bio (160 char max) - Organization README content for .github/profile/README.md -
  Pinned repositories recommendation (6 repos) - Repository descriptions for each package in the
  modular ecosystem

Focuses on the new modular architecture with openadapt as the unified entry point, highlighting
  openadapt-ml, openadapt-capture, openadapt-evals, openadapt-viewer, openadapt-grounding, and
  openadapt-retrieval packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add Qwen3-VL embedding research and design documentation
  ([`a2e81a0`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a2e81a076db53ac9e83536dcac5f8cf89937c28f))

Add comprehensive documentation for Qwen3-VL vision-language embedding: -
  qwen3_vl_embedding_research.md: Literature review of VLM embedding extraction methods, including
  early exit strategies, hidden state extraction, and multimodal representation learning -
  qwen3_vl_embedding_design.md: Technical design document for extracting and using Qwen3-VL
  embeddings for GUI element retrieval and similarity-based action prediction

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add viewer architecture survey and comparison
  ([`894d03d`](https://github.com/OpenAdaptAI/openadapt-ml/commit/894d03da6f86906a10fe8e49121a4818639a3a28))

Survey of viewer technologies and frameworks for training/benchmark visualization, comparing options
  like Gradio, Streamlit, Panel, and custom HTML solutions for the unified viewer architecture.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add website redesign plan
  ([`1a25483`](https://github.com/OpenAdaptAI/openadapt-ml/commit/1a254837a92bf4ff892a305dc2648678ab46f33c))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Pivot desktop app to uv-first distribution and propose meta-package architecture
  ([`21a05ac`](https://github.com/OpenAdaptAI/openadapt-ml/commit/21a05acf83347b7fed2496ad17009df1e0a0f2c1))

- desktop_app_plan.md: Switch from PyInstaller to uv-based installation - Tier 1: Single command
  install via uv tool - Tier 2: Optional uv bundled installer (~15MB) - Tier 3: PyInstaller full
  bundle (deferred) - Reduces annual cost from $500-700 to $0

- new_openadapt_architecture.md: Propose Option B+ thin CLI wrapper - Create unified 'openadapt'
  meta-package - Re-export common items from sub-packages - Unified CLI (openadapt
  capture/train/eval) - Phase-based implementation over 2 weeks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update openadapt-web repository reference to new name
  ([`ca60945`](https://github.com/OpenAdaptAI/openadapt-ml/commit/ca6094581801c9aab3bd0b8b86a6dfe7e3c3448e))

Update repository link from OpenAdapt.web to openadapt-web following the rename to match the
  lowercase-hyphen naming convention.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Features

- Add safety gate, perception integration, and representation experiments
  ([`5ef0ef8`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5ef0ef8455e97441b963f3ccff3c94fc06c2c63c))

New modules: - runtime/safety_gate.py: Deterministic safety gate for action validation -
  perception/integration.py: Bridge between openadapt-grounding and openadapt-ml -
  experiments/representation_shootout/: Coords vs Marks ablation framework -
  benchmarks/trace_export.py: Export benchmark traces to various formats

Tests: - Reorganize tests from root to tests/ directory - Add integration tests in
  tests/integration/ - Add test_gemini_grounding_imports.py for grounding module

Scripts: - p1_episode_success_ab_test.py: A/B test for demo-conditioned episode success

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add unified baseline adapters for VLM comparison
  ([`9c323db`](https://github.com/OpenAdaptAI/openadapt-ml/commit/9c323db0fbb17beb33b975a2120eece30e014e79))

Implements a provider abstraction layer and unified baseline system for comparing Claude, GPT, and
  Gemini across multiple evaluation tracks.

New modules: - openadapt_ml/models/providers/ - API provider implementations - base.py:
  BaseAPIProvider ABC - anthropic.py: Claude support - openai.py: GPT support - google.py: Gemini
  support

- openadapt_ml/baselines/ - Unified baseline system - config.py: TrackConfig, BaselineConfig, MODELS
  registry - prompts.py: Track-specific prompt templates - parser.py: Response parsing with JSON and
  regex fallback - adapter.py: UnifiedBaselineAdapter main class - cli.py: CLI commands (run,
  compare, list-models)

Tracks supported: - Track A: Direct coordinate prediction - Track B: ReAct-style reasoning with
  coordinates - Track C: Set-of-Mark element selection

Usage: uv run python -m openadapt_ml.baselines.cli list-models uv run python -m
  openadapt_ml.baselines.cli run --model claude-opus-4.5 --track A --image screenshot.png --goal
  "Click submit"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **experiments**: Add representation shootout and SOM evaluation results
  ([`9951c40`](https://github.com/OpenAdaptAI/openadapt-ml/commit/9951c4078edcd1a409a38d82c9667785bf5df3f1))

Add experiment results and artifacts: - representation_shootout results comparing embedding
  extraction methods - qwen_login 2b_dev_fixed plots showing base vs fine-tuned comparison -
  registration_som_eval.json evaluation metrics for SOM-based action prediction

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **waa**: Refactor CLI and fix Python 3.9 compatibility
  ([#14](https://github.com/OpenAdaptAI/openadapt-ml/pull/14),
  [`5557130`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5557130255cc23fa07841ef89520e35dd14f4464))

- Refactor CLI from 6800 to ~1300 lines with flat command structure - Add analyze command to parse
  and summarize benchmark results - Add --num-tasks flag to limit number of tasks to run - Fix
  Python 3.9 compatibility by copying Python from vanilla WAA image (fixes transformers 4.46.2
  compatibility with GroundingDINO) - Add coverage and analysis artifacts to .gitignore

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

### Refactoring

- **benchmarks**: Consolidate to re-export from openadapt-evals
  ([#17](https://github.com/OpenAdaptAI/openadapt-ml/pull/17),
  [`7f171e4`](https://github.com/OpenAdaptAI/openadapt-ml/commit/7f171e41abfc6a3d6065ec2abb11f73e34b11abe))

* docs: add verified repo consolidation plan

- Two-package architecture: openadapt-evals (foundation) + openadapt-ml (ML) - Verified audit
  findings: 10 dead files confirmed, 3 previously marked dead but used - CLI namespacing: oa evals
  <cmd>, oa ml <cmd> - Dependency direction: openadapt-ml depends on openadapt-evals (not circular)
  - Agents with ML deps (PolicyAgent, BaselineAgent) move to openadapt-ml - adapters/waa/
  subdirectory pattern for benchmark organization

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add openadapt-evals as optional dependency

Add [benchmarks] optional dependency for benchmark evaluation: - pip install
  openadapt-ml[benchmarks]

This is part of the repo consolidation to establish: - openadapt-evals: Foundation for benchmarks +
  infrastructure - openadapt-ml: ML training (depends on evals for benchmarks)

* docs(cli): clarify serve vs dashboard command naming

- oa ml serve: serve trained models for inference - oa ml dashboard: training dashboard for
  monitoring

This distinguishes the two use cases clearly: - serve = model inference endpoint - dashboard =
  training progress UI

* refactor(benchmarks): consolidate to re-export from openadapt-evals

Migrate benchmark infrastructure to two-package architecture: - openadapt-evals: Foundation package
  with all adapters, agents, runner - openadapt-ml: ML-specific agents that wrap openadapt-ml
  internals

Changes: - Convert base.py, waa.py, waa_live.py, runner.py, data_collection.py, live_tracker.py to
  deprecation stubs that re-export from openadapt-evals - Keep only ML-specific agents in agent.py:
  PolicyAgent, APIBenchmarkAgent, UnifiedBaselineAgent - Update __init__.py to import from
  openadapt-evals with deprecation warning - Update tests to import from correct locations - Remove
  test_waa_live.py (tests belong in openadapt-evals)

Net: -3540 lines of duplicate code removed

* refactor(benchmarks): delete deprecation stubs, import from openadapt-evals

Remove deprecation stubs since there are no external users. Tests now import directly from
  openadapt-evals (canonical location).

Deleted: - base.py, waa.py, waa_live.py, runner.py, data_collection.py, live_tracker.py

Kept: - agent.py (ML-specific agents: PolicyAgent, APIBenchmarkAgent, UnifiedBaselineAgent) -
  __init__.py (simplified to only export ML-specific agents)

* docs(readme): add WAA benchmark results section with placeholders

Add section 15 for Windows Agent Arena benchmark results with clearly marked placeholders. Results
  will be filled in when full evaluation completes. Warning banner indicates PR should not merge
  until placeholders are replaced.

Sections added: - 15.1 Benchmark Overview - 15.2 Baseline Reproduction (paper vs our run) - 15.3
  Model Comparison (GPT-4o, Claude, Qwen variants) - 15.4 Domain Breakdown

* docs(readme): move WAA benchmark results to openadapt-evals

WAA benchmark results belong in openadapt-evals (the benchmark infrastructure package) rather than
  openadapt-ml (the training package).

See: https://github.com/OpenAdaptAI/openadapt-evals/pull/22

* feat(cli): add VNC auto-launch and --fast VM option

- Add setup_vnc_tunnel_and_browser() helper for automatic VNC access - Add VM_SIZE_FAST constants
  with D8 series sizes - Add VM_SIZE_FAST_FALLBACKS for automatic region/size retry - Add --fast
  flag to create command for faster installations - Add --fast flag to start command for more QEMU
  resources (6 cores, 16GB) - Opens browser automatically after container starts

* docs: add WAA speedup options documentation

- Document --fast VM flag usage - Explain parallelization options - Detail golden image approach for
  future optimization

* docs(readme): add benchmark execution logs section

- Add section 13.5 with log viewing commands - Add benchmark run commands with examples - Renumber
  screenshot capture tool section to 13.6

* docs(readme): clarify --run flag for benchmark execution logs

- Add logs --run command for viewing task progress - Add logs --run -f for live streaming - Add logs
  --run --tail N for last N lines

* docs(readme): add example output for logs commands

- Add example output for `logs` (container status) - Add example output for `logs --run -f`
  (benchmark execution)

* feat(cli): add --progress flag for benchmark ETA

- Add _show_benchmark_progress() function - Parse run logs for completed task count - Calculate
  elapsed time and estimated remaining - Show progress percentage

Example usage: uv run python -m openadapt_ml.benchmarks.cli logs --progress

* docs(research): add cua.ai vs openadapt-ml WAA comparison

Comprehensive analysis of Cua (YC X25) computer-use agent platform: - Architecture comparison
  (composite agents, sandbox-first) - Benchmark framework differences (cua-bench vs openadapt-evals)
  - Training data generation (trajectory replotting) - Recommendations: adopt patterns, not full
  migration

Key findings: - Cua's parallelization uses multiple sandboxes (like our multi-VM plan) - Composite
  agent pattern could reduce API costs - HTML capture enables training data diversity

* feat(cli): add parallelization support with --worker-id and --num-workers

WAA natively supports parallel execution by distributing tasks across workers.

Usage: # Run on single VM (default) run --num-tasks 154

# Run in parallel on multiple VMs VM1: run --num-tasks 154 --worker-id 0 --num-workers 3

VM2: run --num-tasks 154 --worker-id 1 --num-workers 3

VM3: run --num-tasks 154 --worker-id 2 --num-workers 3

Tasks auto-distribute: worker 0 gets tasks 0-51, worker 1 gets 52-103, etc.

* docs(research): add market positioning and strategic differentiation

Expand cua_waa_comparison.md with: - Success rate gap analysis (38.1% vs 19.5%) - Market positioning
  comparison (TAM, buyers, value props) - Where sandbox approach fails (Citrix, licensed SW,
  compliance) - Shell applications convergence opportunities - Bottom line: Windows enterprise
  automation is hard, validates OpenAdapt approach

* docs(waa): add parallelization and scalable benchmark design docs

- Add WAA_PARALLELIZATION_DESIGN.md documenting: - Official WAA approach (Azure ML Compute) - Our
  dedicated VM approach (dev/debug) - When to use each approach

- Add WAA_UNATTENDED_SCALABLE.md documenting: - Goal: unattended, scalable, programmatic WAA -
  Synthesized approach using official run_azure.py - Implementation plan and cost estimates

- Update Dockerfile comments to clarify: - API agents (api-claude, api-openai) run externally -
  openadapt-evals CLI connects via SSH tunnel - No internal run.py patching needed

* style: fix ruff formatting

* fix(imports): update internal code to import from openadapt-evals

Replace imports from deleted benchmark files with direct imports from openadapt-evals:

- azure.py: BenchmarkResult, BenchmarkTask, WAAAdapter - waa_demo/runner.py: BenchmarkAction,
  WAAMockAdapter, etc.

This completes the migration to the two-package architecture where openadapt-evals is the canonical
  source for benchmark infrastructure.

* fix(imports): add missing EvaluationConfig import

- Update azure.py to import BenchmarkAgent from openadapt_evals - Add EvaluationConfig to runner.py
  imports

Fixes CI failure: F821 Undefined name `EvaluationConfig`

* fix(deps): require openadapt-evals>=0.1.1

v0.1.0 uses task ID format "browser_1" but tests expect "mock_browser_001" which was added in
  v0.1.1.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

- **benchmarks**: Migrate to openadapt-evals package
  ([`2e81378`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2e8137830d5e42abfbd2847973684ee9313284ae))

BREAKING CHANGE: Benchmark code moved to openadapt-evals.

- Update CLAUDE.md with migration guide - Add deprecation warning to benchmarks/__init__.py - Old
  imports still work but emit DeprecationWarning

Migration: # OLD (deprecated) from openadapt_ml.benchmarks import WAAMockAdapter

# NEW (preferred) from openadapt_evals import WAAMockAdapter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Testing

- Add unit tests for providers and baselines modules
  ([`4942982`](https://github.com/OpenAdaptAI/openadapt-ml/commit/49429822ad5abc57eec40971eaf13558087255bf))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>


## v0.2.0 (2026-01-09)

### Bug Fixes

- Resolve test failures and SSE dashboard state conflicts
  ([`76fa63c`](https://github.com/OpenAdaptAI/openadapt-ml/commit/76fa63c42c31991016eca9559811cfe11b487ccd))

Test fixes: - test_action_parsing.py: Handle 4-value return from predict_action_from_sample() -
  test_api_adapter.py: Fix mock patch locations (openai.OpenAI, anthropic.Anthropic) - trainer.py:
  Change logger.save() to logger._save_log() - policy.py: Allow negative coords in CLICK regex for
  clamping tests

SSE dashboard fixes: - Add phase: "ready" to Azure VM Host tasks to prevent Starting+completed
  conflict - Improve frontend phase inference from status when phase is missing - Add debug console
  logging for SSE troubleshooting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **cli**: Use localhost for VNC URLs via SSH tunnel
  ([`03b23fc`](https://github.com/OpenAdaptAI/openadapt-ml/commit/03b23fcd08b58dab58d760f60f95f5af568346ae))

Probe output now correctly shows localhost:8006 instead of public IP which is not accessible without
  SSH tunnel.

- **waa**: Add full Python dependencies for benchmark client
  ([`a2cd826`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a2cd8262b2f1e6a4e582ef2f293a4e5878887f78))

- Add build-essential, ffmpeg, and X11 libs for package compilation - Install core packages:
  gymnasium, fabric, transformers, torch (CPU) - Install ML packages: opencv, easyocr, matplotlib,
  accelerate - Create python -> python3 symlink for compatibility - Separate pip installs into
  layers for better caching

- **waa**: Add missing pydrive and other client dependencies
  ([`ebdc4f6`](https://github.com/OpenAdaptAI/openadapt-ml/commit/ebdc4f6a6d266d392a1a981b7b5101147919680d))

- **waa**: Add remaining WAA client dependencies (openpyxl, docx, etc.)
  ([`02e5e2f`](https://github.com/OpenAdaptAI/openadapt-ml/commit/02e5e2f40bec8fd5d880048af0bbfd37c66f612d))

- **waa**: Copy OEM files to Samba share at container startup
  ([`4fb26fe`](https://github.com/OpenAdaptAI/openadapt-ml/commit/4fb26fe2162996b28c2c3e1ce2b54f27eed74230))

Add /copy-oem.sh startup script that copies OEM files from /oem to /tmp/smb (Samba share) at
  container startup. This fixes Windows not finding setup scripts because smb.conf is generated at
  runtime.

Also update experiment doc to remove timeline estimates and add WAA baseline as in-progress.

- **waa**: Copy Python env from official image to avoid 3.13 compat issues
  ([`e5b3dc0`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e5b3dc04997ce1b2a6a86bf69ee09c30aaa3b3b2))

### Chores

- Bump version to 0.2.0 for PyPI release
  ([`3fac13d`](https://github.com/OpenAdaptAI/openadapt-ml/commit/3fac13de1f593676cf3aba896799bca4965db2c2))

Features in this release: - TRL + Unsloth training integration (2x faster, 50% less VRAM) -
  Standardized on uv for package management - Enhanced VM CLI and WAA deployment - Comprehensive
  documentation updates

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Remove old waa/Dockerfile (moved to waa_deploy/)
  ([`e3bd4e3`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e3bd4e3f0ffcaaeaacd02ba2ad6c36f8d359d000))

- Standardize on uv for package management
  ([`7d5c2fe`](https://github.com/OpenAdaptAI/openadapt-ml/commit/7d5c2feaeff9a0f20e136ffae6f7499f0b775379))

- Replace all `pip install` with `uv add` in docs - Update cloud GPU training to use `curl ... | sh`
  for uv install - Update CLAUDE.md with enhanced VM operations guidance - Consistent `uv sync` for
  local development

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update CLAUDE.md and minor fixes
  ([`8dbe154`](https://github.com/OpenAdaptAI/openadapt-ml/commit/8dbe15434b8474552e57e82ca657824b9f1c6efe))

- Updated CLAUDE.md with new features and documentation - trainer.py: minor improvements -
  eval_policy.py: updated for new schema - uv.lock: dependency updates

- Update uv.lock
  ([`6b82505`](https://github.com/OpenAdaptAI/openadapt-ml/commit/6b82505a453f8b972235b82c8cbab689970ba4c0))

### Documentation

- Add benchmark viewer screenshot to README
  ([`e5c0516`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e5c0516171f79ec7504e19329e0eb355d456a721))

- Add capture format decision framework
  ([`c60088f`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c60088f147d966bd7ffe27fb569b6c804aa0460d))

Explores options for data format interoperability: - Option A: Native Episode output from
  openadapt-capture - Option B: Conversion layer in openadapt-ml (recommended) - Option C: Shared
  schema package - Option D: Dual output

Recommends Option B with clear guidelines for the conversion layer. Includes text demo format
  specification for WAA experiment.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add comprehensive capture migration guide with a11y integration
  ([`9307ef6`](https://github.com/OpenAdaptAI/openadapt-ml/commit/9307ef690437c28334d2442a32d15d61253a4d80))

- Add design documents for SSE, retrieval, and parallelization
  ([`fca9c94`](https://github.com/OpenAdaptAI/openadapt-ml/commit/fca9c945836a3a45973ba07de32019fe3dbdc2a2))

- SSE architecture and integration guides - Demo retrieval design and experiments - WAA
  parallelization and live adapter plans - Chrome extension design for capture - Benchmark viewer UX
  improvements

- Add openadapt-capture to openadapt-ml migration plan
  ([`f3d6194`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f3d6194a06844c5c9c4b53ccc72c40010494b9a0))

- Add schema consolidation plan
  ([`b24ea42`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b24ea4260264ccc115147a23e7b68472cd138b9b))

Detailed migration plan for consolidating from two schema modules to one: - DELETE:
  openadapt_ml/schemas/ (dataclass-based, legacy) - KEEP: openadapt_ml/schema/ (Pydantic-based,
  canonical)

Includes: - Dependency analysis (22 files affected) - Field mapping between old and new - 7-phase
  migration strategy - Testing strategy - Rollback plan - Timeline estimate (~8-10 hours)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add staged export hierarchy to enterprise guide
  ([`e6b985b`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e6b985b29f21dfe31a01851b941d6220cd4fedea))

- Episode JSON as canonical, Parquet/WebDataset as projections - Expand data loss table for flat
  formats - Mark exporters as Planned with design doc links - Add multi-step evaluation caveat

- Add WAA demo recording guide for Windows captures
  ([`213ab7b`](https://github.com/OpenAdaptAI/openadapt-ml/commit/213ab7bb7b89294501049b7f71c31b584cdfedc0))

Step-by-step instructions for recording the 3 complex demos: - Task #4: Fill blank cells
  (LibreOffice Calc) - Task #5: Create chart (LibreOffice Calc) - Task #9: Archive folder (File
  Explorer)

Includes setup, recording steps, export, and transfer instructions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Consolidate WAA CLI workflow documentation
  ([`263ca42`](https://github.com/OpenAdaptAI/openadapt-ml/commit/263ca42e97491a4564f98cca2e9862e0051440a5))

- Add Quick Start: Single VM section to azure_waa_setup.md - Replace manual SSH steps in CLAUDE.md
  with CLI commands - Document custom waa-auto Docker image that fixes OEM folder issue - Add vm
  probe, vm reset-windows, and other useful commands

- Strengthen enterprise integration guide positioning
  ([`e6ea3ba`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e6ea3ba1001a3a3707c95a47bcc81c6b2baad0b4))

Add decision boundary, requirements, retrofitting cost sections. Add data portability note
  addressing vendor lock-in concern. Add optional metadata extension pattern. Add typical
  integration workflow. Add open schema independence statement.

- Update README for TRL training and PyPI installation
  ([`a8a6055`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a8a60559c020ccc127bb6375b9de4f95e7e87330))

- Add Installation section with PyPI instructions (uv add openadapt-ml) - Update training section to
  reflect TRL + Unsloth integration - Update repository structure with trl_trainer.py reference -
  Add PyPI badge - Fix section numbering throughout - Update test descriptions for TRL trainer

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Validate demo-conditioning at n=45 (46.7% → 100%)
  ([`49d5a20`](https://github.com/OpenAdaptAI/openadapt-ml/commit/49d5a206bc710a430155e5339107543fa60d9b7d))

- Zero-shot: 46.7% (21/45), Demo: 100% (45/45), Control: 57.8% - Improvement: +53.3 percentage
  points across 15 macOS categories - Add Parquet export design doc (derived analytics format) -
  Update enterprise guide with validated result

- **experiment**: Strengthen demo-conditioning doc for expert review
  ([`d49af59`](https://github.com/OpenAdaptAI/openadapt-ml/commit/d49af596ea96fe058640a2d86c94e8a53af231d3))

- Add interpretation note framing result as "trajectory-conditioned disambiguation" not general
  task-solving - Highlight length-matched control in executive summary - Frame shared first action
  as intentional controlled variable - Add "Positioning Relative to Fine-Tuning" section connecting
  to prompting-first methodology - Expand limitations with actionable specifics (WAA running, SoM
  conventions, episode success vs first-action)

- **schema**: Comprehensive documentation for Episode schema
  ([`b20b879`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b20b8794fc142a619d485022b62c2da56f64450a))

- Add detailed Quick Start with code examples - Document all 24 action types with categories -
  Explain pixel vs normalized coordinate systems - Add validation and format conversion examples -
  Document extension points (raw, metadata fields) - Add docs/schema/README.md as standalone
  reference

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Features

- Add demo-conditioned prompting experiment and retrieval module
  ([`8745752`](https://github.com/OpenAdaptAI/openadapt-ml/commit/874575278f9886afb1acfc1be5508228438193a6))

Validated that demo-conditioning improves first-action accuracy from 33% to 100% (n=3, preliminary
  signal). Key findings: - Benefit is semantic, not token-length (length-matched control: 67%) -
  Demos generalize across task variations (toggle polarity, parameters) - Zero-shot has systematic
  spatial bias that demos correct

Added retrieval module (TF-IDF + domain bonus) for automatic demo selection. Added demo-conditioned
  training mode to train_from_json.py. Added enterprise integration guide for workflow data export.

Statistical note: n=3 is insufficient for significance. Validation at n≥30 on expanded task set in
  progress.

- Add Episode JSON schema and polish benchmark viewer
  ([`c18ae67`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c18ae67eecccee331704fe04abde4b805f92f87a))

Episode Schema (openadapt_ml/schema/): - Pydantic models for Episode, Step, Action, Observation -
  Schema version 1.0.0 with semver evolution policy - WAA format converter (from_waa_trajectory,
  to_waa_trajectory) - JSON Schema export for documentation/tooling - 20 action types (click, type,
  key, hotkey, scroll, drag, etc.)

Benchmark Viewer Improvements: - Fix SSE memory leak (clearAllIntervals on reconnect) - Add
  ThreadedTCPServer for concurrent request handling - Polish UI with color-coded status, loading
  spinners, error banners - Add refresh buttons to all panels with feedback - Prominent VNC button
  with copy-to-clipboard IP

CLI Enhancements: - Add --auto-shutdown flag to deallocate VM after benchmark - Add --timeout flag
  for Azure ML job auto-cancellation - Add vm cleanup-stale command for finding stale jobs/VMs - Add
  refresh button support in Azure Jobs API

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add WAA demo-conditioned experiment with 7 manual demos
  ([`c54b261`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c54b261107600725a5ddcf6f48f73d9ee5729862))

Implements hybrid demo approach for WAA benchmark: - 7 manual demos for simple tasks (settings,
  toggles, linear flows) - 3 placeholders for complex tasks needing recorded demos

Tasks covered: 1. Do Not Track (Edge) - manual 2. Bookmark to bar (Edge) - manual 3. Font size
  (Edge) - manual 4. Fill blank cells (Calc) - needs recording 5. Create chart (Calc) - needs
  recording 6. Center align (Writer) - manual 7. Notifications (Settings) - manual 8. Night Light
  schedule (Settings) - manual 9. Archive folder (Explorer) - needs recording 10. Details view
  (Explorer) - manual

Includes runner CLI for listing tasks and viewing demos.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Enhanced VM CLI and WAA deployment
  ([`1689ab4`](https://github.com/OpenAdaptAI/openadapt-ml/commit/1689ab41a34991099b3313d1f0882cfe4dec172e))

CLI improvements: - Add vm deallocate, start, exec, fix-oem, docker-prune, stop-build actions - SSH
  keepalive settings (60s interval) to prevent timeouts - Docker startup check after VM restart -
  Better probe checking (curl from inside container)

WAA deployment: - Move Dockerfile to waa_deploy/ with api_agent.py - Add api-claude and api-openai
  agent support - P0 demo persistence validation script

Demo persistence validated: - scripts/p0_validate_demo_persistence.py confirms demo included at all
  steps - ApiAgent properly passes demo through multi-step episodes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Trl + Unsloth training integration
  ([`99e34b2`](https://github.com/OpenAdaptAI/openadapt-ml/commit/99e34b2fb0cb98933683da4c3d7e845a644987aa))

- Add trl_trainer.py with SFTTrainer + Unsloth optimizations - Update train_from_json.py to use TRL
  trainer (2x faster, 50% less VRAM) - Remove legacy custom training loop from trainer.py - Add
  [training] optional dependencies (trl, datasets) - Support --use-unsloth / --no-unsloth flags

Training command: uv run python examples/train_from_json.py --data episodes/ --output results/

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **benchmark**: Add Run Benchmark UI panel and fix VNC/Docker issues
  ([`8316bda`](https://github.com/OpenAdaptAI/openadapt-ml/commit/8316bdacbdc4ff7a3eca727de3a26921fffb0365))

- Add Run Benchmark panel to benchmark viewer (model, tasks, agent, domain selection) - Add POST
  /api/benchmark/start endpoint to launch benchmarks from UI - Add --domain and --task-ids CLI flags
  for filtered benchmark runs - Fix VNC link to use localhost:8006 (SSH tunnel) instead of direct VM
  IP - Fix Docker build to use --no-cache --pull to prevent stale dockurr/windows layers - Add
  docs/waa_network_architecture.md explaining the localhost-based network topology - Add
  docs/benchmark_run_ui_design.md with UI design specification

The Docker cache issue caused dockurr/windows v0.00 scripts (no auto-download) to be used instead of
  v5.14 (with auto-download). Fixed by adding --no-cache --pull.

- **benchmarks**: Waa CLI improvements, result analysis, and viewer enhancements
  ([`8483233`](https://github.com/OpenAdaptAI/openadapt-ml/commit/84832335f1cc75203300245f28c0d3fdc49db9c7))

WAA CLI: - Add `analyze` command for programmatic WAA result analysis - Remote analysis via SSH
  (--vm-ip --remote) - Local directory analysis (--results-dir) - Per-domain success rates, JSON
  export - Fix invalid model name: gpt-5.2 → gpt-4o - Add --skip-build tip for faster reruns - Add
  vm_monitor.py for VM status tracking - Add live_tracker.py for real-time benchmark progress

Documentation: - Add docs/waa_setup.md - WAA setup guide - Add docs/GEMINI_GROUNDING_QUICKSTART.md -
  Add docs/background_task_visibility.md - Add implementation summaries

Viewer: - Add benchmark_viewer.py for WAA result visualization - Enhance local.py serve command -
  Integrate benchmarks into unified viewer

- **export**: Add Parquet exporter and toolbox positioning
  ([`e0f93f2`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e0f93f2bd567916ab805b5395378446c5dae25dc))

Add first-class Parquet export support: - to_parquet() / from_parquet() for Episode serialization -
  CLI: python -m openadapt_ml.export parquet --input <dir> --output <path> - Optional summary table
  generation - pyarrow as optional dependency

Update enterprise integration guide: - Add "What is openadapt-ml?" toolbox section - Frame as
  composable utilities, not monolithic framework - Update Parquet section with real implementation
  examples - Scope canonical claim to within openadapt-ml

- **retrieval**: Add demo retrieval system and WAA live adapter
  ([`21d48cf`](https://github.com/OpenAdaptAI/openadapt-ml/commit/21d48cf824a837f03d692c0ed448430d19ad77ae))

Demo Retrieval: - embeddings.py: improved embedding generation with caching - demo_retriever.py:
  semantic search for relevant demonstrations - Support for goal-based and screenshot-based
  retrieval

Benchmark Viewer: - viewer.py: standalone HTML viewer for benchmark results - waa_live.py: live
  evaluation adapter for WAA benchmarks - Integrated with dashboard for real-time monitoring

- **schema**: Add converters between internal and external formats
  ([`f992029`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f992029b4e296f404772a588cad7da56e7109bba))

- from_internal_episode(): Convert schemas.sessions.Episode to schema.Episode -
  to_internal_episode(): Convert schema.Episode back to internal dict format - Document field
  mapping in README

This enables bidirectional conversion between: - Internal training format (schemas.sessions): id,
  goal, t, image_path, x/y - External interop format (schema.episode): episode_id, instruction,
  step_index, etc.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **schema**: Add select_monitor and normalized coordinates
  ([`e398412`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e398412b2ec7e44d8adf71e15170568836afeab8))

- Add action types: select_monitor, window_focus, window_resize, window_move - Add monitor_id field
  for select_monitor action - Add window_title field for window_focus action - Add
  normalized_coordinates (0.0-1.0) as alternative to pixel coords - Add normalized_start/end for
  resolution-independent drag actions

This enables cu-episode-v1 alignment without loss of information.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **waa**: Auto-build Docker image and fix tunnel detection
  ([`0a04998`](https://github.com/OpenAdaptAI/openadapt-ml/commit/0a0499802371781d1a9f4cd18f7ab623676bc07d))

- CLI run-waa now automatically builds waa-auto image if missing - Added --rebuild flag to force
  image rebuild - Dockerfile: fixed IP patching, added playwright for web automation -
  ssh_tunnel.py: fixed tunnel status to check actual port state instead of just internal tracking,
  correctly reports external tunnels

Closes #XX

### Refactoring

- Consolidate schema to single Pydantic-based Episode module
  ([`8c3d7c9`](https://github.com/OpenAdaptAI/openadapt-ml/commit/8c3d7c97de82263e05873ee7da8a5e1abe303d70))

- Migrate from dual schema modules (schemas/ dataclass-based) to single canonical Pydantic schema
  (schema/episode.py) - Delete old openadapt_ml/schemas/ directory (sessions.py, validation.py) -
  Update all imports across 27 files to use openadapt_ml.schema

Schema field mappings (old -> new): - Episode.id -> Episode.episode_id - Episode.goal ->
  Episode.instruction (required), Episode.goal (optional) - Step.t -> Step.step_index (int) +
  Step.timestamp (float) - Step.thought -> Step.reasoning - Observation.image_path ->
  Observation.screenshot_path - Action.x, Action.y -> Action.normalized_coordinates (tuple) -
  Action.type (str) -> Action.type (ActionType enum) - Action.element_index ->
  Action.element.element_id (via UIElement)

Added fields to Observation: app_name, url for benchmark compatibility

Converters in schema/converters.py handle legacy format conversion.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

### Testing

- Add comprehensive tests for WAA demo experiment module
  ([`b5a5d02`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b5a5d02e4b2fb6b3107ee848df5dead339281577))

28 tests covering: - Task definitions (10 tasks, domains, difficulties) - Demo content (7 complete,
  3 placeholder) - Integration (task/demo consistency, retrieval) - Format validation (DEMONSTRATION
  header, Goal line)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add tests for demo retrieval and WAA live adapter
  ([`93565d0`](https://github.com/OpenAdaptAI/openadapt-ml/commit/93565d0cab4263ddbafba66439fcd4c190f39921))

- test_demo_retrieval.py: comprehensive tests for demo retriever - test_waa_live.py: tests for live
  WAA adapter - Updated test_retrieval.py for new embedding features - demo_retrieval_example.py:
  usage examples

- Update tests for TRL trainer refactor
  ([`b17e22d`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b17e22d75ee93fb357049aabb585547e374d721b))


## v0.1.0 (2025-12-16)

### Bug Fixes

- Consistent 'Viewer' label in nav tabs
  ([`5e2f847`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5e2f8470c3b014e1ba7cef34fc2f7978e3ee4387))

- **dashboard**: Improve viewer/dashboard consistency and CLI commands
  ([`a23a881`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a23a88194fbcbd18de5330d28b3de9da8581d597))

- **dashboard**: Remove duplicate job ID and make header full-width
  ([`15260ec`](https://github.com/OpenAdaptAI/openadapt-ml/commit/15260ecc0885c92c66ea96170bac055045f98222))

- **dashboard**: Use subprocess for simpler http server startup
  ([`0f9c9f9`](https://github.com/OpenAdaptAI/openadapt-ml/commit/0f9c9f9bf7e147d6cd1a532506b5083b9c1024e2))

- **plots**: Add model size labels to hardened benchmark plots
  ([`4c828f2`](https://github.com/OpenAdaptAI/openadapt-ml/commit/4c828f20066533f1b0f7b74ac35ce20c46deedec))

- **serve**: Remove stale refresh args, stop button now works
  ([`c190ea5`](https://github.com/OpenAdaptAI/openadapt-ml/commit/c190ea567ac1d8ee7a950d2dc55f8c88288beab4))

- **stub**: Auto-copy real screenshot for evaluation samples
  ([`700f56a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/700f56a60556bd50547cb017588776cb9f2b3139))

- **viewer**: Extract predictions from window.comparisonData and fix elapsed time loading
  ([`afb5bb8`](https://github.com/OpenAdaptAI/openadapt-ml/commit/afb5bb8128d0a48fe956ac54ef339e9eee6aa2e9))

- **viewer**: Sync audio speed with playback, add visual feedback to overlay toggles
  ([`bfafcba`](https://github.com/OpenAdaptAI/openadapt-ml/commit/bfafcba05188a354c3c0cff0c9ab9b9dbfed0505))

### Chores

- **gitignore**: Ignore synthetic and ephemeral training artifacts
  ([`ff3a6e3`](https://github.com/OpenAdaptAI/openadapt-ml/commit/ff3a6e30e406fd534b90c6f07380fd52f09d9933))

- **plots**: Track hardened v2 experiment plots and scope ignore to top-level
  ([`1511e6d`](https://github.com/OpenAdaptAI/openadapt-ml/commit/1511e6d0f69731a2e80ae262b12e53b7ff2d5105))

- **readme**: Point synthetic plots at hardened v2 experiment artifacts
  ([`6f35079`](https://github.com/OpenAdaptAI/openadapt-ml/commit/6f35079a9828b8fdbd267aa83a2d0113950788b9))

### Documentation

- Add benchmark viewer integration design and update TODOs
  ([`f62ba9c`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f62ba9c1ef302bec660a728f73f5c7f1700965f0))

- Add early termination controls as high priority TODO
  ([`454d137`](https://github.com/OpenAdaptAI/openadapt-ml/commit/454d137aa7a15cc695f99b019b70114b4a94903a))

- Document need for auto-termination, dashboard stop button, checkpoint download - Fix shared header
  in unified viewer template (trainer.py) - Remove 'Dashboards:' label from compare.py nav

- Add GUI-Actor integration plan for coordinate-free grounding
  ([`964d92a`](https://github.com/OpenAdaptAI/openadapt-ml/commit/964d92acc2069eb0e6b14c409cb5ab1ac085cfa2))

- Add training feedback UX critical path analysis
  ([`8c8edef`](https://github.com/OpenAdaptAI/openadapt-ml/commit/8c8edef14e2ca37d38a106c47af23bd794d003fc))

- Add unified compute architecture design and PyPI TODO
  ([`db9e008`](https://github.com/OpenAdaptAI/openadapt-ml/commit/db9e008ef8d3b3912c688d5b3312912815274406))

- **readme**: Add 2b training log snippet and clarify qwen3 masking roadmap
  ([`f02ebfd`](https://github.com/OpenAdaptAI/openadapt-ml/commit/f02ebfd557dd01a520ee3f73608da6cc7f0c0039))

- **roadmap**: Mark Priority 5a complete and update plotting achievements
  ([`2da03f2`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2da03f2ee9a1f7780e336323f036b21d293776ca))

- **viewer**: Add timeline visualizer and eval integration design
  ([`27c4130`](https://github.com/OpenAdaptAI/openadapt-ml/commit/27c4130b8753f4fe2817feb4b77c72b3a4a93b4a))

### Features

- Initial commit of openadapt-ml pipeline (synthetic login, qwen adapters, eval + training)
  ([`ec92d6b`](https://github.com/OpenAdaptAI/openadapt-ml/commit/ec92d6b250dc6e4c184e9b2ff07c12781979ba1d))

- V0.1.0 release with benchmark integration, grounding module, and cloud training
  ([`7887890`](https://github.com/OpenAdaptAI/openadapt-ml/commit/7887890a6903aa3f7edb32fcf2e41f53b252a537))

- **benchmark**: Add qwen login orchestrator and refine docs
  ([`efcce00`](https://github.com/OpenAdaptAI/openadapt-ml/commit/efcce008ecf4839857cae154b94b29499d959809))

- **cloud**: Add Lambda Labs training, benchmarks, and training visualization
  ([`7eece80`](https://github.com/OpenAdaptAI/openadapt-ml/commit/7eece80e9974b0b6c5414db9cb597408487cd596))

- **config**: Add pydantic-settings configuration and API benchmarks
  ([`2e7bfd1`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2e7bfd1923b15223e7bf59181244f465cd300d7a))

- **dashboard**: Add early termination controls and /api/stop endpoint
  ([`d6f5df4`](https://github.com/OpenAdaptAI/openadapt-ml/commit/d6f5df4704d45301a7ee9820ea5ba17c538b3c7b))

- **dashboard**: Enhance evaluation samples with model thinking display
  ([`340fe36`](https://github.com/OpenAdaptAI/openadapt-ml/commit/340fe36b6f715aca5480e77709801b2c029541e3))

- **dashboard**: Show model thinking by default, add legend
  ([`5828d67`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5828d6764891261d688fd741dcbbbecc4250369d))

- **docs**: Add dashboard screenshots and fix viewer predictions
  ([`5b729a9`](https://github.com/OpenAdaptAI/openadapt-ml/commit/5b729a9c1f01d304260f266081900f4cae04c8a7))

- **lambda**: Add early termination controls with auto-stop, checkpoint download, and dashboard stop
  button
  ([`1157af8`](https://github.com/OpenAdaptAI/openadapt-ml/commit/1157af89f00041fc0bafe8e468f9edea9e626dc6))

- **lambda**: Auto-symlink capture screenshots and rewrite paths
  ([`40af8db`](https://github.com/OpenAdaptAI/openadapt-ml/commit/40af8dbfb424de02dd7cb3fe18bd0f023853083c))

- **local**: Add local training CLI for CUDA/Apple Silicon
  ([`6edee26`](https://github.com/OpenAdaptAI/openadapt-ml/commit/6edee26b1bf251cac3f61d866867b6e1b839f9f5))

- **plots**: Add legend to comprehensive comparison and streamline README
  ([`b710d49`](https://github.com/OpenAdaptAI/openadapt-ml/commit/b710d4939e4aee97d9553838cf78947c3c99590e))

- **plots**: Add legend to qwen_vs_apis comparison plot
  ([`eee3f25`](https://github.com/OpenAdaptAI/openadapt-ml/commit/eee3f254f2d8900dc5dc92628fc534e7d3d69040))

- **plots**: Update individual plots with consistent color coding and legend
  ([`e934e05`](https://github.com/OpenAdaptAI/openadapt-ml/commit/e934e050cccbffb34cbdaa41fd34f84136d1c784))

- **qwen-login**: Harden benchmark and add plots, GIF, and output docs
  ([`1b29003`](https://github.com/OpenAdaptAI/openadapt-ml/commit/1b290031a0402cb77b5db1aae4c0e0d0bd5a1a17))

- **synthetic-login**: Harden jitter, prompts, and Qwen3-VL 2B/8B results
  ([`9b27555`](https://github.com/OpenAdaptAI/openadapt-ml/commit/9b275558dc77fdcac1abc1ddaf5bacb8f9b9c2d8))

- **training**: Add job-scoped directories and HTTP server for dashboards
  ([`73fb601`](https://github.com/OpenAdaptAI/openadapt-ml/commit/73fb6011ead9690807db497c128186d2c7b36794))

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- **training**: Add stub adapter for rapid UI testing without GPU
  ([`79a95a9`](https://github.com/OpenAdaptAI/openadapt-ml/commit/79a95a91efa0fca926991598f51a8755cb7ebbde))

- **viewer**: Add benchmark tab with WAA integration WIP state
  ([`a31dbf9`](https://github.com/OpenAdaptAI/openadapt-ml/commit/a31dbf98e4cd48faa28ef41ec67e2a0a77640d3a))

- **viewer**: Add parseModelOutput for SoM parsing and truncation
  ([`3396594`](https://github.com/OpenAdaptAI/openadapt-ml/commit/3396594aaac19e9b0ff9e1b79d5eb2070b790721))

- **viewer**: Add screenshots to README and smart auto-scroll
  ([`3a5a256`](https://github.com/OpenAdaptAI/openadapt-ml/commit/3a5a2568888b493ccaab12e3d00242336df650e4))

- **viewer**: Add SoM action parsing for model predictions
  ([`2dd29fb`](https://github.com/OpenAdaptAI/openadapt-ml/commit/2dd29fbe0ac33a1971c1507610372819b29df428))

- **viewer**: Add transcript/audio sync, copy-all button, and extract shared UI
  ([`bd54110`](https://github.com/OpenAdaptAI/openadapt-ml/commit/bd54110b457558ecb4f3a50114cc194d338fede7))

- **viewer**: Extract viewer module with evaluation gallery and badges
  ([`fc003b0`](https://github.com/OpenAdaptAI/openadapt-ml/commit/fc003b06179a33a140840881159a9912f112a7eb))

### Refactoring

- Rename --eval-on-training-data to --overfit
  ([`19b1c94`](https://github.com/OpenAdaptAI/openadapt-ml/commit/19b1c9447031e654ec8bbc68ba0fe0d10785db57))

- **viewer**: Consolidate to standalone HTML generation
  ([`6126aeb`](https://github.com/OpenAdaptAI/openadapt-ml/commit/6126aeb864b4b4529f9df34e1e93758fd8c73475))

### Testing

- **local**: Add tests for local CLI with early stopping
  ([`51f7f32`](https://github.com/OpenAdaptAI/openadapt-ml/commit/51f7f326b4a986467b108971098570498e361a17))
