Metadata-Version: 2.5
Name: harbor-glovebox
Version: 0.2.3
Summary: Run a Harbor trial in a hypervisor-isolated, egress-filtered glovebox microVM
Project-URL: Homepage, https://github.com/AlexanderMattTurner/agent-glovebox
Project-URL: Source, https://github.com/AlexanderMattTurner/agent-glovebox
Author: AlexanderMattTurner
License-Expression: Apache-2.0
Keywords: ai-control,evals,harbor,microvm,sandbox,terminal-bench
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Requires-Python: >=3.12
Requires-Dist: glovebox-driver>=0.8.0
Requires-Dist: glovebox-monitor>=0.4.4
Requires-Dist: harbor<0.24,>=0.23
Description-Content-Type: text/markdown

# harbor-glovebox

Run a [Harbor](https://github.com/harbor-framework/harbor) trial inside a glovebox microVM instead of a shared-kernel container. Each trial boots one hardware-isolated VM behind glovebox's outgoing-traffic filter. The host's records of that trial land in the trial directory, and the host checks the trial's trajectory against its own record of every model call.

## Status

The `harbor-trial` live check drives one trial's environment calls on a real microVM in CI, then runs `harbor run` with Harbor's oracle agent over a one-task dataset through this environment. The `harbor-agent-trajectory` live check runs Harbor's `claude-code` agent on two one-task trials. In one, the harness rewrites the session log in the guest after the agent ends. The check requires the trajectory check below to catch that trial alone.

The daily `eval daily survey` workflow runs every task of the published `terminal-bench@2.0` dataset with the oracle agent through this environment, on KVM runners. The oracle runs each task's reference solution, so a task passes when its verifier awards reward 1 and the trial's stop kept its traffic and model-call records. The oracle calls no model, so the trajectory check reads unchecked, not unbacked. Each task's latest outcome is in [`tools/harbor-measurements.json`](../tools/harbor-measurements.json). No task of that dataset names a `network_mode`, so Harbor gives each one `public`, which this environment refuses (below). The survey therefore rewrites its downloaded copy of each task to `allowlist`, with the hosts in [`bin/checks/vm/harbor-survey-hosts.toml`](../bin/checks/vm/harbor-survey-hosts.toml): the hosts every verifier fetches from, plus that task's own. A task that fails names each host the filter refused.

## Use

Install the released package:

```bash
pip install harbor-glovebox
```

To run the unreleased code on `main` instead, install all four packages from the repository at one revision. That needs `git` on the machine. Set `R` to the same tag or full commit for all four, since two people who install `main` on different days get different code. `--upgrade --force-reinstall` replaces a released sibling package that already satisfies the version pins:

```bash
R=git+https://github.com/AlexanderMattTurner/agent-glovebox.git@<tag-or-full-commit>
pip install --upgrade --force-reinstall \
  "glovebox-hosts @ $R#subdirectory=glovebox-hosts" \
  "glovebox-driver @ $R#subdirectory=glovebox-driver" \
  "glovebox-monitor @ $R#subdirectory=glovebox-monitor" \
  "harbor-glovebox @ $R#subdirectory=harbor-glovebox"
```

Then run a trial:

```bash
harbor run --dataset <dataset> --agent claude-code \
  --env harbor_glovebox.environment:GloveboxEnvironment
```

The host needs what any glovebox launch needs. `glovebox sandbox preflight` checks it, and each trial runs that check before it builds anything.

## Checking the trajectory

Claude Code writes its session log inside the VM, and Harbor builds `agent/trajectory.json` from that log. So the agent can rewrite what its trajectory says it did. <!-- # allow-dangling-path: a path inside a Harbor trial directory --> The host keeps its own record of each request to and reply from `api.anthropic.com`, outside the VM's reach. When a trial stops, this environment copies that record into the trial directory and checks the trajectory against it:

```text
<trial>/glovebox/model-calls.jsonl      the host's record of every model call (a second environment on the trial, such as a separate verifier, writes `glovebox-2/`)
<trial>/glovebox/trajectory-check.json  each claim the record does not back
```

A tool call whose name or arguments no recorded reply asked for is a finding, and so is a recorded tool call the trajectory leaves out, a call it claims twice, and a reply the host could keep only in part. The check judges what the agent ran, not the text it wrote or the results it read. A trial with findings logs a warning and still finishes. The check sets `checked` to false when it had nothing to compare: an agent whose model the host does not record. A record of model calls with no trajectory is a finding. A multi-step task is checked across every step's archived trajectory. Re-run the check on a finished trial with `python -m harbor_glovebox.trajectory_check <trial>`, which exits 1 when the record fails to back a claim.

Two things stay unchecked. The run details Harbor appends after a tool result, such as `[stdout]`, never reach the model, so the record cannot back them. Where the host withheld a secret from its record, any text matches in that place.

These records sit in the trial directory itself, never under `artifacts/`, because Harbor copies the VM's `/logs/artifacts` there.

## Users

A command runs as the `user` Harbor names, else the task's `[agent].user` or `[verifier].user`. With neither named, it runs as glovebox's de-privileged account, never as the image's own `USER`. Before that account's first command, it takes over the image's working directory and what the image's `USER` owned in its home. A task that names the image's own account keeps those files as they are. Every command but root's inherits the image's `ENV`, so the image's `PATH` never picks the binary a root command runs. Harbor names root itself for its own setup steps, such as installing the agent. Every command, root's included, sends its traffic through the in-VM filter and then the host filter outside the VM.

## The model key

The model key never enters the VM. The environment reads the host's `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` or `ANTHROPIC_AUTH_TOKEN` and publishes it to the trial's host-side proxy, which writes it into each request to the model endpoint. Every guest command sees the placeholders `proxy-managed` or `proxy-managed-oauth` instead. An API key wins unless `CLAUDE_FORCE_OAUTH` is truthy and `CLAUDE_CODE_OAUTH_TOKEN` is set, as in Harbor's claude-code agent. `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN` are billed through `x-api-key`, and `CLAUDE_CODE_OAUTH_TOKEN` through `authorization`. A command that Harbor hands a different credential, such as one set with `--ae`, or an `ANTHROPIC_BASE_URL` other than `api.anthropic.com`, fails with an error: export the credential in the host environment instead. A host with none of the three runs the trial unchanged, and a trial whose publish fails does not start.

## What it refuses, and why

- **`network_mode = "public"`.** glovebox has no unfiltered mode. Use `allowlist` and list every host the task needs. Put the agent's model endpoint in Harbor's `extra_allowed_hosts`.
- **An allowlist entry glovebox cannot grant**: an IP address or range, a bare top-level name such as `com`, or a wildcard over a public suffix such as `*.github.io`. The trial's boot refuses it. A public suffix with a dot, such as `github.io`, reaches that one host alone, and `*.example.com` reaches every name below `example.com`.
- **A download larger than 1 GiB.** The guest's archive arrives in the host's memory whole, so the transfer fails rather than grow without bound.
- **A task image that already ships `/logs`, `/tests`, `/solution` or `/harbor`.** The guest's root is read-only, so each of Harbor's fixed trees is a link onto the guest's writable `/tmp` disk, and the image build refuses to replace a tree the image carries.
- **Multi-container tasks** (a `docker-compose.yaml` in the task's environment directory). <!-- # allow-dangling-path: a file in a Harbor task's own directory, not in this repo -->

`storage_mb` is ignored, as Harbor's own Docker environment ignores it.
