Metadata-Version: 2.4
Name: postman-bootstrap
Version: 0.9.2
Summary: AWS Transform agent that bootstraps a Postman validation workspace and release gate from a pre-transform repository
Author: Postman Field Engineering
License: Apache-2.0
Keywords: aws-transform,postman,api-governance,regression-testing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: pyyaml>=6.0
Provides-Extra: agent
Requires-Dist: agent-builder-sdk-aws-transform>=0.1; extra == "agent"
Requires-Dist: agent-builder-agentic-mcp-aws-transform>=0.1; extra == "agent"
Requires-Dist: strands-agents>=1.0; extra == "agent"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: jsonschema>=4.0; extra == "dev"
Requires-Dist: hypothesis>=6.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.6; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"

# postman-bootstrap

An AWS Transform agent that builds the Postman workspace and release gate proving a modernized system still behaves like the one it replaced.

Transform answers "did we modernize the code." This answers "does the modernized system behave correctly, and can you prove it before shipping."

Run it on the **pre-transform** checkout. That ordering isn't optional. Governance and contract tests can be rebuilt from the code any time, but the original system's behavior only survives in what you captured before it changed.

## What you get

Run it against a repo and you end up with a `postman/` directory, a `ci/gate.sh`, and a live Postman workspace holding:

| Thing | What it answers |
| --- | --- |
| OpenAPI 3.1 spec | What is this estate's contract? |
| Governance ruleset (Spectral) | Is that contract well-formed and safe to ship? |
| Workspace governance rules | Does it also satisfy your own team's API standards? |
| Service collection | One request per operation, for humans and agents |
| Contract tests | Does each endpoint honor its own contract? |
| Regression suite | Does the new system match the old one, case by case? |
| Golden corpus | What the pre-transform system actually did |
| 4 environments | `baseline` / `target` / `mixed` / `mock` |
| Mock server + monitor | Test before deploy, catch drift after |
| AWS Transform skill | The gate as a client-side skill, so Transform validates against it as it works |

Any of those failing exits non-zero and blocks the merge. That's the gate.

## The loop it's built for

1. Agent runs on the pre-transform repo. Captures golden output, emits the gate, publishes the workspace.
2. Customer runs AWS Transform.
3. Customer points `target` at the modernized service and runs `./ci/gate.sh`.
4. Red. They fix, re-run, repeat until green.
5. Green means the new system provably does what the old one did. Ship it.
6. They keep the workspace. Spec as the estate's contract, gate in CI, mock for downstream teams, monitor for post-go-live drift.

Step 4 is the whole point. Here's the gate catching a one-cent rounding change in a ported fees module, the kind of thing a port introduces without anyone noticing:

```
$ postman-bootstrap check --repo .
gate FAILED: release blocked
  PASS governance
  SKIP contract
  PASS contract-live
  FAIL regression

--- regression (exit 1) ---
 1.  AssertionError    matches the pre-transform golden output
                       expected { accountId: 'A0004', …(4) } to deeply equal { … }
```

5% of a $900.10 balance is $45.005. The original rounds banker's to $45.00; the port rounds half-up to $45.01. One cent, on every account that lands on that tie, across the estate. Gate exits 1, merge blocked.

## Four ways to run it

**As an AWS Transform agent.** Orchestrator plus 3 subagents on the [agent builder toolkit](https://github.com/awslabs/agent-builder-toolkit-aws-transform), containerized, deployed to Bedrock AgentCore, registered so it can fire on job completion. Needs your account allowlisted for the [composability initiative](https://aws.amazon.com/transform/partners/).

**As a CLI, no model and no AWS.** Given a spec and a case list, building the gate involves zero judgment, so it doesn't need an LLM. Everything in `core/` is a pure function and directly callable:

```bash
pip install -e .   # not on PyPI yet -- docs/adr/0005-packaging-and-distribution.md

postman-bootstrap atx          --repo .   # what a Transform bundle contains, if there is one
postman-bootstrap scan         --repo .   # stack, routes, legacy interface class
postman-bootstrap shim         --repo . --routes shim-routes.json   # if there's no HTTP surface
# --baseline-url is optional: it is inferred from the spec's servers entry.
postman-bootstrap init         --repo . --api-name "Northwind LSC" \
                               --baseline-url http://localhost:4000 \
                               --spec postman/northwind.openapi.yaml
postman-bootstrap capture-plan --repo . --cases cases.json
postman-bootstrap review       --repo .   # the plan, and what it does not reach
postman-bootstrap capture      --repo .   # record golden output from the original
postman-bootstrap infer-schemas --repo .  # turn the corpus into real response schemas
postman-bootstrap build-gate   --repo .   # emit the whole gate
postman-bootstrap publish-workspace --repo .   # mirror it into a live workspace
postman-bootstrap check        --repo .   # run it; non-zero blocks release
```

And after the transform lands:

```bash
postman-bootstrap set   --repo . --target-url https://lsc.internal
postman-bootstrap check --repo .
```

Requires the [Postman CLI](https://learning.postman.com/docs/postman-cli/postman-cli-installation/) and `@stoplight/spectral-cli` on PATH. `POSTMAN_API_KEY` is required for `publish`, and two of the gate's six steps use it at check time as well — the next section is the exact breakdown.

**As two local phases, for the judgment only.** The pre-transform half without a Transform
job around it: read the repository, write the spec, choose the capture cases, stop. Needs
Bedrock access and nothing deployed.

```bash
pip install -e ".[agent,dev]"
postman-bootstrap init --repo . --api-name "Northwind LSC" --legacy-interface http_service
AWS_REGION=us-east-1 postman-bootstrap-author --repo .     # spec, then plan. No capture.
postman-bootstrap review --repo .                          # approve it, or fix it
postman-bootstrap capture --repo . --boot                  # where the system runs
```

Capture stays here on purpose: it talks to the running original, which happens under an
operator who can see what it is pointed at. The two phases carry different tools, so the one
that writes the spec cannot see a verdict and the one that chooses the cases cannot record an
answer -- an agent holding both will, given a red step, rewrite the expectation rather than
fix the port. `docs/standalone-authoring.md` is the design record.

**As an MCP server, driven by AWS Transform.** The same `core/`, over stdio, for a
Transform job to build the gate as it goes:

```bash
pip install -e ".[mcp]"
postman-bootstrap-mcp --repo .        # or set TARGET_REPO_ROOT; what `mcp.json` runs
```

Fourteen tools: scan, initialize, adopt or write a spec, plan a capture (mechanically, or
authored by this agent's own prompt and model), capture, infer schemas, build, status,
interrogate, and three for diagnosing a case that failed.

Two things it deliberately cannot do. **It cannot tell you whether the gate passed** —
that is `ci/gate.sh` in your repository, and its exit code is the answer, because a
verdict reachable through a conversational tool is a verdict that can be renegotiated.
And **it cannot re-record golden output** once a gate has been built from it: capturing
again after the transform lands records the new behaviour as the expectation, which passes
by construction. Re-recording is `postman-bootstrap capture`, deliberately, where it is a
decision someone makes rather than a step in a loop. `docs/adr/0004-mcp-server-surface.md`
argues both.

It also **cannot take a capture plan you wrote.** `author_capture_plan` writes one with this
agent's prompt and model, because establishing the gate is our job and a supplied case set
would mean the system being graded chose what the gate can catch. It needs Bedrock access;
without it, say so and use `propose_capture_plan`, which is mechanical and honest about it.
A route table for a system with no HTTP surface still comes from the CLI or the agent — that
needs the repository read, which is an agentic loop rather than one model call.

## Running without a Postman account

No API key is needed to build the gate or to run most of it. This is also the mode CI lands in whenever the `POSTMAN_API_KEY` secret is absent, so it is worth knowing precisely what it costs.

| Step | Without a key | Why |
| --- | --- | --- |
| `governance` | **runs** | Spectral against the committed spec and ruleset |
| `governance-workspace` | skipped | needs a workspace to resolve your team's rules from |
| `contract` (mock) | **WAIT**, until you decline the key | needs a published mock server |
| `contract-live` | **runs** | committed collection against your deployed target |
| `oracle-integrity` | **runs** | proves the corpus was not edited to reach green |
| `regression` | **runs** | committed corpus against your deployed target |

The regression step — the one the whole engagement rests on — runs with no key at all.

What you give up is real and worth naming. The mock leg is the one that reaches *every* documented status code, because a Postman mock can be told which response to return and a live service cannot; without it, documented 4xx and 5xx paths are exercised only if the deployed system happens to produce them. And `governance-workspace` is what enforces your own organization's API standards, as opposed to the ruleset this agent ships. Neither passes quietly: each carries the reason it did not run.

**Say so once, or every run is red.** `contract` waits on a mock server, and a step waiting on an input nobody supplied and nobody declined is what makes the verdict `INCOMPLETE` -- which exits 1, so the gate is red on every PR. Record the decision instead:

```bash
postman-bootstrap set --decline api_key
```

That says the key is never coming. It cascades -- no key means no workspace, therefore no mock -- so `contract` reports `SKIP` rather than `WAIT`, the verdict can reach `PASSED` once a target URL exists, and the manifest carries who narrowed the claim and to what. (`governance-workspace` is a plain `SKIP` either way: it waits on nothing declinable, so it never held the verdict open.) Declining is how you say "out of scope"; leaving it unsaid means "not done yet". What it must never be used for is `target_url`: that retires the regression step, which is the claim the whole exercise rests on.

Two things you do still need, neither of which is a Postman account:

- **The Postman CLI on PATH.** The gate shells out to `postman collection run`. You never type it — `ci/gate.sh` and the generated workflow do, and the workflow installs the CLI itself — but the binary has to be present.
- **A target URL**, once the modernized service exists. No pre-transform artifact can know where the new system lives, so `postman-bootstrap set --target-url` supplies it. The pre-transform baseline needs no such help: it is inferred from the spec's `servers` entry.

## Regression mode: what your claim is actually worth

The legacy interface decides this, and it isn't a preference. `scan` classifies it and `init` derives the mode:

| Legacy interface | Mode | What you can claim |
| --- | --- | --- |
| Already a service (REST/SOAP/gRPC) | **A** | Live differential. Golden output recorded from the running original, re-recordable any time. Strongest. |
| Batch files, CICS/IMS terminal, MQ, in-process library | **B** | Recorded replay against a frozen snapshot. Strong, but the corpus can't be re-captured once the original is gone. |
| Direct DB / stored procs, no public contract | **C** | Contract conformance only. Materially weaker than behavioral parity, and the agent says so out loud. |

Mode C skips the regression step with an explicit reason rather than shipping an empty suite that looks like coverage. Same for cases that fail to capture: they're dropped from the collection and reported, never turned into a test that asserts nothing.

## Normalization, and why it starts strict

Byte equality is usually wrong. Blanket fuzzy matching is worse. So comparison starts exact and every suppression is a declared decision recorded in the manifest:

```bash
# The one almost every mainframe estate needs. COBOL packed decimal against
# IEEE floats reads as a false regression on monetary fields.
postman-bootstrap normalize --repo . --numeric-tolerance '{"interestRate": 4}'
postman-bootstrap normalize --repo .            # show what's in force
postman-bootstrap normalize --repo . --clear    # back to exact
```

Over-suppress and real regressions slip through, which is exactly the failure this whole thing exists to prevent. The agent is told never to widen a rule to make a run green.

## Layout

```
src/postman_bootstrap_agent/
├─ core/              deterministic, no model, coverage-floored in CI
│  ├─ manifest.py     the run's record: mode, cases, golden output, workspace ids
│  ├─ repo_scan.py    stack + route hints, legacy interface classification
│  ├─ specs.py        OpenAPI → operation list, example synthesis
│  ├─ collections.py  service / contract / regression builders
│  ├─ capture.py      boot, replay, record, write fixtures
│  ├─ shim.py         HTTP facade generator for surfaces that don't exist
│  ├─ schemas.py      infer response schemas from the captured corpus
│  ├─ atx.py          read an AWS Transform output bundle
│  ├─ governance.py   the Spectral ruleset
│  ├─ gate.py         step assembly, rendering, execution
│  └─ emit.py         write the postman/ tree
├─ tools/             Strands tool surface over core/, sandboxed to the repo
├─ subagents/         spec_author, golden_capture, gate_builder
├─ mcp_server.py      MCP surface over core/ for AWS Transform; no verdict, no re-record
├─ orchestrator.py    the runbook, wired up
├─ postman_client.py  Postman API (workspace, spec, collections, mock, monitor)
└─ cli.py             the no-model path
```

The split matters. Judgment lives in the prompts (what the API surface is, which cases are worth capturing, which differences are legitimate). Everything mechanical lives in `core/`, so the same inputs always produce the same gate and it's all reviewable in a diff.

## Deploying it as a Transform agent

Two halves, and only one of them currently works. `docs/publishing.md` is the
full runbook; this is the summary.

**Deployment works.** Verified end to end in AWS account `780401591112`:

```bash
./scripts/validate.sh --deploy          # strict; refuses a placeholder accountId
docker build --platform linux/arm64 -t postman-validation-bootstrap:latest .
./scripts/deploy.sh                     # ECR repo, login, build, push
```

Publishing happens from `432983378316` (Postman production), not from the labs
account the work was done in — `RegisterAgent` binds the agent to whoever calls
it, permanently. `deploy.sh` passes the calling account to `validate.sh`, which
refuses to proceed if it disagrees with the publisher named in `agent-card.json`.
For a sandbox image push, say so: `ATX_ALLOW_ACCOUNT_MISMATCH=1`.

Then create the runtime with the execution role from `infra/roles.cfn.yaml`
(or reuse an existing `AgentCoreExecutionRole`, which is what the AWS dev guide
prescribes):

```bash
aws bedrock-agentcore-control create-agent-runtime   --agent-runtime-name postman_validation_bootstrap   --agent-runtime-artifact '{"containerConfiguration":{"containerUri":"<ecr-uri>:0.1.0"}}'   --role-arn arn:aws:iam::<account>:role/AgentCoreExecutionRole   --network-configuration '{"networkMode":"PUBLIC"}'   --environment-variables '{"REGION":"us-east-1","STAGE":"prod","TARGET_REPO_ROOT":"/workspace/repo"}'
```

Invoking it returns a well-formed JSON-RPC error rather than a crash, which is
the container contract working:

```json
{"jsonrpc":"2.0","error":{"code":-32603,
 "message":"Missing required initialization context in JSON-RPC request"},"id":"1"}
```

The agent initializes only with a real Transform job context: `workspaceId`,
`jobId`, `agentInstanceId` and an `authorizationToken` that Transform issues.
None of those can be fabricated, so everything downstream of initialization —
A2A delegation, HITL, worklogs, artifact upload — needs a live job.

**Registration does not work without the allowlist.** Check before you build
anything, because registration is the last step and the slowest thing to
discover:

```bash
./scripts/check_allowlist.sh <aws-profile>
```

A `403 AccessDeniedException` with an **empty message body** is account-level
allowlisting rather than IAM — an IAM denial names the action it refused. That
is a conversation with your PDM, not a policy you can fix. `docs/publishing.md`
has the ask, written out and ready to send.

Worth knowing before you plan around it: AWS's own developer guide (v0.3, beta
preview) ends with a section called *"Publish Agent to be consumed in customer
account"* whose body reads *"Instructions would be added later."* Being
allowlisted may unblock registration and still leave the customer-facing path
undefined.

One trap the probe works around: botocore does not ship the registry service
model, so the toolkit's own client raises `UnknownServiceError` before it makes
a request. The model is in the toolkit at `knowledge/data/registry_api.json`,
and the script stages it into a temporary `AWS_DATA_PATH`.

## Contributing

`develop` is the integration branch. Branch off it, PR back into it. `main` only
moves at a release. See [CONTRIBUTING.md](CONTRIBUTING.md).

## Tests

```bash
pip install -e ".[dev]"
pytest
```

The suite needs no AWS account and no Postman key. The 2 governance tests that shell out to Spectral skip if it isn't installed, and they're the ones worth having: they lint a clean spec (expect 0 errors) and a deliberately sloppy one (expect 12 specific rules to fire). A ruleset that loads but flags nothing is the failure mode you can't catch by reading YAML.

`validate.sh` also runs `postman spec lint` when the Postman CLI is on PATH, as a second opinion from the linter your team sees in Spec Hub. One Postman-side false positive is filtered — the CLI rejects `application/problem+json` under OpenAPI 3.1 — and `scripts/speclint.py` records exactly which finding and why, bounded so it can't hide a real one.

## Known limits

- Route extraction is regex over Express, Fastify, FastAPI, Flask, and Spring. It misses dynamically registered routes and router mounts that rewrite prefixes, which is why it reports a confidence level and the spec author is told to verify rather than trust it.
- Endpoint pairing is manual when the transform doesn't hand over a legacy-to-new map. One COBOL transaction can fan out into several endpoints, or several can collapse into one. N:M mapping isn't free and the agent raises a HITL task for it.
- `capture` executes a shell command to boot the original. It only ever runs a command a human put in the manifest, never one it inferred.
- Contract tests can only force specific status codes against a Postman mock. A live service ignores `x-mock-response-code`, so against `target` they assert documented-status plus schema only.
