Trust agent signers and set regression limits

Keep the gates separate: trusting a public key does not accept an agent. A valid signature does not prove that an agent is correct. Passing admission does not accept a plan. Accepting a plan does not apply it.

What each check establishes

Check	What it establishes
Artifact SHA-256	The model, knowledge, training, benchmark, and history files still match the manifest.
Ed25519 signature	The trusted private-key holder signed this exact agent ID and manifest fingerprint for a bounded time.
Corpus validation	Studio independently parses the actual model and both CSV files, checks exact headers and row bounds, requires complete matching label coverage, and rejects normalized training/benchmark overlap.
Admission thresholds	The signed manifest records sufficient benchmark quality and no excessive regression against the last accepted version.
Independent metric replay	The packaged Perl evaluator recomputes predictions from `model.json` plus `benchmark.csv`. In this release it is an explicit operator check, not an automatic Studio admission step.
Named review	An accountable person judged the bundle or plan suitable for this workspace.

1. Create a signing identity once

Run this outside the review-bundle directory. The command prompts twice for a password and creates a password-encrypted private key plus a shareable public-key document. Existing files are never overwritten.

vfs-federation-agent-actions keygen `
  --label "VFS agent release signer" `
  --private-key "C:\VFS-Keys\agent-release-signer.pem" `
  --public-key "C:\VFS-Keys\agent-release-signer.public.json"

Keep the .pem private key outside workspaces, bundles, source control, cloud-sync folders, and exported packages.
Share only the .public.json file and its printed key ID.
Confirm the complete key ID through a separate trusted channel before trusting it.

2. Trust the public key in a workspace

Open the workspace and choose Chronicle Center → Agent trust.
Enter the accountable person or role and a short note describing how the key ID was confirmed.
Choose Trust public key and select the .public.json file.
Compare the displayed complete key ID with the independently confirmed value, then choose Trust verified key.

The default workspace trust expires after 365 days. The command-line equivalent accepts an earlier explicit UTC expiry:

vfs-federation-agent-actions trust-key `
  federation.vfsworkspace.json `
  agent-release-signer.public.json `
  --authority "Release security reviewer" `
  --note "Key ID confirmed by voice call" `
  --expires-at 2027-09-22T00:00:00Z

3. Sign one retained agent bundle

Train the branch with --bundle-output, inspect that directory, and then sign it. A signature lasts 90 days by default and can never exceed 366 days. A bundle with an existing signature.json is immutable; train into a new versioned directory instead of replacing it.

vfs-federation-agent-actions sign-bundle `
  "C:\VFS-Demo\review-bundles\workspace-planner-v2" `
  --private-key "C:\VFS-Keys\agent-release-signer.pem" `
  --expires-days 90

4. Verify trust and admission before registration

vfs-federation-agent-actions bundle-inspect `
  "C:\VFS-Demo\review-bundles\workspace-planner-v2" `
  --workspace "C:\VFS-Demo\federation.vfsworkspace.json"

Continue only when trust_status is trusted, every value in corpus_validation reports a match with normalized overlap 0, and admission.accepted is true. Record the bundle fingerprint, signer key ID, signature expiry, measured accuracy, benchmark digest, row counts, label order, and regression result in the review.

Replay the measured accuracy before signing off

From the generated Perl distribution, write replay evidence outside the immutable bundle:

New-Item -ItemType Directory -Force "C:\VFS-Demo\review-evidence" | Out-Null
perl .\bin\plnn-text-evaluate `
  --model "C:\VFS-Demo\review-bundles\workspace-planner-v2\model.json" `
  --data "C:\VFS-Demo\review-bundles\workspace-planner-v2\benchmark.csv" `
  --json `
  --output "C:\VFS-Demo\review-evidence\workspace-planner-v2.metrics.json" `
  --predictions "C:\VFS-Demo\review-evidence\workspace-planner-v2.predictions.csv"

The command must exit successfully. Its rows and accuracy must equal the values bound by the signed manifest; inspect its confusion matrix and per-class results as well. A mismatch is a stop condition. Studio 1.1.5 does not yet run this evaluator inside admission, so passing bundle-inspect alone must not be described as an independently replayed accuracy result.

5. Set admission thresholds

The initial workspace policy requires at least 80.00% accuracy, permits at most 2.00 percentage points of regression, requires at least 8 independent benchmark rows, and always requires an independent benchmark. In Agent trust, adjust the three numeric thresholds and choose Save thresholds.

Permanent safety rails prevent lowering accuracy below 70%, allowing more than 5 percentage points of regression, using fewer than 8 benchmark rows, or disabling the independent benchmark requirement.

vfs-federation-agent-actions policy-set federation.vfsworkspace.json `
  --minimum-accuracy 0.90 `
  --maximum-regression 0.01 `
  --minimum-benchmark-rows 40 `
  --authority "Model quality owner" `
  --note "Production reconfiguration threshold"

How regression comparison works

The first accepted signed bundle for an agent ID establishes its baseline.
A replacement using that agent ID must retain the same ordered labels and exact benchmark digest.
The new measured accuracy may fall only by the configured number of percentage points.
If labels or the benchmark set intentionally change, use a new agent ID and review it as a new baseline.
Tightening policy immediately affects pending and previously accepted agents before any later plan acceptance or application.

6. Revoke a compromised or retired key

Select the key in Chronicle Center → Agent trust.
Enter the accountable authority and the incident or retirement reason.
Choose Revoke selected key and confirm.

Revocation is retained permanently. Every accepted agent signed by that key is blocked immediately from proposal acceptance and application. Create and verify a new signing identity; do not restore the revoked key.

Troubleshooting

Message	Safe response
Signer is not trusted	Confirm the public-key ID through a separate channel, then trust that public key in this workspace.
Signature is missing, invalid, or expired	Do not edit `signature.json`. Produce a new immutable bundle and sign it with an active trusted key.
Signer has been revoked	Stop using every bundle signed by that key. Investigate, rotate the key, and create a newly signed bundle.
Accuracy is below the floor	Improve training or use a more suitable specialist; never lower policy merely to admit a failing bundle.
Regression exceeds the limit	Compare the same held-out failures, improve the model, and train a new bundle. Do not replace benchmark questions.
Labels or benchmark changed	Use a new agent ID so the different task receives a fresh, explicit baseline review.
Corpus label coverage or leakage failed	Correct the training source and held-out questions, then train a new bundle. Do not change manifest counts or digests by hand.
Metric replay differs from the manifest	Stop. Confirm model and benchmark digests, evaluator version, label order, and row count; retain both results for investigation.