Overview
Three items need a decision this week
Portfolio 91 · floor 85 · 43 agents clearRan in monitor mode for 9 days. Of 1,204 checks in the last 24 hours, 38 would have been blocked, 31 of them by pii_in_collections_script. Nothing was blocked for the caller.
Scored 71 against a floor of 85 on the 12 March run. Two obligations failed: disclosure completeness and vulnerable-customer handling.
Registered 47 days ago and has been serving traffic for 41 of them. No certification has ever been attempted.
Mock: the first card is the only new element. It sits in the same decision-led list, uses the same card, button and tone classes, and links to the same Traces and Human review pages.
Guards
| Guard | Mode | Policy set | ControlBook | Last 24h | |
|---|---|---|---|---|---|
collections-guard Owner collections-outreach · 4 policies |
Monitor · day 9 | 9f3c…e1 v7 active · v8 draft |
osfi-e23 v2023.1 | 1,204 checks 38 would be blocked (3.2%) |
Ready to promote |
kyc-guard Owner kyc-onboarding-bot · 6 policies |
Enforce | 41b0…7a v12 active |
osfi-e23 v2023.1 | 1,962 checks 112 blocked (5.7%) |
Active |
wealth-guard Owner wealth-suitability · 3 policies |
Draft | c7d2…09 v1 draft |
osfi-e23 v2023.1 | No checks | Needs 2 approvers · 0 of 2 |
Mock: same table primitive and tone classes as Agents. "Mode" reuses the enforce / monitor words already on the Authorization page. "Policy set" is the pinned fingerprint shown on every decision.
collections-guard
Policies
- Repeating a full account or SIN number back to the caller
- Disclosing the debt to anyone other than the account holder
- Confirming the last four digits the caller just gave
- General questions about payment options
Try it
Runs against draft v8. Records nothing and counts toward no report.
Why it failed (advisory, not signed): a full SIN is restated in the response.
Promote to enforce
Changes what the guard does to real traffic, so it needs two approvers. You drafted v8, so you cannot approve it.
collections-guard
Result on the sealed test cases
Rates with 95% intervals. The cut-off was frozen on the other 640 cases first.
| Measure | v7 | v8 | Change |
|---|---|---|---|
| Precision | 0.90 [0.81–0.95] | 0.93 [0.85–0.97] | not distinguishable |
| Recall | 0.78 [0.68–0.86] | 0.84 [0.75–0.90] | not distinguishable |
| False-positive rate | 0.09 [0.04–0.17] | 0.06 [0.03–0.14] | not distinguishable |
| F1 | 0.84 | 0.88 | paired test needed |
Cannot claim a false-positive rate under 5%. 78 sealed negatives are too few: 5 false positives give an upper bound of 14%. At a 2% observed rate about 210 negatives would be needed.
Confusion matrix, v8
| Flagged | Allowed | |
|---|---|---|
| Should block (82) | 69 | 13 |
| Should allow (78) | 5 | 73 |
By policy
A run is signed with this policy's fingerprint and the dataset hash, so the numbers stay tied to the exact policy they measured.
Mock figures are invented to show the layout. Labelled cases come from Human review decisions, which are already signed.
Traces
| Agent | Timestamp | Result | Guard | |
|---|---|---|---|---|
| collections-outreach | 2026-03-14 09:41:02 | Would block | collections-guard | |
| kyc-onboarding-bot | 2026-03-14 09:40:55 | Blocked | kyc-guard | |
| collections-outreach | 2026-03-14 09:40:31 | Pass | collections-guard | |
| kyc-onboarding-bot | 2026-03-14 09:39:58 | Pass | kyc-guard | |
| collections-outreach | 2026-03-14 09:39:12 | Degraded · blocked | collections-guard |
Checked before the response reached the caller. Guard in monitor mode, so the response was delivered. The text is not in this record.
Pending means the decision is saved and the verdict was returned; it joins the signed chain a moment later.
Governance
Obligation coverage
Every obligation in the ControlBook, and where the evidence for it comes from, in the last 30 days.
| Obligation | Evidence | Pass rate | |
|---|---|---|---|
| E23.3.1 Model inventory completeness | Scheduled runs | 100% | Covered |
| E23.3.2 PII and data governance controls Needs 99% · requirement from the ControlBook |
Guard kyc-guard · Enforce 2 of 2 listed policies attached · 8,744 response checks |
99.8% 112 caught before delivery, not counted as failures | Covered |
| 6F.2.1 Suspicious pattern escalation Needs 99% |
Guard collections-guard · Monitor 1 of 2 listed policies attached · 9,870 response checks |
98.7% Monitor mode: flagged responses were delivered, so they count as failures | Coverage gap |
| E23.5.1 Ongoing performance monitoring | Scheduled runs | 88% | Thin evidence |
The report counts response checks only. In enforce mode a violation the guard caught before delivery is shown as "caught", not as a failed check; in monitor mode it was delivered, so it counts as a failure. The second row also shows the new check that finds a guard missing one of the policies an obligation lists.