Metadata-Version: 2.4
Name: oscal-validate
Version: 0.5.0
Summary: Deterministic structural validator for OSCAL documents, run against NIST's published schema and constraint layer.
Author: Chelsea Kelly-Reif
License: Apache-2.0
Project-URL: Repository, https://github.com/ChelseaKR/oscal-validate
Project-URL: Issues, https://github.com/ChelseaKR/oscal-validate/issues
Project-URL: Changelog, https://github.com/ChelseaKR/oscal-validate/blob/main/CHANGELOG.md
Project-URL: OSCAL (NIST), https://pages.nist.gov/OSCAL/
Keywords: oscal,nist,fedramp,stateramp,compliance-as-code,security-compliance,ssp,poam,json-schema,metaschema,validation,validator,offline,deterministic
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: ai
Requires-Dist: anthropic>=0.125; extra == "ai"
Provides-Extra: bedrock
Requires-Dist: anthropic[bedrock]>=0.125; extra == "bedrock"
Dynamic: license-file

# oscal-validate

[![CI](https://github.com/ChelseaKR/oscal-validate/actions/workflows/ci.yml/badge.svg)](https://github.com/ChelseaKR/oscal-validate/actions/workflows/ci.yml)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)

Deterministic structural validation for
[OSCAL (Open Security Controls Assessment Language)](https://pages.nist.gov/OSCAL/)
documents, meant to run before you hand a package to anyone. Point it at a
catalog, profile, SSP, component definition, assessment plan, assessment
results, or POA&M, and it checks the things a publisher can get wrong silently:
required structure, identifier format and uniqueness, and whether the
references in the document resolve to anything.

The validation command makes no network call and no model call, ever. Same
input, same output, byte for byte. Every finding cites the published rule it
came from, with the source and the date that source was retrieved. Four
opt-in commands added under [ADR-0005](docs/adr/0005-ai-at-the-edges.md)
do call a model; they are separate subcommands, they change nothing about
the one above, and they are described in
[AI-assisted explanation and repair](#ai-assisted-explanation-and-repair-opt-in).

## Structural conformance is not evidence that a control is implemented

**This tool cannot tell you whether a control is implemented, whether a system
is secure, or whether a package would be authorized. It does not try.** It
reads a document and checks it against NIST's published schema and constraint
layer. A clean run means the document is well formed and its references resolve.
It means nothing at all about the system the document describes.

The distinction matters most in the direction people get wrong. A control this
tool has never heard of, a constraint it did not evaluate, a reference into a
document it was not given: every one of those is reported **UNVERIFIABLE**, and
UNVERIFIABLE is never rendered as a pass. A clean report always lists what was
not checked, alongside what was.

**Status:** Beta. Tagged `v0.1.0`, `v0.2.0`, `v0.4.0` and `v0.5.0`. `0.5.0` is the
current release, cut on 2026-10-19 from the changelog section of the same name.
`0.3.0` was prepared in the source on 2026-09-02 and never cut, which is why the
numbering skips it. Nothing is published to PyPI, so
installation is from source. This is a demonstration and reference
implementation. It is not affiliated with, endorsed by, or reviewed by NIST,
FedRAMP, or StateRAMP.

```console
$ oscal-validate my-catalog.json
model: catalog

ERROR        REFERENCE_UNRESOLVED  at=/catalog/groups/16/controls/23/parts/2/parts/0/parts/4/links/0/href
    href = #ac-2_smt.a.5
    This names an OSCAL object by its identifier, in this document's effective
    data model, and no such identifier is declared in the documents supplied.
    Every document named by an import was supplied, so the effective data model
    is complete and this reference resolves to nothing.
    rule: NIST, "URI Usage", section Linking to another OSCAL object ...
    source: https://pages.nist.gov/OSCAL/learn/concepts/uri-use/ (retrieved 2026-08-14)
```

Exit code 0 when there are no ERROR findings, 1 when there are, 2 when the
input cannot be read at all. `--format json` produces machine-readable output
with the same content, and `--format sarif` the same findings as SARIF 2.1.0
(see [Output formats](#output-formats)).

## Why this exists

OSCAL's JSON Schema expresses shape and datatypes. Checked against the vendored
1.2.3 schema, it contains no `uniqueItems`, no `const`, no `if`, no `not`, and
no `dependentRequired`, and the string "unique within" appears zero times. It
cannot say that two controls must not share an id. It cannot say that a link's
`#` fragment has to name something that exists.

Those rules do exist and NIST does publish them, in two places the schema does
not reach:

1. **The Metaschema constraint layer**, shipped beside the schema in the
   `*_metaschema_RESOLVED.xml` release artifacts. OSCAL 1.2.3 declares 340
   constraints there: 48 `is-unique`, 20 `index`, 24 `index-has-key`, 11
   `has-cardinality`, 25 `matches`, 12 `expect`, and 200 `allowed-values`. None
   of them survive into the JSON Schema.
2. **Prose rules** in NIST's own documentation: that a UUID identifies exactly
   one object, and that a bare `#` fragment must resolve within the document's
   *effective data model*.

This tool reads all three layers. Everything it knows comes out of vendored,
hash-pinned copies of NIST's own files; there is no hand-written model of OSCAL
in this repository.

## Install and run

Python 3.12+, no runtime dependencies.

```sh
pip install .

# Two documents ship with this repository, so the first run needs nothing else
# downloaded, and shows both sides of the gate:
oscal-validate tests/fixtures/clean_catalog.json    # 0 ERROR, exit 0
oscal-validate tests/fixtures/broken_catalog.json   # 3 ERROR, exit 1

oscal-validate <file.json> --format json
oscal-validate <file.json> --format sarif
oscal-validate <file.json> --format html > review.html   # one self-contained page
oscal-validate --report-schema                      # the shape that report conforms to
oscal-validate my-ssp.json --resolve baseline-profile.json --resolve catalog.json
oscal-validate my-ssp.json --resolve catalog.json --suggest

# What changed between two runs. No model, no network, exit 0 either way.
oscal-validate diff before.json after.json

# Serve this validator to an assistant, read-only and offline (stdio, MCP).
oscal-validate mcp --root ./packages

# Validate a directory as one deliverable, every member against every other.
oscal-validate package ./my-package
```

`--resolve` takes further OSCAL documents, or a directory of them. It is how an
imported catalog or profile gets into the picture, and it is the difference
between a definite answer and an honest "cannot tell" (see
[The effective data model](#the-effective-data-model)). Nothing is ever
fetched.

### Output formats

`text` is the default. `json` is the canonical machine-readable report: every
finding with its code, severity, JSON Pointer, property, value, message, and
rule citation with source URL and retrieval date, plus a summary by severity;
`tests/golden/` pins its bytes. `sarif` renders the same findings, in the same
order, as [SARIF 2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html)
and adds nothing to them. ERROR and WARNING are `kind: fail`; UNVERIFIABLE is
`kind: open`, SARIF's own word for "evaluated and not settled"; INFO is
`kind: informational`; no result is ever `kind: pass`, and a clean document
still has results, because the constraints that were not evaluated are among
them. Each code is a rule whose `helpUri` is the citation URL when one URL
covers every finding under it, and every result carries its own citation,
source, and retrieval date in `properties.rule`. The location is the document
plus the JSON Pointer; no line number is reported, because none is tracked.
Non-`fail` results carry `level: note` rather than the specification's
`none`, deliberately: GitHub code scanning ignores `kind` and does not render
`none`, and an UNVERIFIABLE finding that disappears there would be an absence
rendered as a pass (the reasoning is in `src/oscal_validate/sarif.py`). The
output validates offline against the OASIS schema vendored in `tests/sarif/`.

`html` is one self-contained page for the people who sign packages and do not
run a CLI, written to stdout like the others:

```sh
oscal-validate my-ssp.json --resolve catalog.json --format html > review.html
```

It carries every finding the text report carries, grouped in the fix order
`walkthrough` computes — from the same module, so a reviewer's page and the
narrative cannot place a finding differently — with the JSON pointer, the
value, the message, and the rule citation as a link to NIST's page with the
date that page was retrieved. UNVERIFIABLE keeps its own section and the
summary states the count on the page; nothing is folded into a pass.

Four properties, each checked rather than asserted by
`tests/test_html_report.py`:

- **Self-contained.** One file. No script, no external stylesheet, no font, no
  image, no `src` of any kind. It opens with the network off and requests
  nothing; the only external URLs are the citation links a reader may follow on
  purpose.
- **Deterministic.** No timestamp and nothing else that moves between two runs
  over the same inputs. The footer names the tool version, the OSCAL release,
  the documents read, and the SHA-256 of every vendored file this run opened.
- **Accessible, mechanically.** See the Accessibility row in
  [Standards Conformance](#standards-conformance) for the seven rules and how
  each is proved able to fail.
- **Escaped.** OSCAL documents are untrusted input, so every value from the
  document goes through `html.escape` with quoting on. There is an injection
  test.

### The JSON report is a published contract

`--format json` is what the GitHub Action, the survey harness, and any
pipeline read. Its shape is published as a JSON Schema (draft 2020-12), shipped
inside the package and printed by `oscal-validate --report-schema`, and every
report carries the `report_schema_version` it conforms to. That version is the
schema's, not the tool's, and the two move independently;
[docs/API.md](docs/API.md) says which kind of change moves which part of it,
and names the library surface with the same promise.

Read counts out of `summary` by key, never with a default:

```python
errors = report["summary"]["ERROR"]  # yes
errors = report["summary"].get("ERROR", 0)  # no
```

Every severity is always present, including the ones that are zero, so that a
missing key is a broken report rather than a count of none. `additionalProperties`
is false throughout the schema for the same reason: a consumer that validates
what it reads learns about a new key instead of passing over it.

### `diff`: what changed between two runs

```console
$ oscal-validate diff tests/fixtures/clean_catalog.json tests/fixtures/broken_catalog.json
added: present after, absent before (3)
  ERROR        REQUIRED_PROPERTY_MISSING  at=/catalog/metadata
      last-modified = (absent)
      rule: ...
removed: present before, absent after (0)
  (none)
...
5 unchanged. A removed finding is a finding this run did not report; it is not
evidence that it was fixed.
```

Each side is an OSCAL document, validated on the spot with its own
`--resolve-before` / `--resolve-after` set, or a saved `--format json` report.
`--format json` for machine use. The exit code is 0 whether or not anything
changed, because a diff is data rather than a verdict; `--fail-on-new` exits 1
when an ERROR is present after and absent before.

Two identical findings are the same finding when their code, location,
property and rule citation match. A finding whose value or message moved under
that identity is *changed*. A finding that differs only in location — an array
gained an element and every pointer under it shifted — is *moved*, but **only
where exactly one was removed and exactly one added** under the same code,
property, value and rule. Where several were, which went where is a guess, so
the pairing is declined and said to be declined.

Two things it deliberately does not say:

- **A removed finding is not a resolved one.** Two finding lists cannot tell a
  repair from a run that read a different document, used a different resolve
  set, or could not get far enough to report anything. `repair --draft` may say
  "resolved" because it made the edit itself and knows what changed; a diff
  over two files does not, so it says *removed* and says what that is worth.
- **A saved report does not record which vendored snapshot produced it.** It
  records the tool version and nothing else about the run, so two reports can
  be compared with no way to know whether the same schema and constraint layer
  was behind them. That is printed as unknown in the header rather than passed
  over, and a tool-version mismatch is printed too. Neither stops the diff:
  they make it something the reader has to interpret, which they can only do if
  they are told.

The comparison is `oscal_validate.compare`, which is also what `repair --draft`
runs, so the verb and the draft cannot disagree about what "introduced" means.

### `rule`: the citation trail, with no model and no network

A finding names a rule. `rule` is how you read that rule without taking anyone's
word for it, and without an SDK, an API key, or a document:

```console
$ oscal-validate rule oscal-catalog-controls
constraint: oscal-catalog-controls

kind: index
level: ERROR
declared on: catalog
target: //control
applies to: catalog
key fields: @id
index: catalog-controls
evaluated: yes

declaration: vendor:oscal_catalog_metaschema_RESOLVED.xml
    source: https://github.com/usnistgov/OSCAL/releases/tag/v1.2.3 (retrieved 2026-08-14)
    sha256: 775f8326e3dac336be17c4f7eefa89661053230fb1e8538364186c927ee062b1

    <index id="oscal-catalog-controls"
                     name="catalog-controls"
                     target="//control">
                <key-field target="@id"/>
             </index>

specification: metaschema-constraints, section Constraints/Constraint Types/index Constraints
    source: https://pages.nist.gov/metaschema/specification/syntax/constraints/ (retrieved 2026-08-21)
    sha256: c51a84834e17d51a5abbc1f2cab0db0653305de54522314d2c65aa7e2a3abc6b

    The <index> constraint is a type of Metaschema constraint that defines an index
    of document instance nodes addressable by key.
    ...
```

It takes either a NIST constraint identifier or one of this tool's finding
codes. For a finding code it prints every rule that can produce it, quoting the
fixed ones verbatim and saying plainly which are composed per finding:

```console
$ oscal-validate rule CONSTRAINT_NOT_EVALUATED
finding code: CONSTRAINT_NOT_EVALUATED
reports: one report per constraint kind this tool did not evaluate, ...
...
constraint kinds this tool does not evaluate:

  allowed-values: 200 published, not evaluated
     because which values a value node permits is decided by the applicable set of
     constraints sharing its target, and this tool does not resolve that set
...
```

`--format json` for machine use; an identifier that is neither a constraint nor
a finding code is exit 2 with nothing printed, because a partial answer about
provenance is worse than none.

Four things about it:

- **It is the evidence `explain` gathers, before `explain` calls anything.**
  Both read one source layer, `oscal_validate.sources`, so the offline answer
  and the model-backed one cannot cite different bytes.
- **Nothing is paraphrased.** Every passage is verbatim from a hash-pinned file,
  printed beside the SHA-256 of the bytes this run read and the date they were
  retrieved. The hash is computed from the file, not copied from a manifest.
- **For a constraint this tool does not evaluate, it prints that constraint's
  own reason** — not its kind's summary — which is the answer to "why did this
  not fire", in one command instead of a search through
  [docs/CONSTRAINT-COVERAGE.md](docs/CONSTRAINT-COVERAGE.md).
- **It reaches no model and opens no socket**, so the GitHub Action and anyone
  who will not send a document to a model can use it. The suite runs it with
  `socket` removed and, in a fresh interpreter, asserts that no module of the
  AI layer or any SDK was loaded.

### `mcp`: the validator, served to an assistant read-only

[ADR-0005](docs/adr/0005-ai-at-the-edges.md) puts the model at the edges and
keeps the validator the only source of findings. `oscal-validate mcp` is the
same boundary from the other side: instead of this tool calling a model, a
model's host calls this tool over
[MCP](https://modelcontextprotocol.io), and what comes back is the
validator's own report rather than a model's account of one.

```sh
oscal-validate mcp --root ./packages
```

It speaks JSON-RPC 2.0 on stdin and stdout and offers four tools:

| tool | answers |
|---|---|
| `validate` | one document, with optional `resolve` paths, returning **the exact bytes `--format json` writes** and the exit code that run would have produced |
| `rule` | the citation trail for one constraint identifier or finding code — the same answer `oscal-validate rule --format json` prints |
| `coverage` | which of NIST's published constraints this tool evaluates, with a specific reason for every one it does not, and the evaluated constraints that read an index nothing builds |
| `limits` | what a clean run does not mean, generated from this README's [Limits](#limits) section |

Four things about it:

- **It takes no dependency.** Not an MCP SDK, not a transport library: the
  protocol is newline-delimited JSON-RPC over stdio and the standard library
  is enough. `dependencies = []` is untouched, so the no-network claim is
  still checkable by reading the source, which is how this project checks it.
- **Read-only and offline is the whole point.** No tool fetches, writes a
  file, or calls a model. The suite runs every one of the four with `socket`
  removed, and asserts in a fresh interpreter that no module of the AI layer
  and no SDK was loaded by the server process.
- **Every path is checked against `--root` before it is opened**, and
  symbolic links are resolved *before* the check, so a link out of the root
  is refused rather than followed. `--root` defaults to the working directory
  the server was started in, because a server with no root is a file-reading
  primitive for whatever is driving the assistant.
- **A refusal is an answer.** An unreadable document comes back as a refusal
  carrying exit-2 semantics, never as a report with an empty findings list —
  an empty findings list is what a *clean* document looks like. A request for
  a judgment this tool does not make is refused with the boundary stated, in
  the vocabulary [`ai/guard.py`](src/oscal_validate/ai/guard.py) already uses;
  the two lists are held together by a test rather than by an import, because
  this server may not import that package.

### `package`: a directory validated as one deliverable

A FedRAMP or agency package is a directory: an SSP, the profile it imports, the
catalog behind that. Validating each file alone, with a hand-assembled
`--resolve` list, answers a smaller question than the one a reviewer is asking.
`oscal-validate package` takes the directory as the thing being validated.

```sh
oscal-validate package ./my-package
oscal-validate package ./my-package --format json
oscal-validate package --report-schema     # the shape that report conforms to
```

It is **exactly N runs of the command line, plus what only a set can show**.
Every member is validated with every other member as its resolve set — the
documents `oscal-validate <member> --resolve <directory>` would read — through
the same composition function the command line reaches, so each member's report
is byte for byte that command's, and the suite compares them in both formats.
What it adds is what no single run can see:

| section | lists |
|---|---|
| `import_graph` | every import each member writes itself, what it matched, and how |
| `imports_not_in_package` | imports naming a file the directory does not contain |
| `unreferenced` | members nothing else imports — the package's roots, not its defects |
| `uuid_collisions` | UUIDs declared in more than one member, with every place each is declared |

Five things about it:

- **It fails closed.** A file in the directory that is not JSON, is not an
  OSCAL document, or nests too deeply to walk is named on stderr with its
  reason, and the run exits 2 and writes no report. Validating the rest would
  report imports of that file as not supplied, about a file that was, and a
  report with a short document list is exactly what a smaller clean package
  looks like. An empty directory is exit 2 as well.
- **The cross-document sections never gate.** The exit code is the command
  line's own verdict over the members: 1 if any member has an ERROR, else 0.
- **A UUID declared in two members is package policy, not a finding.** NIST's
  constraint layer, and this tool, check UUID uniqueness within one document;
  a duplicate *inside* one member is still that member's ERROR. Across members
  it is listed with every declaration, for a reviewer to decide.
- **There is no "ambiguous imports" section, though issue #60 asked for one.**
  Import matching keys on a file name, then on the file name without its
  extension, and within one directory no two `.json` files can share either —
  so no import can resolve to more than one member here, and the section could
  only ever print "none".
- **The report has its own published schema,** `package_report_schema_version`
  `1.0.0`, independent of the per-document report schema that each member's
  embedded report conforms to. Nothing about the per-document report changed.

The GitHub Action does not run this verb yet; its `path` input still validates
each document on its own. `.pre-commit-hooks.yaml` publishes a hook that does,
`oscal-validate-package`: set its `args` to the directory holding your package.

### `--suggest`: the identifier that *is* declared

A reference that resolves to nothing is a true statement and not the sentence
that fixes the file. The 2026-08-15 imports survey measured 178 real unresolved
references, and the largest class of them was one zero-pad away from resolving:
an SSP naming `ac-2_odp.01` against a baseline that declares `ac-02_odp.01`.

`--suggest` names up to three identifiers that *are* declared and are close to
the one written, and says how they differ:

```console
$ oscal-validate my-ssp.json --resolve baseline.json --suggest
ERROR        REFERENCE_UNRESOLVED  at=/system-security-plan/.../set-parameters/3/param-id
    param-id = ac-2_odp.01
    This names a parameter, and no such identifier is declared in the documents
    supplied. ...
    rule: ...
    source: ...
    also declared: ac-02_odp.01  (differs by zero-padding)
```

Four things it deliberately is not:

- **Not on by default.** Without the flag this command emits exactly the bytes
  it always did; `tests/golden/` pins them.
- **Not a claim about intent.** It says an identifier is declared and how it
  differs. It never says it is the one you meant, and the finding's code,
  severity and message do not change.
- **Not a search of the whole document.** Candidates come only from the
  identifiers that reference kind may name — a group id is never offered for a
  control reference — and only from the documents you supplied. Nothing is
  fetched and no model is called.
- **Not offered for an UNVERIFIABLE reference.** When an import was not
  supplied, the index is short by definition, and the nearest entry in a
  partial index is not evidence about anything.

Ranking is fixed and documented: identifiers differing only by a leading `#`,
case, the separator character, or zero-padding come first; then anything within
an edit distance of two, adjacent transpositions counted as one edit; ties
broken lexically.

### `--locations`: the line, beside the pointer

A finding's location is a JSON Pointer such as
`/catalog/groups/16/controls/23/id`. That is exact, it survives reformatting,
and it is not where anyone's editor is. `--locations` adds the position in the
source bytes **beside** the pointer, never in place of it:

```console
$ oscal-validate broken_catalog.json --locations
ERROR        REQUIRED_PROPERTY_MISSING  at=/catalog/metadata  broken_catalog.json:4:17
    last-modified = (absent)
    ...
```

`--format json` gains `line` and `column` on every finding. They are
three-valued and the third value is the one that matters:

| | meaning |
|---|---|
| the keys are **absent** | the flag was not given, and the run makes no claim about where anything is |
| an **integer** | the 1-based line or column where the pointed-at value begins |
| **`null`** | the flag was given and this run has no position for that pointer |

Never `0`. There is no line 0, so a consumer never has to decide whether a
zero is a position or an absence — the same rule as the `UNVERIFIABLE`
severity, one level down.

Three more things it deliberately is not:

- **Not on by default.** Without the flag the bytes are what they always were
  and no source index is built, which is why a 26 MB catalog pays nothing for
  a feature it was not asked for. `tests/golden/` pins those bytes.
- **Not a new key.** The pointer stays the identity of a finding.
  `--baseline`, `diff` and the goldens all key on it, and reformatting a file
  moves every line without moving a single pointer —
  `tests/test_locations.py` reformats a fixture at indent 4 and asserts
  exactly that.
- **Not in `sarif` or `html` yet.** Those two formats are unchanged by the
  flag today, and a test asserts they are byte-identical with it and without
  it, so the flag cannot look like it reached a format it did not.

The [GitHub Action](#github-action) asks for positions on every run, so a
finding annotates the line rather than the file. If you run the CLI yourself
in a workflow, `.github/problem-matcher.json` turns the text report into
annotations the same way:

```yaml
- run: echo "::add-matcher::${{ github.workspace }}/.github/problem-matcher.json"
- run: oscal-validate my-ssp.json --locations
```

Its three matchers map this tool's four severities onto GitHub's three
annotation levels using the same table `tools/action_runner.py` uses, and a
test holds the two together. What the suite proves about the matcher is that
its patterns match the bytes this tool prints and capture the right file, line
and column; it does not prove GitHub applies them, because nothing in a
workflow can read its own annotations back.

### `--baseline`: acknowledge a finding without hiding it

NIST's own SP 800-53 rev 5 catalog carries an ERROR this tool reports: a link
naming a statement that does not exist. A team that imports that catalog cannot
edit NIST's file, so today their options are to ignore the exit code or to turn
the gate off, and both end with nobody reading the report.

```sh
oscal-validate my-ssp.json --write-baseline > oscal-baseline.json
# write a reason and a date into every entry, then commit the file
oscal-validate my-ssp.json --baseline oscal-baseline.json
```

**An acknowledged finding is still a finding.** It is printed, it keeps its
severity, it is counted in the summary, and it appears in the JSON report, the
SARIF log and the HTML page exactly as it did — with the reason and the date
beside it. The single thing an acknowledgment changes is whether the finding
gates the exit code. A report that showed three ERRORs and a summary saying
zero would be the failure this tool exists to prevent.

Four refusals keep it from becoming a suppression list:

- **A written reason is required.** An entry with none, or with an empty one,
  is refused at exit 2 — not defaulted and not warned about. This is also why
  `--write-baseline` writes to stdout with every `reason` empty: the generated
  file is refused as generated, so it cannot be committed and pointed at
  without someone saying why each entry is there.
- **A baseline cannot outlive the defect it excused.** An entry that matches
  nothing in this run is reported as `BASELINE_STALE` at WARNING, quoting the
  reason that was written for it, and `--fail-on-stale` makes that gate. It is
  a WARNING by default because the document may simply have been fixed.
- **UNVERIFIABLE cannot be baselined.** There is nothing to acknowledge in an
  answer the tool did not reach; acknowledging one would turn "this could not
  be settled" into "this was decided to be acceptable". Refused at exit 2, and
  `--write-baseline` never offers one.
- **Nothing reads the clock.** `acknowledged_on` is checked for being a real
  `YYYY-MM-DD` calendar date and is never compared against today. A gate that
  goes red on a calendar rather than on a commit stops every unrelated change
  in a repository.

An entry names a finding by code, JSON Pointer, property and value — never by
its message, which is prose this project edits and whose rewording would
silently restore a gate. One consequence is worth knowing before you write a
reason: **a key does not always name exactly one finding.** The same duplicated
identifier can be reported under two different NIST constraints, differing only
in message and rule, and one entry acknowledges both. `--write-baseline` writes
one entry per key for that reason.

## What it checks

| # | Check | Codes | Rule source |
|---|---|---|---|
| 0 | Which imports were supplied, which were not, and which were supplied more than once | `IMPORT_RESOLVED`, `IMPORT_NOT_SUPPLIED`, `IMPORT_AMBIGUOUS` | The report's own audit trail for every severity below |
| 1 | Document shape against the published JSON Schema: required properties, properties the schema forbids, JSON type, arrays shorter than `minItems`, and objects no declared alternative accepts | `REQUIRED_PROPERTY_MISSING`, `PROPERTY_UNDECLARED`, `TYPE_MISMATCH`, `ARRAY_TOO_SHORT`, `NO_SCHEMA_ALTERNATIVE`, `SUBTREE_NOT_READ` | [`oscal_complete_schema.json`](https://github.com/usnistgov/OSCAL/releases/tag/v1.2.3) |
| 2 | Scalar values against the datatype the schema declares at that position: UUID form (v4 or v5), timestamps with a required timezone, URIs, non-empty markup lines, and the lower bounds on OSCAL's two integer datatypes | `DATATYPE_MISMATCH`, `DATATYPE_BELOW_MINIMUM`, `PATTERN_NOT_CHECKED` | the same schema's own datatype patterns and bounds |
| 3 | NIST's constraint layer: `is-unique`, `index` uniqueness, `index-has-key` cross-references, `has-cardinality`, and the `matches` constraints whose regex or datatype the vendored files carry | `CONSTRAINT_NOT_UNIQUE`, `CONSTRAINT_CARDINALITY`, `CONSTRAINT_VALUE_MISMATCH`, `REFERENCE_UNRESOLVED`, `REFERENCE_UNVERIFIABLE`, `CONSTRAINT_NOT_EVALUATED` | the vendored `*_metaschema_RESOLVED.xml` modules, at NIST's declared severity |
| 4 | One UUID, one object, across the whole document | `UUID_NOT_UNIQUE` | [Identifier Use and UUIDs](https://pages.nist.gov/OSCAL/learn/concepts/identifier-use/) |
| 5 | Identifier references the constraint layer does not cover: `control-id`, `with-id`, `param-id`, `statement-id`, and bare `#` fragments | `REFERENCE_UNRESOLVED`, `REFERENCE_UNVERIFIABLE` | [URI Usage](https://pages.nist.gov/OSCAL/learn/concepts/uri-use/) |
| 6 | Which OSCAL release the document was authored against, versus the one it was judged by, and which of this report's ERRORs were therefore not checked against the release the document names | `OSCAL_VERSION_DIFFERS`, `VERSION_SKEW_SUSPECTED` | the schema's own `oscal-version` description; [ADR-0008](docs/adr/0008-version-skew-is-flagged-not-resolved.md) |

Every finding carries its rule citation, source URL, and retrieval date in the
output itself, in both text and JSON.

## The effective data model

This is the idea the whole tool turns on, and it is NIST's, not mine:

> "The effective data model of a document includes all objects identified with
> the document and any directly or transitively imported documents."
> — NIST, [URI Usage](https://pages.nist.gov/OSCAL/learn/concepts/uri-use/)

A catalog imports nothing, so its effective data model is itself: a reference in
it that resolves nowhere is **wrong**, and the tool says ERROR. A profile
imports a catalog, so its effective data model is invisible unless the catalog
is handed over: the same unresolved reference is **unknown**, and the tool says
UNVERIFIABLE and names the file it was missing.

```console
$ oscal-validate baseline_profile.json
UNVERIFIABLE REFERENCE_UNVERIFIABLE  at=/profile/imports/0/include-controls/0/with-ids/3
    with-ids = ac-2
    ... 1 imported document(s) named by this document were not supplied
    (NIST_SP-800-53_rev5_catalog.xml), so the effective data model is
    incomplete. Supply the imported document with --resolve to settle this.

$ oscal-validate baseline_profile.json --resolve NIST_SP-800-53_rev5_catalog.json
0 ERROR
```

Imports are matched to supplied files by file name, and by file name without
its extension when that fails, because a JSON profile in the wild routinely
imports the XML serialization of its catalog. Which file each import matched is
reported, so an UNVERIFIABLE finding always comes with the reason it could not
be settled.

`--resolve` is repeatable and takes directories, so the same file reaches it
twice for reasons that are not mistakes: a directory named twice, a directory
and a file inside it, a path with and without a trailing slash. One file
reached twice is one file, identified by its resolved path, and every one of
those spellings produces the same report as passing the file once.

What is left after that is real: **two different files answering to one name**,
which happens as soon as two publishers' `catalog.json` are handed over
together. That is `IMPORT_AMBIGUOUS`, and it is deliberately not
`IMPORT_NOT_SUPPLIED`. The documents *were* supplied; what cannot be determined
is which one the import means, so neither is admitted to the effective data
model and references into it stay UNVERIFIABLE. The finding names every
candidate path, and the fix is the opposite of the fix for a missing document —
narrow `--resolve` rather than widen it — so the report says that instead.

## GitHub Action

If the documents you deliver live in a repository, [`action.yml`](action.yml)
validates them on every pull request and annotates each finding on the file it
came from:

```yaml
name: Validate OSCAL
on: [pull_request]

permissions:
  contents: read

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      # Prefer a commit SHA. A tag is shown for readability; v0.2.0 is the
      # first release that carries the action.
      - uses: ChelseaKR/oscal-validate@v0.2.0
        with:
          path: oscal/
```

`path` takes one document, a directory (searched recursively for `*.json`), or
a glob such as `oscal/**/*.json`. Five further inputs, all optional:

- `resolve`: space-separated documents or directories to resolve imports and
  references against, passed through as repeated `--resolve`. Same rules as
  the CLI, including the file-name matching described above, and nothing is
  fetched.
- `fail-on`: `error` (default), `warning`, or `info`. The CLI itself gates on
  ERROR and only ERROR; a lower threshold is applied by the action, from the
  counts in the CLI's own `--format json` summary. UNVERIFIABLE is gated at no
  setting, because it marks what the supplied documents cannot settle and is
  never a pass.
- `sarif-file`: a path to write SARIF 2.1.0 to, for
  `github/codeql-action/upload-sarif`. Empty by default, which writes nothing.
  See [below](#code-scanning).
- `baseline`: a committed baseline file, as described
  [above](#--baseline-acknowledge-a-finding-without-hiding-it). Findings it
  names are still annotated at their own severity and still counted in
  `error-count`; they no longer count towards `fail-on`. An entry with no
  written reason fails the job rather than being skipped.
- `fail-on-stale`: `"true"` to fail the job when a baseline entry matched
  nothing in this run. Setting it without a `baseline` fails the job rather
  than passing silently.

```yaml
      - uses: ChelseaKR/oscal-validate@v0.2.0
        id: oscal
        with:
          path: oscal/**/*.json
          resolve: baselines/ NIST_SP-800-53_rev5_catalog.json
          fail-on: warning
      - run: echo "${{ steps.oscal.outputs.unverifiable-count }} finding(s) unsettled"
```

The counts are published as outputs: `error-count`, `warning-count`,
`info-count`, `unverifiable-count`, `acknowledged-count`,
`stale-baseline-count`, and `files-validated`. Watch the unverifiable count: a
run of a large package with imports withheld can be green and still have
settled very little, and the number is how you see that. Watch
`acknowledged-count` for the same reason in the other direction: it is how many
of those findings a baseline is holding open, and it is deliberately *not*
subtracted from `error-count`.

### Code scanning

Annotations vanish with the workflow run. To keep the findings, set
`sarif-file` and hand the file to `github/codeql-action/upload-sarif`; the
findings then appear in the Security tab and inline on the pull request. Every
UNVERIFIABLE finding arrives as an alert at level `note`, on purpose; filter by
rule, never by hiding them.

```yaml
permissions:
  contents: read
  security-events: write

jobs:
  code-scanning:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: ChelseaKR/oscal-validate@v0.2.0 # `sarif-file` is newer than v0.2.0; pin a commit that carries it
        id: oscal
        with:
          path: oscal/
          resolve: baselines/
          sarif-file: oscal-validate.sarif
      - if: ${{ !canceled() && steps.oscal.outcome != 'skipped' }}
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: oscal-validate.sarif
          category: oscal-validate
```

The `if:` is what carries a *failing* run's findings to the Security tab:
without it the upload is skipped exactly when there is something to upload.

Every document goes into one SARIF run, not one run each, because GitHub
accepts at most twenty runs per file and a delivery is routinely more than
twenty documents. The file is written only when every document produced a
complete SARIF run, and the run fails without writing it otherwise. That is
deliberate and worth knowing: `upload-sarif` treats an upload as the complete
picture and resolves any alert it does not contain, so a file that had quietly
lost one document's findings would close those alerts as fixed. A missing file
fails the upload step loudly instead.

`tool.driver.properties.vendoredSnapshot` in the log records the OSCAL release
and the SHA-256 of every vendored file the run read, computed from the files
themselves. An alert outlives the checkout that produced it, and the version
of the tool alone does not say which bytes decided the verdict.

The CLI can still be run directly for the same output on one document —
`oscal-validate <file> --format sarif` — which is what to do outside Actions.

The exit codes are the CLI's, unchanged: 0 when nothing meets the threshold, 1
when something does, 2 when a document could not be read. A `path` that
matches no file at all is also exit 2, because a run that validated nothing is
not a run that passed.

There is no install step and no lock file to hash-pin, because there is
nothing to install: `oscal-validate` has zero runtime dependencies and ships
`python -m oscal_validate`, so the action runs the checked-out source directly,
vendored NIST schema and metaschema included, and resolves nothing from PyPI
while it runs. `actions/setup-python` is pinned to a commit SHA and to the same
Python 3.12 the rest of this repository uses.

The gate is tested in both directions, because a gate that cannot fail is
worse than no gate: `tests/test_action_runner.py` asserts the exit code for
clean documents, gated findings, unreadable input, and a `path` that matches
nothing, and CI runs the composite action itself over the clean catalog and
over a copy with one required property removed, failing the build if the
broken copy passes.

## AI-assisted explanation and repair (opt-in)

A finding such as `CONSTRAINT_NOT_UNIQUE` at
`/catalog/groups/16/controls/23/id`, cited to a Metaschema constraint by
identifier, is correct and is not easy to act on. Four subcommands put a
language model beside the validator for the people who author these
documents. They are the direction change recorded in
[ADR-0005](docs/adr/0005-ai-at-the-edges.md), and they keep three things
fixed: the validator is the only thing that produces a finding, NIST's
published text is the only evidence, and a verifier that is not a model sits
between every reply and the screen.

```sh
pip install '.[ai]'                       # the public anthropic SDK; nothing else changes
                                          # (from a checkout: nothing is on PyPI yet)
export ANTHROPIC_API_KEY=...              # from the environment only; never written to a file

oscal-validate explain my-ssp.json --severity ERROR
oscal-validate repair --draft my-ssp.json --label F6 --out my-ssp.patched.json
oscal-validate walkthrough my-ssp.json
oscal-validate ask "What does the is-unique constraint oscal-catalog-controls require?"
```

- **`explain`** runs the validator, labels its findings F1..Fn in report
  order, and for each selected finding gathers the NIST text that bears on
  it: the JSON reference entry at that pointer, the constraint's declaring
  element verbatim from the vendored metaschema, the Metaschema
  specification's section on that constraint kind, the concept page the rule
  cites. The model explains the finding by quoting that text. Every quote is
  looked up verbatim in the source it names, and every quotation written
  inline in the prose is looked up across all sources; one that is not there
  is withheld, its marker struck, and counted. The output says how many
  quotes verified and how many were withheld.
- **`repair --draft`** asks for an RFC 6902 patch (`add`, `remove`,
  `replace` only), applies it to a copy, and runs the deterministic validator
  on the copy with the same `--resolve` documents. What that run found is the
  report: `resolves F6; 6 finding(s) untouched; 1 other(s) also resolved; 0
  changed; 0 introduced`, with the counts before and after and a diff. The
  model claims nothing about the effect; the validator states it. Values
  only the author knows are placeholders and are listed as such. The
  original file is never written; `--out` writes the patched copy elsewhere
  and refuses the original or any `--resolve` path. A patch whose values
  carry an implementation narrative is refused whole.
- **`walkthrough`** groups the findings in the tool's fix order — supply what
  the document imports before the references it leaves unsettled, shape the
  validator could not read before what is beneath it, identifiers before
  references — labels the groups G1..Gn, and has the model narrate over the
  labels. A label the validator never produced is struck and counted; a
  group the narrative never mentions is appended with every finding in it;
  the full index is printed last. Nothing is invented and nothing is
  suppressed, and the header says how many groups the narrative covered.
- **`ask`** answers a question about OSCAL's published rules from the
  corpus, through the same verifier; with `--document`, the validator runs
  first and its findings are given to the model as labels.

**The boundary, enforced twice.** The model is instructed to refuse every
form of "is this control implemented", "is this system secure", "would this
get an ATO", "is this FedRAMP-ready", "does this satisfy AC-2", and to
redirect to structural conformance and qualified assessment. Then a
deterministic guard screens every sentence that would be shown and withholds
any that carries such a judgment, and says how many it withheld. The guard
is lexical on purpose: a guard that asked a model whether the model had
overstepped would be no guard at all. It errs toward withholding, and
sometimes withholds a sentence that was not a judgment.

**What leaves the machine.** A model call carries the findings, the corpus
passages, and excerpts of the document around each finding's location. A
system security plan can contain sensitive detail about a real system. The
default command sends nothing anywhere; the GitHub Action never runs these
commands; and a document sent through them is subject to the model
provider's handling of it.

**Honest refusals.** A document the validator cannot parse is refused
before any model call. A finding whose rule is this tool's own policy is
explained as such and never attributed to NIST. A document declaring an
OSCAL version other than 1.2.3 carries a note that every finding was judged
against 1.2.3
([ADR-0008](docs/adr/0008-version-skew-is-flagged-not-resolved.md)).
A reply that cannot be parsed shows nothing.

**Model and provider.** The public `anthropic` SDK, default
`claude-sonnet-5`, configurable with `OSCAL_VALIDATE_AI_MODEL`;
`OSCAL_VALIDATE_AI_PROVIDER=bedrock` with `AWS_REGION` uses Amazon Bedrock
(`pip install '.[bedrock]'` from a checkout), where the default is a different
model, `global.anthropic.claude-sonnet-4-6`. Bedrock grants model access per
account rather than per SDK, so the two defaults answer different questions:
the Bedrock one is the model every recorded eval and cassette here was actually
produced with, and Sonnet 5 returns `AccessDeniedException` on that account
whatever its entitlement API says. Set `OSCAL_VALIDATE_AI_MODEL` if your own
account is entitled to something newer. Credentials come from the
environment and are never written anywhere. `OSCAL_VALIDATE_AI_CASSETTE`
names a file of recorded replies that the commands replay without a
network, which is how the tests run and how an evaluation can be re-scored.

**Measured, with provenance.** [`docs/evals/README.md`](docs/evals/README.md)
is the write-up of the committed harness in [`evals/`](evals/). Run on
Amazon Bedrock `claude-sonnet-4-6` on 2026-08-21 (Sonnet 5 was not
available to the account that ran it): on the boundary suite of 100
adversarial phrasings, the shown text carried no judgment in 80 of 80
refuse-cases by both the guard and an independent judge call, all 80 were
explicit refusals, and all 20 structural control questions were answered;
on twelve published NIST documents with injected defects, 62 of 62 repair
drafts resolved their finding: 61 with nothing introduced anywhere, and 1
where re-validation caught and reported a finding the patch itself
introduced (never applied, never shown as clean); across 48 explanations, 61
quotes verified and 20 were withheld, every withheld one naming a source
outside the corpus; and 12 of 12 walkthroughs covered every group with no
label invented. Every results file names the provider, model, prompt
version, commit, and date, and a test rejects one that does not.

## Methodology

### Severities, honestly defined

- **ERROR**: the document violates a cited structural rule. Gates the exit code.
- **WARNING**: a cited signal that something is very likely wrong, or a NIST
  constraint declared at `level="WARNING"`. Severity for constraint findings is
  NIST's, not this tool's.
- **INFO**: worth a human look, not a defect.
- **UNVERIFIABLE**: the answer cannot be determined from the documents supplied,
  and the tool refuses to guess. Never rendered as a pass or a fail, never
  gates the exit code.

The rule behind UNVERIFIABLE is the rule behind the whole tool: never punish
what you cannot see, and never bless it either. Four things always report as
UNVERIFIABLE rather than passing quietly:

- references into a document that was not supplied,
- the 227 published constraints this tool does not evaluate,
- references checked against an index that no evaluated constraint builds,
- values governed by a pattern this tool cannot compile.

The third of those is the mirror of the rule and was added after it bit. One of
the 113 constraints this tool evaluates is an `index-has-key` constraint whose
index is populated by an `index` constraint it skips, so the lookup misses every
key no matter what the document says. Reporting that as a failure would render a
rule that was never evaluated as somebody else's defect, which is the same
mistake in the other direction. That one is listed in
[`docs/CONSTRAINT-COVERAGE.md`](docs/CONSTRAINT-COVERAGE.md) under a generated
heading so the count cannot drift, and where the prose rule in check 4 reaches
the same reference and would settle it, the unsettled report is the one
published ([ADR-0006](docs/adr/0006-one-reference-one-report.md)).

### Break the gate before trusting it

`tests/test_break_the_gate.py` starts from a document proven clean, corrupts
one thing at a time (removes a required property, adds one the schema forbids,
breaks a UUID's version nibble, drops a timezone, duplicates a control id,
dangles a back-matter fragment, points a profile at a control that does not
exist, replaces a whole assembly with a scalar, empties an array the schema
requires items in, puts a port number below the schema's minimum), and asserts
each corruption is caught. A gate that has not been deliberately broken is a
gate you are trusting on faith.

Two of those tests assert the opposite direction too: a profile reference that
misses is an ERROR *only* when the catalog was supplied, and UNVERIFIABLE
otherwise; and a port range inside its bounds must stay clean, so the bound
check cannot pass by reporting everything.

The last three corruptions were added because they were not caught. A catalog
whose `metadata` was the JSON value `null` exited 0 with no ERROR finding: the
walk filed the scalar where an assembly belonged, everything below it became
unreachable, and the report was the verdict a valid document gets.
`{"catalog": null}` did the same for a document with no body at all. Separately,
a port range of `start: -1, end: 99.5` produced output byte for byte identical
to `start: 443, end: 443`, because the two facets `NonNegativeIntegerDatatype`
states, an `integer` base type and `"minimum": 0`, were both dropped on the way
out of the schema.

### Determinism

`tests/test_determinism.py` asserts byte-identical output for repeated runs,
including across separate interpreter processes, and that no timestamp or
duration appears anywhere in the report. There is nothing to seed: no sampling,
no clock, no network.

### No network in the validator, proved rather than promised

`tests/test_offline_guarantee.py` removes `socket` and runs the validator
anyway — and, since the MCP server landed, every one of that server's four
tools with it, driven through the real request loop. A separate test asserts
that no module inside the installed package
outside `oscal_validate/ai/` imports `urllib.request`, `http.client`,
`socket`, `requests`, `httpx`, or `anthropic`; that nothing outside `ai/`
imports `ai/`; and that `ai/` names the SDK only inside a function, so
importing it costs nothing. That scan is a file set, and a file set is a
claim about what it contains, so one more test asserts the server's own
module is inside it. `tests/test_default_path_byte_identity.py` runs
the default command in a fresh process and asserts it loaded neither the
package nor the SDK, and compares its exact bytes over the fixtures and
nine published documents against [`tests/golden/`](tests/golden/),
first captured from commit `6978895`, the last commit before any model-backed
command existed.

**The model-backed layer has never moved those bytes, and that is what this
gate is for.** They have moved nine times, every one for an unrelated reason.
On 2026-08-29 (#35): the `CONSTRAINT_NOT_EVALUATED` finding for
`allowed-values` carried a sentence that said something false about NIST's
`allow-other` semantics, and correcting a sentence the report prints is a
change to the report. On 2026-09-01 (#30): eleven `matches` constraints became
evaluated, so the line counting the unevaluated ones went from 25 to 14. On
2026-09-02 (#38), cutting 0.3.0: the JSON report stamps the tool's own
version, so twelve lines moved, one per JSON golden, and the twelve text
goldens did not move at all. On 2026-09-06 (#81): every JSON report gained
`report_schema_version`, again twelve lines and no text golden. On 2026-09-07
(#88), cutting 0.4.0: the version stamp again, twelve lines and no text
golden. On 2026-09-07 (#92): the report schema went to `1.1.0` for the two keys
`--baseline` adds, so every JSON report's `report_schema_version` line moved —
twelve lines, one per JSON golden, and no text golden, because the text format
does not print it. On 2026-09-09 (#95): the report schema went to `1.2.0` for
the two keys `--locations` adds, `line` and `column`, neither of which any
golden run produces, because no golden run passes the flag — so again the whole
diff is the `report_schema_version` line, twelve lines and no text golden. On 2026-09-12 (#77): a document that declares a non-vendored
release *and* carries an ERROR gains one `VERSION_SKEW_SUSPECTED` INFO
finding, which two of the twelve cases do; the other ten are untouched and no
ERROR count or exit code moves. On 2026-10-19 (#128), cutting 0.5.0: the
version stamp again, twelve lines and no text golden. Each time the
goldens were recaptured from the same documents, each verified by SHA-256
against the manifest that recorded them, and every other byte of the output is
unchanged. Those nine are the only recaptures since `6978895`;
[CHANGELOG.md](CHANGELOG.md) and
`tests/test_default_path_byte_identity.py` record all nine, and
`tests/golden/capture.py` now refuses to write a manifest smaller than the
committed one, so a recapture on a machine without the cached documents cannot
quietly shrink what this compares.

The one thing in this repository that opens a socket is
[`tools/fetch.py`](tools/fetch.py), a development harness that is not installed
and is not reachable from the CLI. It exists to collect the documents behind
[`docs/findings/`](docs/findings/). Its posture — robots.txt fetched first and
obeyed with no override flag, an unreachable robots.txt treated as a complete
disallow per RFC 9309 2.3.1.4, an identifying User-Agent, five redirects with
robots re-checked at each hop, a byte cap, a timeout, and a per-host rate limit
a site's `Crawl-delay` can lengthen — is tested against a server on localhost.

## Pointed at reality

[`docs/findings/2026-08-14-published-oscal-survey.md`](docs/findings/2026-08-14-published-oscal-survey.md)
is what came back from running this against 52 published OSCAL documents,
26.1 MB across seven of the eight models, from NIST, FedRAMP, and four
third-party publishers. Thirteen carried at least one ERROR. Among them: a link
in NIST SP 800-53 rev 5 naming a control statement that does not exist, 64
dangling fragments of one shape in SP 800-171 rev 3, 501 in a resolved LOW
baseline, and a catalog whose own UUID is also one of its back-matter
resources'. Every one was
verified by hand against the document before publication.

That run's own stated limit was that most of what it saw it could not settle:
5,501 references reported UNVERIFIABLE against 568 reported wrong.
[`docs/findings/2026-08-15-imports-supplied-survey.md`](docs/findings/2026-08-15-imports-supplied-survey.md)
is the same 52 documents with their imports located and handed over. **5,216 of
those 5,501 references resolved to something that exists, 178 resolved to
nothing, and 107 still cannot be settled.** All four FedRAMP rev 5 baselines
went from a combined 2,787 unanswerable control references to zero errors. The
178 are four shapes, and the largest is a parameter identifier that is one
zero-pad away from resolving: an SSP setting `ac-2_odp.01` against a baseline
that defines `ac-02_odp.01`.

The 107 that stayed unknown are a finding in their own right. Nine documents
import a profile whose only `rlink` is a GitHub release ZIP; five import a
relative path into a directory the publishing repository does not have; one
imports a back-matter UUID it never declares. And 29 were this tool's fault, a
defect the run exposed and the write-up describes.

Those two runs shared one sample, and it had two limits they both stated: seven
of the eight models, and 30 of 52 documents from NIST.
[`docs/findings/2026-08-19-widening-the-corpus-survey.md`](docs/findings/2026-08-19-widening-the-corpus-survey.md)
is 43 more documents from twenty-one publishers none of the first two reached —
the German BSI, the Australian Cyber Security Centre, NIST's BLOSSOM program,
GSA, the OSCAL Plugfest, the Linux Foundation's OSCAL Compass, Red Hat, MITRE and
others — taking the corpus to **95 documents and all eight models**. Nine carried
at least one ERROR, each verified by hand: four UUIDs used twice in one SSP, 35
implemented requirements with no `description`, 13 control titles with newlines
in a field declared as one line, three catalog groups that hold both subgroups
and controls where NIST's schema permits either but not both, and Go's zero-value
timestamp published as a date.

The eighth model arrived with a result attached. All seven mapping collections
reported `SUBTREE_NOT_READ` at `/mapping-collection/mappings` — the whole
substance of the model is outside what this tool resolves, and it says so on
every document rather than reporting a clean run. Three of the seven also declare
an `oscal-version` from before OSCAL 1.2, a release whose schema has no mapping
model at all; that was measured against NIST's own published v1.1.2 schema, not
assumed.

That limit is gone. `mappings` is the one place in the vendored schema NIST
writes a repeatable assembly as "one, or an array of one or more" rather than
as an array, and the walk now reads both spellings
([ADR-0007](docs/adr/0007-read-the-eighth-model.md)). The seven documents above
report no `SUBTREE_NOT_READ` today and carry 31 ERROR findings between them
where the run recorded none: an `id_ref` written where the schema declares
`id-ref` and forbids anything else, a percentage written as a string where the
schema declares a decimal, an array present and empty where the schema requires
an item, and resource fragments naming a resource's `props/id` rather than its
`uuid`. The survey above is what that run found and is left as the record of
it.

The survey harness and its target lists are in [`tools/`](tools/), and all three
runs are reproducible: the findings a run records are now independent of where
its cache lives, which they were not before, and `--provenance` carries a
document's retrieval record forward into any later run that reads it from cache.

## Limits

Everything below is a thing this tool does **not** do. If a check could not be
implemented correctly it was left out and named here rather than shipped
approximate.

**It does not judge implementation.** Whether a control is implemented, whether
an assessment was performed, whether evidence supports a claim: none of that is
visible in a document's structure, and nothing in this tool looks at it. The
model-backed commands refuse the question and a guard withholds any answer
that slips through; the boundary suite in [`evals/`](evals/) measures both.

**It evaluates 113 of NIST's 340 published constraints.** Every one of the
other 227 is listed with its reason in
[`docs/CONSTRAINT-COVERAGE.md`](docs/CONSTRAINT-COVERAGE.md), which is generated
from the vendored files and checked by a test so it cannot drift, one reason
per constraint rather than one per kind. In summary: of the 200
`allowed-values` sets, 60 declare `allow-other` and 140 do not, and the
Metaschema specification makes the absent attribute mean the set is closed
("no: (default) Identifies the expected value set as closed"), so what is
missing is not a judgment about openness but the applicable-set resolution that
decides which values a value node permits; 11 of the 25 `matches` constraints
are evaluated and the other 14 name a target this tool cannot read, a datatype
the vendored schema does not define or defines without a pattern, or a regex
whose meaning depends on which of two published dialects applies; 12 `expect`
constraints each carry a Metapath `test` this tool does not implement, and the
coverage document prints each one; and exactly one target remains
outside the parsed subset — `oscal-ssp-by-component-uuid-index`, which
dereferences a second document through `doc()`. The predicate and
interior-descendant targets that used to block 25 constraints are parsed under
the bounded grammar of
[ADR-0004](docs/adr/0004-bounded-predicate-grammar.md), enumerated from the
vendored files rather than the Metapath specification. The order in which the
rest of that gap is intended to close, and what each step is waiting on, is
[`docs/EXPANSION-PLAN.md`](docs/EXPANSION-PLAN.md). A constraint is also
only applied to documents of the models its module governs, since assembly
names repeat across models and a catalog's `part` is not assessment-common's.

**It vendors one OSCAL schema and cannot tell a defect from a release difference.**
Everything is checked against the vendored 1.2.3 release,
whatever release a document declares. Where a document declares a different one
*and* carries an ERROR, the report says so in its own terms — a
`VERSION_SKEW_SUSPECTED` INFO finding naming how many ERRORs there are, under
which codes, and that whether each is also an error under the declared release
was not determined here, because no schema for that release is vendored. It
never says the ERROR is wrong: the ERROR is true of the release it cites.
Settling the second question needs NIST's schema for that release, which is
[ADR-0008](docs/adr/0008-version-skew-is-flagged-not-resolved.md)'s open
question rather than its answer.

**It does not check the `TokenDatatype` pattern.** OSCAL's token pattern uses
the ECMA-262 Unicode property escapes `\p{L}` and `\p{N}`, which Python's `re`
module does not implement. Substituting a hand-written approximation would be a
rule encoded from memory, which is the one thing this tool refuses to do, so
those values are reported unchecked with a count.

**It parses XML with the standard library, on purpose.** The metaschema modules
are vendored, hash-pinned package data that no user supplies. Both attacks the
standard library's parser is criticized for need a DTD, so any vendored file
carrying `<!DOCTYPE` or `<!ENTITY` is refused before it reaches the parser, and
a test asserts none do. Taking a hardened third-party parser as a runtime
dependency would cost the zero-dependency property that makes the no-network
and no-model claims mechanically checkable.

**It is not a JSON Schema implementation.** The walk reads `$ref`,
`properties`, `required`, `additionalProperties`, `items`, `type`, `pattern`,
`minItems`, and `minimum`. It does not evaluate `enum` (except to suppress a
pattern finding where the schema offers a literal as an alternative), `format`,
or `contentEncoding`. `maximum`, `maxItems`, `minLength`, `maxLength`,
`uniqueItems`, `const`, `if`, and `not` are absent from the vendored schema, so
there is nothing there to evaluate. Where the schema combines alternatives in a
form the walk does not resolve, the subtree is reported `SUBTREE_NOT_READ`
rather than passed over.

**It does not resolve profiles.** A profile's `modify`, `merge`, and
`insert-controls` are not applied, so it cannot check anything that is only
true of the resolved catalog. Profile `by-id` and `objective-id` references,
which are only resolvable after resolution, are not checked.

**It checks no referential integrity in the assessment models.** The
`oscal_assessment-common`, `oscal_assessment-results`, and `oscal_poam` modules
declare zero `index` and zero `index-has-key` constraints, so NIST publishes no
rule tying a finding's `observation-uuid` to an observation that exists.
Implementing that check here would mean inventing a rule and citing myself for
it.

**It cannot tell you a resolved reference is the wrong one.** If a link points
at a real identifier belonging to the wrong control, it resolves, and this tool
passes it in silence. See Finding 1 in the survey for a live example.

**One stated interpretation.** The Metaschema specification says a key-field
that selects nothing contributes a null to the key. Read literally, every
object missing an entire key would collide with every other; NIST's own
catalogs contain many properties with no `uuid` and are not treated as invalid.
So an entry whose composite key is null in *every* part is left out of the
index and out of the uniqueness comparison. Entries with some parts present are
compared with the absent parts as nulls. This is the one place the tool chooses
a reading, and it says so in the code and here.

## Where the rules come from

The schema and the thirteen metaschema modules are vendored unmodified in
[`src/oscal_validate/vendor/oscal/`](src/oscal_validate/vendor/oscal/), with
source URLs, retrieval dates, and SHA-256 hashes recorded in
[SOURCES.md](src/oscal_validate/vendor/SOURCES.md) and enforced by
`tests/test_vendor_integrity.py`, which also fails if a file arrives in
`vendor/` without a hash row. All fourteen were retrieved 2026-08-14 from the
OSCAL v1.2.3 release. Prose rules are quoted verbatim in
`src/oscal_validate/rules.py` with their page URLs and the date each page said
it was last updated. No rule is encoded from memory.

A new OSCAL release is not a version bump for this tool; it is a change to what
it can say, so the cost is made visible before it is paid.
[`tools/revendor.py <version>`](tools/revendor.py) fetches that release through
the same robots-first fetcher the surveys use -- or reads one already on disk
with `--from-dir` -- refuses any file carrying `<!DOCTYPE` or `<!ENTITY`, and
prints the difference against the vendored snapshot: constraints added,
removed, re-leveled or re-targeted, targets outside the parsed grammar, schema
definitions that changed, the vendored files whose bytes differ, and every
golden that would move, with the finding codes that moved in it. The bytes
decide whether anything changed and the inventory explains what, because the
inventory does not record an `allowed-values` set and a release that changed
only one would otherwise read as unchanged. It writes only with `--write`,
after the diff, and lists what is still a person's to do. It is a development
harness, not part of the package, and it never runs in CI.

## Development

Uses [`uv`](https://docs.astral.sh/uv/) with a locked toolchain
(Python 3.12, see `.python-version`):

```sh
uv sync --locked
make verify   # lockfile check + lint + format + strict types + coverage-gated tests + pip-audit
```

`make verify` is the exact gate CI runs; see [CONTRIBUTING.md](CONTRIBUTING.md)
for the individual targets. Its first step is `uv lock --check`, and the sync
uses `--locked` rather than `--frozen`: `--frozen` installs from `uv.lock`
without reading `pyproject.toml`, so it exits 0 on a lock that no longer
matches the manifest and cannot be a drift gate. The comment in the `Makefile`
records the measurement.

## Releasing

`.github/workflows/release.yml` is the only path that publishes anything, and
it runs only when the maintainer dispatches it:

```sh
git tag -s vX.Y.Z -m "vX.Y.Z" && git push origin vX.Y.Z
gh workflow run release.yml --ref main -f tag=vX.Y.Z
```

Dispatch rather than a tag-push trigger, because a `push: tags:` workflow runs
the definition stored *at the tagged ref* — whoever can push a tag would also
choose the release workflow. Dispatching from `main` keeps the release
authority on the reviewed branch and leaves the tag as data the workflow
checks.

What it does, in order: the shared `ChelseaKR/.github` authorization workflow
proves the tag is annotated, signed against
[`.github/allowed_signers`](.github/allowed_signers), and reachable from
`main`; `make verify` re-runs at the tagged commit, and the tag, the
`pyproject.toml` version and the CHANGELOG section must agree; the wheel and
sdist are built once, with SLSA build provenance and a CycloneDX 1.7 SBOM
attested through Sigstore; a checkout-free job publishes the GitHub release
with the CHANGELOG section as its notes; and PyPI is uploaded over OIDC
Trusted Publishing after the artifact digests are re-checked against the
attested manifest. No job rebuilds what it publishes, and no PyPI token is
stored anywhere. `tests/test_release_workflow.py` holds each of those
properties.

**PyPI is not wired up yet.** Trusted Publishing needs a one-time registration
that only the project owner can make, on pypi.org under *Your projects →
Publishing*, or *Add a pending publisher* while the project does not exist
yet: project `oscal-validate`, owner `ChelseaKR`, repository
`oscal-validate`, workflow `release.yml`, environment `pypi`. Until that
exists the `pypi-publish` job fails at the upload and nothing reaches PyPI;
every job before it — verification, build, attestation, the GitHub release —
still runs and still succeeds.

## Disclosure

This tool was built quickly with AI assistance (Claude), then reviewed and
tested by a human. Since ADR-0005 it also *contains* AI: the four opt-in
commands above call a model at run time, and say so in their first line of
output. The specification research was done first: the schema, the
constraint layer, and every prose rule were retrieved from NIST on 2026-08-14
and vendored before any check was written, and every ERROR reported in the
survey was verified by hand against the source document before publication.
Read the citations critically; if a cited source has changed since retrieval,
the vendored snapshot, not this tool's opinion, is what to update.

## Standards Conformance

This repository is part of a portfolio with shared engineering standards.
State against each, with an explicit reason wherever a standard does not apply,
and an explicit gap wherever one applies and is not yet met.

**How this scope was decided.** The portfolio keeps a machine-readable
applicability manifest, and this repository is not in it: as of 2026-08-15
`applicability.yml` on the standards repository's default branch has no
`oscal-validate` entry, which by that file's own header is a failure of the
weekly conformance run rather than a pass. The scope below was therefore
derived here, from each standard's own applicability section, for an offline
command-line tool with no hosted route, no HTML surface, and no persistent
store, whose opt-in model-backed commands are described above. It is this repository's reading, recorded so it can be
checked, and it is not a claim that any registry agrees with it yet.

| Standard | State | Evidence |
|---|---|---|
| Responsible-Tech Framework | Applies | [docs/RESPONSIBLE-TECH-AUDITS.md](docs/RESPONSIBLE-TECH-AUDITS.md): the harm surface is false assurance about a security authorization, and the controls target it directly. |
| Code Quality | Applies | Floors in `pyproject.toml`: Python >= 3.12, ruff >= 0.15, mypy >= 1.18 (strict), complexity <= 10, branch coverage >= 90%; locked with `uv.lock`; reproduced locally by `make verify`. |
| Security & Supply-Chain | Applies | [SECURITY.md](SECURITY.md); SHA-pinned Actions; Semgrep and full-history TruffleHog in CI; pip-audit in `make verify`; Dependabot; gitleaks in pre-commit. |
| CI/CD | Applies | `ci.yml` runs the same `make verify` gate as local development. |
| Observability | Applies (Tier C, library/CLI) | Declared in [docs/ROADMAP.md](docs/ROADMAP.md#observability). Tracing is out of scope because there is no network surface; the report on stdout is the entire observable surface, and its exit-code contract and JSON form are tested in `tests/test_cli.py`. Structured logging is opt-in under this tier and is not implemented; that is recorded as a gap, not as an exemption. |
| Performance | N/A (pure library/CLI with no hosted route, per PERFORMANCE-STANDARD section 0) | Recorded in [docs/ROADMAP.md](docs/ROADMAP.md). No latency-sensitive service and no frontend bundle exist to measure. `--format html` writes a file to stdout: it is not served, it loads nothing, and it carries no script, so there is no route to time and no bundle to size. |
| Accessibility | Applies (since `--format html`, the first human-facing rendered surface) | `tests/test_html_report.py` parses the rendered page with the standard library and holds it to seven structural rules: one `h1` and no skipped heading level, `lang="en"`, a skip link whose target exists, no duplicate ids, a caption and `th scope` on every table with every row exactly as wide as its header, no element that fetches anything, and every form control labeled. Each rule is seeded with the defect it exists to catch, so none of them is a check that has never gone red. Severity is carried by the word, never by color. The mechanical checks are not a full WCAG audit and the file says so: they cannot judge prose or rendered contrast. |
| Internationalization | N/A (findings and model-backed output quote English-language specification prose verbatim; see [docs/I18N.md](docs/I18N.md)) | Multilingual document *data* validates identically. |
| AI Evaluation | Applies (the four opt-in commands of ADR-0005; the validator itself has no model) | [docs/evals/README.md](docs/evals/README.md) and the committed harness in [evals/](evals/): a 100-case boundary suite scored on shown text, raw text, and explicit refusal; repair efficacy by deterministic re-validation on twelve NIST documents; citation grounding by verbatim lookup; walkthrough fidelity by label set. Results carry provider, model, prompt version, commit, and date, enforced by `tests/test_evals.py`; prompts are versioned in `oscal_validate.ai.PROMPT_VERSION`. |
| AI Development Measurement | Applies | `AI-DEV-MEASUREMENT: APPLIES` in [docs/ROADMAP.md](docs/ROADMAP.md). This repository was built with AI assistance, disclosed above, so Track A delivery and quality-debt metrics are mined portfolio-wide from git history. Track B applies to the opt-in commands and is served by the AI Evaluation row. |
| Incident Response | Applies | [docs/incidents/](docs/incidents/) holds the postmortem convention and no incident to date; [SECURITY.md](SECURITY.md) is the reporting channel. The `incident` and `sev1`-`sev4` labels exist as of 2026-09-05, each describing what that severity means for this project rather than carrying a generic word. Zero incidents so far is a count, not an exemption: the convention is exercised the first time it is needed. |
| Data Governance | Applies (L1, public non-sensitive) | Data cards in [docs/data/](docs/data/) for all three ingest sources, with hashes in [vendor/SOURCES.md](src/oscal_validate/vendor/SOURCES.md) enforced by `tests/test_vendor_integrity.py` and in [ai/corpus/MANIFEST.json](src/oscal_validate/ai/corpus/MANIFEST.json) enforced by `tests/test_ai_sources.py`. Survey records are dated per retrieval: `tools/fetch.py` stamps every fetch with `fetched_at` (UTC, RFC 3339), `tools/survey.py` carries it forward under `--provenance` instead of restamping a cached read, and `tests/test_survey_fetch.py` holds both. The five surveys committed under `docs/findings/` predate the field and are not backfilled; their lineage stays dated at file level, which [docs/data/published-oscal-corpus.md](docs/data/published-oscal-corpus.md) records. |
| Documentation | Applies | This README, [CHANGELOG.md](CHANGELOG.md), ADRs in [docs/adr/](docs/adr/), [CITATION.cff](CITATION.cff), [SECURITY.md](SECURITY.md), [CONTRIBUTING.md](CONTRIBUTING.md), [docs/CONSTRAINT-COVERAGE.md](docs/CONSTRAINT-COVERAGE.md). |
| Quality & Metrics | Applies | [docs/ROADMAP.md](docs/ROADMAP.md) names every gate as AUTO, REVIEW, or a reasoned exception. |
| Release & Versioning | Applies | SemVer; `CHANGELOG.md` kept current. `v0.1.0` and `v0.2.0` are tagged and `v0.2.0` is published as a GitHub release. `tests/test_release_metadata.py` measures every release claim against the tags in the repository rather than against another file: `CITATION.cff` must name a tagged version dated with that tag's own date, the Status paragraph above must name the declared version and every tag that exists and say when the declared version is untagged, and the declared version must have a changelog section. It refuses to read an empty tag list from a shallow checkout, and `.github/workflows/ci.yml` fetches tags so the checks run for real in CI. `pyproject.toml` declares `0.3.0` and no `v0.3.0` tag exists; that is stated rather than dated, and the citation file cites `0.2.0`. The release path is now written down and enforced: `.github/workflows/release.yml` (dispatch-on-signed-tag, SLSA provenance, CycloneDX SBOM, PyPI Trusted Publishing over OIDC with no stored token) with sixteen of its properties held by `tests/test_release_workflow.py`, and the maintainer procedure in [Releasing](#releasing). One gap remains: nothing has been published to PyPI, because Trusted Publishing needs a one-time registration only the owner can make, so the distribution `oscal-validate` still resolves to nothing on the index and installation is from source. |

## License

Apache-2.0. OSCAL is a product of the National Institute of Standards and
Technology; the `usnistgov/OSCAL` repository states that the project is in the
public domain within the United States as a work of the US government, and
additionally waives copyright worldwide under CC0 1.0. The vendored files
retain their origin in [SOURCES.md](src/oscal_validate/vendor/SOURCES.md). This
project is not affiliated with, endorsed by, or reviewed by NIST, FedRAMP, or
StateRAMP.
