Metadata-Version: 2.4
Name: skillwitness
Version: 0.1.0
Summary: Security and compliance CI gate for AI skills and agent extensions.
Project-URL: Homepage, https://github.com/1stIndeXR/skillwitness
Project-URL: Source, https://github.com/1stIndeXR/skillwitness
Project-URL: Issues, https://github.com/1stIndeXR/skillwitness/issues
Project-URL: Changelog, https://github.com/1stIndeXR/skillwitness/releases
Author: 1stIndeXR
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: agent-skills,ai-security,ci-gate,mcp,sarif,static-analysis,supply-chain
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: click<9,>=8.1
Requires-Dist: jsonschema<5,>=4.22
Requires-Dist: pathspec<1,>=0.12
Requires-Dist: pydantic<3,>=2.8
Requires-Dist: pyyaml<7,>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=8.2; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: types-jsonschema>=4.23; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

# SkillWitness CI Gate

SkillWitness is a Python 3.12 CLI for producing repeatable security and compliance reports for AI skills, hooks, helper tools, and agent extensions before they are merged into a managed skills repository. It scans local folders or git URLs, applies a policy profile, and emits human and machine-readable reports for CI review.

Copyright 2026 1stIndeXR. Licensed under [Apache-2.0](LICENSE); see [NOTICE](NOTICE).
Status: pre-1.0.

The product requirements are defined in [MD/skillwitness-ci-gate-prd.md](MD/skillwitness-ci-gate-prd.md), the implementation contract in [MD/skillwitness-ci-gate-spec.md](MD/skillwitness-ci-gate-spec.md), and the delivery sequence in [PM/skillwitness-roadmap.md](PM/skillwitness-roadmap.md).

## How This Differs From Other Skill Scanners

Several scanners now detect risky patterns in AI skill and MCP packages. This
one is built around the review record, not just the detection list:

- **Organization-owned policy, not built-in verdicts.** Approval decisions live
  in your `approved-tools.yaml` and registry overlay, versioned in your repo.
  The validator applies them; it does not decide for you what is acceptable.
- **Capability evidence survives approval.** An approved MCP service or local
  command no longer creates an approval violation, but it still contributes to
  the skill's L0-L5 behavioral classification.
- **Signed registry overlays.** Per-skill classification, ownership, expected
  license, approval owner, and approval expiry come from an overlay that can be
  digest- or cosign-signed, so a reviewer can prove which policy produced a
  result.
- **Honest scan coverage.** Every scanner reports success, skipped,
  unavailable, or failed with a reason. A pass with three unavailable scanners
  is visibly different from a clean pass. Silent skipping is treated as a bug.
- **Auditable suppressions.** Suppressions are explicit, scoped, and appear in
  the report while keeping the original evidence count visible.
- **Reviewer and machine surfaces.** Markdown for humans, plus SARIF, JUnit,
  JSON, and CycloneDX for code scanning, CI test surfaces, automation, and SBOM
  consumers.
- **Distinguishable exit codes.** `0` pass, `1` policy fail, and dedicated
  non-zero codes for gate misconfiguration, so a broken pipeline never looks
  like a rejected skill.

Detection rules are the commodity layer here and are deliberately shallow
compared to dedicated LLM-assisted scanners. Running this alongside one of those
is a reasonable setup: they find, this one records the decision.

## Security Review Summary

This tool is intended to run as a CI approval gate for AI skills and agent
extensions before they are merged into a managed skills repository. Its purpose
is to make review evidence repeatable: it inspects submitted skill folders,
applies a named risk profile and dependency allow-list, and produces reports
that show whether the submission passed policy, which findings require review,
and which scanner/tool coverage was available during the run.

The validator is read-only with respect to submitted skills. It reads the target
directory, optional metadata overlays, optional allow-list policy files, and
local git state for diff scans; it writes only the explicitly requested report
artifacts. It must not create, normalize, or patch files inside third-party or
author-submitted skill packages. For local scans, `--output`, `--output-dir`,
and `--evidence-output` are rejected when they point inside the scanned target
so CI evidence cannot accidentally modify the reviewed package.

The main security checks are:

- structure and metadata validation for `SKILL.md`
- declared runtime behavior, permissions, hooks, and filesystem access
- local tools, MCP servers, external domains, URLs, scripts, binaries, package
  manifests, and dependency allow-list violations
- telemetry declarations and disable mechanisms
- prompt-injection, obfuscation, dangerous command, and risky script patterns
- secret scanning when supported secret scanners are installed
- license and dependency-license evidence, including conflict detection
- known-vulnerability checks for package manifests and lockfiles
- upstream health checks when a source URL is declared
- signature and SLSA checks when profile policy and binary inputs make them
  applicable
- suppressions, registry overlays, scoring, and final pass/fail enforcement

The validator integrates with CI by returning exit code `0` when the scan passes
and exit code `1` when policy fails. Policy/configuration problems use dedicated
non-zero exit codes so CI can distinguish "the skill failed review" from "the
gate was misconfigured." Reports are available as Markdown for reviewers, JSON
for automation, SARIF for code-scanning systems, JUnit for CI test surfaces, and
CycloneDX for SBOM-oriented consumers.

The tool has limited permissions by design. In the normal local-directory CI
path it needs read access to the skills repository, read access to policy files,
execute access to optional scanner binaries on `PATH`, and write access to CI
artifact storage. It does not need write access to the submitted skill folders.
Network access is not required for purely local rule checks, but it is required
when scanning a git URL, running OSV-Scanner, running OSSF Scorecard, or using
external scanner installations that fetch their own data. CI operators should
make those network egress points explicit in the build environment.

Missing optional tools are reported in scan coverage instead of being hidden.
The Markdown report includes a `Scan Coverage` table with one row per scanner or
external tool, its status, finding count, and skip/failure reason. A green tool
status means that scanner completed; it does not mean the skill has no findings.

## Tools And Integrations

Most policy logic is built into this repository. Third-party tools are used for
specialized evidence where mature scanners already exist, such as secrets,
license detection, dependency vulnerabilities, static analysis, upstream health,
signatures, and SBOM generation. Optional external tools are invoked only when
they are applicable and available; missing, skipped, unavailable, and failed
tool runs are shown in report scan coverage.

### Built-In Scanners And Policy Logic

| Built-in component | What it does | Why it is used |
|---|---|---|
| SkillWitness CLI | Orchestrates local file walking, policy-profile loading, scanner execution, scoring, suppressions, and report generation. | It provides the deterministic CI gate that turns many specialized checks into one auditable pass/fail result. |
| File walker and diff mode | Walks skill folders, honors configured ignore rules, enforces max file size handling, and can limit scans to files changed against a git base ref. | CI needs deterministic target selection so scans are repeatable and do not accidentally include generated or irrelevant test context. |
| Structure scanner | Validates skill layout, `SKILL.md` frontmatter, unknown frontmatter keys, and oversized file reporting. | Skill submissions need a consistent shape before security findings can be interpreted reliably. |
| Permission scanner | Detects declared or implied tool permissions, broad file access, execution intent, and potentially privileged capability requests. | Permission signals are the first indicator that a skill needs stronger review than prompt-only content. |
| Dependency manifest scanner | Detects package-manager manifests and lockfiles such as `package.json` and `package-lock.json`. | Package files indicate dependency supply-chain exposure and determine whether vulnerability and license scanners have useful inputs. |
| Approved dependency scanner | Extracts `mcp`, `local`, and `external` dependencies from skill content and compares them with `approved-tools.yaml`. | Trust decisions depend on explicitly approved tools, binaries, services, and domains; implicit use inside prompts is not sufficient. |
| Network scanner | Finds URLs, domains, and outbound-request command patterns such as `curl` or similar network use. | Network references are common exfiltration and supply-chain paths, so reviewers need them visible and policy-checked. |
| Telemetry scanner | Detects telemetry/tracking indicators and checks for documented disable mechanisms. | Telemetry must be declared and controllable so skill behavior meets privacy and compliance expectations. |
| Script and binary scanner | Detects executable scripts, binary artifacts, and command patterns that imply local execution. | Executable content materially changes the risk profile and may require signatures, provenance, or explicit local-tool approval. |
| Filesystem scanner | Detects read/write behavior, generated artifact output, `.env` references, and sensitive path access patterns. | Skills that read repository state or write derived artifacts can expose source data and need reviewer-visible handling rules. |
| Hook scanner | Detects shell hooks, command wrapping, command-output mutation, and lifecycle hook behavior. | Hooks can change tool behavior outside the visible prompt path, so they need explicit policy review. |
| MCP scanner | Detects Model Context Protocol server/tool references and evaluates them against policy. | MCP tools can grant broad external capabilities, so they must be visible and approved before merge. |
| Prompt-injection scanner | Detects instructions that attempt to override safety boundaries, leak hidden context, or manipulate model behavior. | Skills are prompt artifacts, so prompt-level abuse patterns need first-class checks beyond code scanning. |
| Obfuscation scanner | Detects encoded, compressed, or suspiciously hidden content that can obscure behavior from reviewers. | Hidden payloads make review unreliable and should be surfaced even when no executable file is present. |
| Provenance scanner | Checks registry-overlay source URL/ref and binary checksum metadata when that reviewed metadata is provided. | Reviewers need stable provenance evidence to understand where a submitted skill came from without modifying the submitted package. |
| License scanner | Combines declared license metadata, local license files, package metadata, and optional ScanCode evidence into one license summary. | License acceptance cannot rely on one signal because third-party submissions often contain conflicting or missing evidence. |
| Vulnerability scanner adapter | Decides whether dependency inputs exist, invokes OSV-Scanner when applicable, and maps OSV results into CI findings. | Vulnerability scanning should run only when there is package evidence and should produce policy-normalized findings. |
| Upstream health scanner adapter | Resolves a source URL from the registry overlay or PEP 621 `[project.urls]`, invokes OSSF Scorecard when applicable, and maps low upstream-health signals into findings. | Upstream health only makes sense for skills with declared source repositories, so the gate must distinguish skipped from unavailable coverage. |
| Signature and SLSA scanner adapter | Decides whether signature/SLSA policy applies, checks binary artifacts, and maps cosign/provenance results into findings. | Binary artifacts need stronger integrity evidence than text-only skills, but signature checks should be skipped when there are no binary inputs. |
| Secret scanner orchestrator | Runs whichever supported secret scanners are installed and normalizes their findings into the common report schema. | CI should surface secret exposure without depending on a single detector being present in every environment. |
| Skill metadata registry overlay | Provides reviewer-owned metadata such as source URL, source ref, expected license, owner, classification, approval status, and per-skill allow-list overrides. | It lets CI review third-party or author-submitted skills without modifying the submitted package itself. |
| `approved-tools.yaml` policy | Defines approved MCP tools, local executables, and external domains/services. | It is the policy source for blocking unapproved dependencies before they enter the managed skills repository. |

### External Scanner Tools

| External tool | What it does | Why it is used |
|---|---|---|
| Semgrep | Runs bundled static-analysis rules against submitted skill files and helper code. | It catches code and script patterns that are better expressed as static rules than ad hoc string checks. |
| OSV-Scanner | Scans dependency manifests and lockfiles for known vulnerabilities from the OSV ecosystem. | Dependency CVEs are a primary supply-chain risk, so package evidence needs a vulnerability check in CI. |
| ScanCode Toolkit | Detects license evidence and dependency-license signals with SPDX-style normalization. | License review requires more reliable evidence than package names or README text alone, especially for third-party submissions. |
| Gitleaks | Scans the submitted tree for committed secrets. | Secret exposure is a high-impact failure mode and Gitleaks provides broad token-pattern coverage. |
| TruffleHog | Scans the submitted tree for high-confidence secrets such as keys and credentials. | It complements Gitleaks with additional detectors and verification-oriented secret evidence. |
| detect-secrets | Scans the submitted tree for entropy and known secret patterns. | It adds lightweight local secret coverage and helps catch tokens that other scanners may miss. |
| OSSF Scorecard | Evaluates declared upstream source repositories for maintenance and supply-chain health signals. | When a skill wraps third-party source, upstream health is relevant evidence for Trust/Security review. |
| cosign | Verifies binary signatures and registry-overlay signatures when configured. | Signed artifacts and signed policies give CI a stronger chain of custody for executable content and reviewer-owned metadata. |
| SLSA provenance verification | Uses cosign-backed provenance checks for binary artifacts when profile policy requires an SLSA level. | Binary submissions need build-provenance evidence so reviewers can distinguish trusted builds from arbitrary uploaded executables. |
| Syft | Generates CycloneDX SBOM content when `--format cyclonedx` is requested. | SBOM output lets downstream security tooling consume component evidence in a standard machine-readable format. |
| Git | Supports scanning remote git URLs, checking out a pinned ref, and diff-mode scans against a base ref. | CI and local reviewers need to scan exact submitted revisions and optionally limit findings to changed files. |

### CI Surfaces And Runtime Libraries

| Component | What it does | Why it is used |
|---|---|---|
| TeamCity | Runs the production CI gate and publishes Markdown/SARIF artifacts for managed skill submissions. | It is the intended company CI integration point where pass/fail enforcement and reviewer evidence are produced. |
| GitHub Actions and GitLab CI examples | Provide non-production/example CI templates for external consumers. | They keep the validator usable outside TeamCity without changing the production integration model. |
| Click | Provides the command-line interface and option validation. | A strict CLI layer keeps CI usage predictable and maps invalid input/configuration to stable exit codes. |
| Pydantic and jsonschema | Validate report schemas, metadata overlays, profile files, and registry structures. | Strong schema validation prevents malformed policy or metadata from silently producing misleading security results. |
| PyYAML | Reads profile, registry, metadata, and approved-tools YAML files. | YAML is the operator-friendly format used for CI policy and reviewer-owned overlays. |
| pathspec | Applies `.gitignore`-style file walking rules. | The scanner should respect repository ignore behavior while still keeping scan target selection deterministic. |

## Install

Released versions are published to PyPI. Pin an exact version in CI so the gate
cannot change under a build:

```bash
python -m pip install "skillwitness==0.1.0"
```

Every tagged release also attaches the sdist, the wheel, and a `SHA256SUMS` file
to its [GitHub Release](https://github.com/1stIndeXR/skillwitness/releases).
Download all three and run `sha256sum --check SHA256SUMS` when your supply-chain
policy requires verifying the artifact you install.

## Development

Install the project with development dependencies and run the local checks:

```bash
python3.12 -m venv .venv
.venv/bin/pip install -e ".[dev]"
.venv/bin/ruff check .
.venv/bin/mypy
.venv/bin/pytest
```

The main local verification set is:

```bash
.venv/bin/ruff check src tests
.venv/bin/mypy src
.venv/bin/pytest -q
```

## Usage

The scanner is a read-only approval gate for submitted skill packages. It reads
the target and writes only the requested report artifacts outside that target.
Do not use scan runs to create, normalize, or patch files inside third-party or
author-submitted skill directories; remediation changes should happen in the
source MR, and reviewer-owned artifacts should live in CI artifact storage or a
separate review/policy workspace. For local directory scans, `--output` and
`--output-dir` are rejected when they point inside the scanned target.

Behavioral gate severity is based on file role. Source files, scripts, manifests,
configuration, `SKILL.md`, and files under an operational `references/` directory
can provide executable or policy-gated evidence. Ordinary documentation such as
README and changelog files contributes `LOW` inventory findings for URLs, install
examples, and telemetry descriptions, but does not create dependency, network,
execution, or telemetry blockers by itself. Verified credentials and private keys
remain content-based blockers regardless of file type.

Scan a local skill folder:

```bash
.venv/bin/skillwitness scan tests/fixtures/skills/well-formed
```

Scan with the stricter CI profile:

```bash
.venv/bin/skillwitness scan ./skills/toolctl --profile company-strict
```

Scan a git remote directly, optionally pinned to a branch, tag, or commit:

```bash
.venv/bin/skillwitness scan https://github.com/example/prompt-skill
.venv/bin/skillwitness scan https://github.com/foo/bar --ref abc123def456
```

Scan only files changed against a git base ref:

```bash
.venv/bin/skillwitness scan ./skills --diff origin/main
```

Produce one full report with aggregate and per-skill findings:

```bash
.venv/bin/skillwitness scan ./skills --output skillwitness-report.md
.venv/bin/skillwitness scan ./skills --format json --output skillwitness-report.json
```

Supported `--format` values are `markdown`, `json`, `sarif`, `junit`, and `cyclonedx`.
JSON report output has a published contract in
[examples/report-schema.json](examples/report-schema.json), a JSON Schema 2020-12
document generated from the report model; validate report files against it, or
call `report_json_schema()` from `skillwitness.schemas.report`.
Use `--target-kind auto|skill-package|source-repository` to override the profile's
target-kind setting for one scan. An explicit CLI value has precedence over the
profile. This option does not change `normalization.mode`.

Normalization mode is profile-only: `legacy`, `shadow`, or `enforced`. CI cannot
override it with a command argument. Every report records the resolved mode.
Use `--evidence-output PATH` to write complete raw evidence as stable JSON Lines;
the artifact contains identifiers and locations but no descriptions, snippets,
or detected values.

Bundled profiles use `shadow`, so legacy gate decisions remain authoritative
while normalized groups and metrics are available for evaluation. For an
organization rollout, copy the selected profile into reviewed policy
configuration and follow the
[rollout checklist](MD/false-positive-reduction-rollout.md).
Rollback is the same profile-only change from `enforced` to `shadow`; use
`legacy` only as a temporary compatibility setting. See the
[evaluation](MD/false-positive-reduction-evaluation.md) and
[release notes](MD/false-positive-reduction-release-notes.md) for the measured
release evidence and counter migration.

With automatic detection, a root `SKILL.md` and no source-repository marker selects
`skill-package`. Missing or mixed evidence selects the safer `source-repository`
context. JSON reports include the resolved kind and stable reason codes.

Write split Markdown reports for a skills folder:

```bash
.venv/bin/skillwitness scan ./skills --output-dir skillwitness-reports
```

The output directory contains `index.md` plus one Markdown report per skill.

## CI Integration

Templates for five CI systems ship under `examples/`:

- [examples/github-actions/skillwitness.yml](examples/github-actions/skillwitness.yml)
- [examples/gitlab-ci/skillwitness.yml](examples/gitlab-ci/skillwitness.yml)
- [examples/teamcity/skillwitness.xml](examples/teamcity/skillwitness.xml)
- [examples/jenkins/Jenkinsfile](examples/jenkins/Jenkinsfile)
- [examples/azure-pipelines/skillwitness.yml](examples/azure-pipelines/skillwitness.yml)

See the [portable integration guide](examples/README.md) for the shared
execution contract, container usage, private Git authentication, and
tracker-neutral approval evidence.

Attach the skills repository as the VCS source, publish the generated Markdown
and SARIF reports as build artifacts, and point the scan at your organization's
dependency policy:

```bash
skillwitness scan ./skills --profile company-strict --approved-tools ./policies/example-org/approved-tools.yaml --format markdown --output skillwitness-report.md
skillwitness scan ./skills --profile company-strict --approved-tools ./policies/example-org/approved-tools.yaml --format sarif --output skillwitness-report.sarif
```

`policies/example-org/` holds sample policies only. Replace them with your own.

Scan with a skill metadata overlay:

```bash
.venv/bin/skillwitness scan ./skills --profile company-strict --registry ./policy/approved-skills.yaml
.venv/bin/skillwitness scan ./skills --registry ./policy/approved-skills.yaml --registry-signature ./policy/approved-skills.sig
```

Scan with an external dependency policy:

```bash
.venv/bin/skillwitness scan ./skills --profile company-strict --approved-tools ./policies/example-org/approved-tools.yaml
```

## Refreshing Reports

Exported reports are static files. Code, rule, profile, or allow-list changes do not update existing `.md`, `.json`, SARIF, JUnit, or CycloneDX files automatically. Re-run the scanner whenever you want MR attachments or CI artifacts to reflect the current tool behavior.

Useful refresh commands for local review prep:

```bash
.venv/bin/skillwitness scan ./skills/my-skill --profile company-strict --output my-skill-scan-report.md
.venv/bin/skillwitness scan ./skills --profile company-strict --output-dir skillwitness-reports
```

Generated scan reports should normally be CI artifacts or MR attachments, not committed source files, unless a specific golden fixture is being updated intentionally.

## Metadata And Allow-Lists

CI gates need metadata for repeatable trust decisions: source provenance, pinned refs, expected license, risk classification, ownership, runtime behavior, telemetry declarations, and approval status. The scanner is read-only. It reads submitted package files such as `SKILL.md`, license files, package manifests, scripts, and references; CI-owned trust metadata should come from the registry overlay, not from scanner-generated files inside submitted packages.

Dependency approval is controlled by `approved-tools.yaml`. The scanner uses `--approved-tools PATH` when provided, otherwise the nearest `approved-tools.yaml` found by walking upward from the skill root, or finally the bundled vendor-neutral seed policy in `src/skillwitness/rules/approved_tools.yaml`. The sample policy used by the CI examples lives at `policies/example-org/approved-tools.yaml`. A permission-aware v2 policy example lives at `policies/example-org/approved-tools-v2.yaml`; v2 policy files can use exact strings, wildcard strings, or object entries with `match`, `permissions`, `approval`, and `blocked_external` blocks.

Version 2 rejects unknown top-level and nested entry fields, validates type-specific match and permission keys, and accepts only `approved` or `review-required` approval statuses. Provider IDs and dependency-entry IDs must be unique, and every MCP provider, domain-group, or registry reference must resolve before scanning. MCP entries can opt into deterministic operation authorization with exact canonical `allowed_operations` and `denied_operations` tokens. Deny entries take precedence, absent operations fail closed, and service-only or dynamic evidence produces an explicit unknown-operation finding. Every decision identifies the policy entry, provider, service and operation when known, resolution method, and evidence location. Runtime scope declarations remain visible as unverified service-side requirements.

Local entries can opt into command scope authorization with `allowed_scopes` and `denied_scopes`. Scope patterns describe command arguments: `status:*` authorizes `status` plus further arguments, a pattern without the trailing `:*` authorizes exactly that argument list, and a pattern that would approve every scope is rejected at policy load. Scope evidence comes from `allowed-tools` declarations, so `Bash(git status:*)` is authorized against the allowlist while `Bash(git:*)` and `Bash(git)` declare no bounded argument scope and report `DEP_COMMAND_SCOPE_UNKNOWN` instead of matching any pattern. Denied scopes are applied before allowed scopes, and every decision records the matched entry, pattern, resolved scope, resolution method, and evidence location.

Local entries can also correlate network intent with `allowed_methods`, `allowed_domain_refs`, `denied_domains`, and `download_execute`. `curl` and `wget` invocations in `allowed-tools` declarations and in scanned scripts are parsed for command, HTTP method, destination host, and whether the download is piped into an execution sink, then classified as `read`, `upload`, or `download-execute`. Referenced external entries act as the domain groups, so an internal mirror is expressed as a normal external entry. Decisions apply in order: unresolved destination, denied domain, download-execute without approval, disallowed method, destination outside the domain groups. A destination assembled at runtime reports `DEP_NETWORK_INTENT_UNKNOWN` and never matches an allowlist.

Known non-enforcing permission metadata is retained in the parsed policy for audit. The `policy_rules` section and descriptive MCP provider metadata remain documentary and do not affect gate decisions. The current implementation also emits policy findings for placeholders, MCP wildcard approvals, review-required entries, runtime guardrail requirements, and blocked external dependencies; deeper command/domain permission-context scoring is still being developed. Per-skill registry `approved_tools` entries override the repo-wide policy for that skill. Unapproved `mcp`, `local`, or `external` dependencies are `HIGH` by default. A version 2 `defaults.unapproved_dependency_severity` value takes precedence when supplied; otherwise the profile's `dependency_approval.unapproved_severity` applies. Operators should keep organization-specific allow-lists outside the generic validator and outside submitted skill packages, either in a fork/internal policy repository or as a CI-managed overlay before running the validator.

External tool integrations are best-effort but explicit. Missing optional tools such as Semgrep, OSV-Scanner, ScanCode, Gitleaks, TruffleHog, or detect-secrets are reported as informational coverage findings instead of silently disappearing. CycloneDX metadata records Syft as successful, unavailable, failed, or skipped.

A profile can pin that tool environment with an optional `external_tools` block: per-tool `approved_versions`, `install_source`, `sha256`, and `timeout_seconds`, plus a `default_timeout_seconds` for declared tools. Verification runs once per scan. A checksum mismatch (`TOOL_ENV_CHECKSUM_MISMATCH`, BLOCKER), an unapproved version (`TOOL_ENV_VERSION_NOT_APPROVED`, HIGH), or an unreadable version (`TOOL_ENV_UNVERIFIED`, HIGH) stops that tool from running and reports `failed` coverage for the scanner it backs. A tool that simply is not installed stays ordinary `unavailable` coverage. Observed versions appear in the scan-coverage table. Subprocesses inherit the runner's proxy and certificate environment unchanged; `osv-scanner`, `scorecard`, and `cosign` are the tools that reach the network, and what they disclose is documented in the [spec](MD/skillwitness-ci-gate-spec.md#101a-external-tool-environment).

An optional `performance` block bounds one scan: `max_scanned_files` caps the files scanned per skill and reports the gap as `STRUCT_TRAVERSAL_CAPPED` instead of truncating silently, `max_external_output_bytes` refuses oversized tool output instead of parsing it, and `upstream_cache_ttl_hours` / `vulnerability_cache_ttl_hours` control the permitted Scorecard and OSV result caches (the vulnerability cache is keyed by a digest of the scanned manifests and lockfiles, and is disabled by default). Scorecard runs once per repository URL per scan, and bundled rule sets are parsed once per process. Per-scanner durations stay out of the report so its bytes remain deterministic; write them with `scan --timing-output PATH`.

A profile can turn that visible coverage into an enforced contract with an optional `coverage` block that names required scanners per target kind (`skill-package` and `source-repository`) plus the severity for `unavailable`, `failed`, and incomplete coverage. Bundled profiles declare no required scanners, so this is opt-in for organization-owned profiles. Required coverage is evaluated per skill: a multi-skill scan whose aggregate row reads `success` because one skill was covered still fails when another skill's required scanner was skipped or never ran. Unknown scanner identifiers are rejected at profile load with exit code `4`; the accepted identifiers are listed in the [spec profile section](MD/skillwitness-ci-gate-spec.md#10-configuration).

## Skill Metadata Overlay

For skills submitted to the managed skills repository, use a reviewed metadata overlay as the org-managed source of truth for CI-owned trust metadata. The overlay declares per-skill `source_url`, `source_ref`, `license_expected`, `classification`, `owner_team`, approval metadata, optional `approval_ref` and `approval_url` evidence, notes, and optional per-skill `approved_tools` overrides for `mcp`, `local`, and `external` dependencies. It does not fetch or rescan remote upstream repositories.

Keep the metadata overlay and dependency allow-list outside submitted skills: for example in a policy repository, a CI-managed workspace, or an internal fork dedicated to security review. Point the validator at the overlay with `--registry` and at the org-wide dependency policy with `--approved-tools`. If the overlay is signed, pass `--registry-signature` and optionally `--registry-cert`; invalid signatures exit with policy configuration error code `4`.

Verify the same registry independently with the option-based standalone command;
`--certificate` is optional for cosign signatures and any invalid signature exits
with code `4`:

```bash
skillwitness verify \
  --registry ./policy/approved-skills.yaml \
  --signature ./policy/approved-skills.sig \
  --certificate ./policy/approved-skills.pem
```

Generate a starter overlay from an existing skills tree:

```bash
.venv/bin/skillwitness generate-registry-template ./skills > approved-skills.yaml
```

The template exits `0` and writes v3 YAML to stdout. Operators fill in the TODO metadata, review the dependencies and classifications, then sign and publish the overlay. Downstream tooling can validate overlay shape with `examples/registry-schema.json` or by calling `registry_json_schema()` from `skillwitness.registry.schema`.

### Registry Coverage

When `--registry` is omitted, the validator scans package-local evidence only. It does not expect a separate metadata file in the submitted skill and does not emit missing-metadata findings for one.

When a registry is provided and a skill has no matching entry, the scanner emits `SKILL_NO_REGISTRY_ENTRY`. Missing entries are `HIGH` for L3+ skills, `MEDIUM` for L0-L2 skills, and `LOW` in the `fast-precommit` profile. A profile can also require registry coverage without passing `--registry` by setting `registry.require: true`.

Install Semgrep outside the project virtualenv, then put its `bin` directory on `PATH` before running reports. The Semgrep Python package currently pins `click` below the version required by ScanCode, so installing both into the same virtualenv can break dependency-license scanning.

```bash
python3.12 -m venv /tmp/skillwitness-semgrep
/tmp/skillwitness-semgrep/bin/python -m pip install semgrep
CERT_FILE=$(/tmp/skillwitness-semgrep/bin/python -c 'import certifi; print(certifi.where())')
PATH=/tmp/skillwitness-semgrep/bin:$PWD/.venv/bin:$PATH \
  SSL_CERT_FILE=$CERT_FILE \
  REQUESTS_CA_BUNDLE=$CERT_FILE \
  .venv/bin/skillwitness scan ./skills --profile company-strict --output skillwitness-report.md
```

## Adoption And Triage

Pilots rarely start from zero findings. `scan --write-baseline baseline.json` records the findings the gate decided on, and `scan --baseline baseline.json` then gates on new findings only: baselined findings stay in the report and in `summary.baselined_findings` but no longer decide pass or fail. Baseline keys come from the normalized fingerprint when one exists, so regenerate the baseline after changing a profile's normalization mode.

Suppressions are organization-owned and live in the registry overlay, never in the submitted package. Each entry declares `rule_id`, `path`, `reason`, `owner`, `approved_by`, `approved_at`, `expires`, and an optional `approval_ref`. A suppression whose lifetime exceeds the profile's `suppressions.max_days` suppresses nothing and reports `SUPPRESSION_INVALID`; expired entries report `SUPPRESSION_EXPIRED`; `BLOCKER` findings need `allow_blocker_suppressions`. Every decision — applied, unused, expired, rejected — is written by `scan --suppression-audit-output audit.jsonl`.

For review surfaces, `--format annotations` emits GitHub Actions workflow commands and `--format codequality` emits GitLab Code Quality JSON. Both cap at 50 findings and state the omitted count in their own output, so a truncated list is never mistaken for a complete one.

`scan --metrics-output metrics.json` records what the validator can measure during a shadow pilot: findings by severity and disposition, override use, baseline use, and coverage counts. Reviewer agreement and false-positive rate need a reviewer decision per finding, so the artifact names those reviewer-supplied fields and carries the denominators. The pilot formulas and the criteria for promoting a rule set from report-only to blocking are in [spec section 11B](MD/skillwitness-ci-gate-spec.md#11b-adoption-and-triage).

## Exit Codes

- `0`: scan completed and policy passed
- `1`: scan completed and policy failed
- `2`: invalid input
- `3`: internal error
- `4`: policy or registry configuration error

## Branch Protection

The default branch should require the `CI / Lint, type-check, and test` workflow job to pass before merge. The rule should also require pull requests for changes to `main`; SARIF upload and release automation are intentionally deferred until later roadmap tasks.
