Metadata-Version: 2.4
Name: scovant-core
Version: 0.19.0
Summary: Open-source, evidence-first scanner for passive AI-agent readiness signals.
Project-URL: Homepage, https://github.com/Scovant/scovant-core
Project-URL: Documentation, https://github.com/Scovant/scovant-core/tree/main/docs
Project-URL: Changelog, https://github.com/Scovant/scovant-core/blob/main/CHANGELOG.md
Project-URL: Scovant Cloud, https://scovant.com/?utm_source=scovant-core&utm_medium=pypi&utm_campaign=oss
Author-email: Scovant <bot@scovant.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai-agents,llms.txt,mcp,robots.txt,scanner,structured-data
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: Site Management
Requires-Python: >=3.12
Requires-Dist: beautifulsoup4<5,>=4.12
Requires-Dist: httpx<1,>=0.27
Requires-Dist: lxml<7,>=5.2
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: tldextract<6,>=5
Provides-Extra: dev
Requires-Dist: jsonschema<5,>=4.20; extra == 'dev'
Requires-Dist: mypy==1.20.2; extra == 'dev'
Requires-Dist: pytest-cov<7,>=5; extra == 'dev'
Requires-Dist: pytest<9,>=8; extra == 'dev'
Requires-Dist: pyyaml<7,>=6; extra == 'dev'
Requires-Dist: ruff==0.16.6; extra == 'dev'
Requires-Dist: tiktoken<1,>=0.7; extra == 'dev'
Requires-Dist: twine<7,>=5; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: mcp<3,>=2.2; extra == 'mcp'
Provides-Extra: tokens
Requires-Dist: tiktoken<1,>=0.7; extra == 'tokens'
Description-Content-Type: text/markdown

# Scovant Core

## Definition

Scovant Core is an open-source, evidence-first scanner for passive AI-agent readiness signals on websites.

Scovant Core measures what a site declares. Scovant Cloud measures what real agents actually experience.

Core does NOT execute real agents.

It reads what the site already exposes on request: robots/crawler policy,
structured data, llms.txt, machine-readable discovery, MCP metadata, WebMCP
declaration, commerce metadata, HTTP semantics, and its own usage in CI.

> A high Technical Readiness rating does not prove that autonomous agents can complete real workflows on the site.

## 30-second install

```bash
uvx --from scovant-core scovant scan https://example.com   # no install, needs uv
pipx run --spec scovant-core scovant scan https://example.com
pip install scovant-core                                    # Python >= 3.12
pip install "scovant-core[tokens]"                          # + bundled tokenizer for PAGE_TOKEN_BLOAT
npx @scovant/core scan https://example.com                  # Node launcher, needs uv/pipx/python3
```

### Reproducible install

Same Core version does not guarantee the same dependency graph months
later. Each release ships exact pins:

    pip install "scovant-core==0.19.0" -c https://raw.githubusercontent.com/Scovant/scovant-core/v0.19.0/constraints/constraints-0.19.0.txt

Every report records what actually ran (`provenance.dependencies`,
`provenance.environment_digest`).

## Example output

Running `scovant scan https://example.com` against a
well-instrumented commerce site produces text output shaped like this (real
output, rendered — never from a live scan — from the `commerce-good` test
fixture; the established readiness checks have nothing to report, so "Top
findings" carries only the site's three `[security]`-tagged findings — the
SECURITY checks are never rated, but a FAIL/WARN among them still surfaces
in "Top findings", tagged `[security]`, appended after any readiness
FAIL/WARN). "Diagnostics" tallies the readiness checks' own verdicts; they
are diagnostics and carry no weight in Technical Readiness. The
"Experimental (not scored)" block lists every
experimental check that evaluated to something other than `N/A` on this
scan (including a WARN, as below — experimental findings are still shown,
just never scored), plus a compact "N/A: ..." line naming the experimental
checks that had nothing to evaluate — so every check the header counts is
named somewhere, never merely counted. No check affects Technical
Readiness; pass `--experimental` to also run the experimental checks gated
behind that flag (omit it and they're skipped entirely, not evaluated as
N/A) and rank experimental findings with the rest — the scan's scope is then
`CUSTOM`. The "Agentic Security & Trust" block below it is the 16 SECURITY
checks' own summary — see `docs/security.md` for the full boundary of what
it does and does not test.

```text
SCOVANT CORE
──────────────────────────────────────────────

Target: https://example.com/
Profile: commerce (auto → commerce, confidence 0.85)
Core version: 0.19.0

Technical Readiness  82 / 100  B · coverage 97% · model R2.1@67df90af85b0
Not measured by Core: MCP_DISCOVERY_INVALID (handshake_not_performed)
Diagnostics (50 readiness checks): 40 pass · 1 warn · 0 fail

Capabilities detected (descriptive, not scored)
  mcp: present
  webmcp: absent
  ucp: present
  llms_txt: present
  openapi: not_checked
  oauth: not_checked
  content_signal: present
  security_txt: present

AgentReady v1.0 (descriptive, not scored): MUST 3/3 measured — 3 pass · SHOULD 5/12 measured — 5 pass · MAY none measured (0/3)
  Mapping: docs/standards/agentready.md

68 checks
51 PASS
3 WARN
1 FAIL
13 N/A
0 ERROR

Top findings

FAIL  [security] CORE-SECURITY-003
      Neither Content-Security-Policy nor X-Frame-Options is sent.

WARN  [security] CORE-SECURITY-002
      No Strict-Transport-Security header.

WARN  [security] CORE-SECURITY-005
      Missing: referrer-policy, x-content-type-options.

Experimental (not scored)
──────────────────────────────────────────────
PASS  CORE-ACCESS-011
      llms.txt links useful same-origin pages and carries no misplaced policy or template text.

PASS  CORE-INTERFACE-008
      A UCP profile is published and valid.

PASS  CORE-MACHINE-012
      The structured price matches a visible price on the page.

WARN  CORE-OPERABILITY-011
      3 of 3 machine surface(s) are not linked from anything an agent reads: llms_txt, mcp, ucp.

PASS  [security] CORE-SECURITY-009
      No administrative/destructive interfaces advertised.

PASS  [security] CORE-SECURITY-011
      No indicators found.

PASS  [security] CORE-SECURITY-012
      No indicators found.

PASS  [security] CORE-SECURITY-013
      No indicators found.

PASS  [security] CORE-SECURITY-014
      No indicators found.

PASS  [security] CORE-SECURITY-015
      No indicators found.

PASS  [security] CORE-SECURITY-016
      No indicators found.

N/A: CORE-INTERFACE-004, CORE-INTERFACE-009, CORE-OPERABILITY-007, CORE-SECURITY-010, CORE-SECURITY-017, CORE-SECURITY-018

AGENTIC SECURITY & TRUST  PASSIVE SIGNALS ONLY
──────────────────────────────────────────────
Critical                  0
High                      0
Medium                    1
Low                       2
auth                      PASS 0  WARN 0  FAIL 0
Data exposure             PASS 3  WARN 0  FAIL 0
Disclosure                PASS 1  WARN 0  FAIL 0
Prompt surface            PASS 6  WARN 0  FAIL 0
Web baseline              PASS 1  WARN 2  FAIL 1
Observed authorization    NOT TESTED
Verified agent identity   NOT TESTED
Prompt-injection resilience NOT TESTED
Tool invocation safety    NOT TESTED
This section evaluates tested AI-agent security controls and machine-facing security signals. It is not an overall website or application security rating.

Not tested by Scovant Core
──────────────────────────────────────────────
Observed WAF access       NOT TESTED
Real agent tasks          NOT TESTED
MCP tool execution        NOT TESTED
WebMCP state parity       NOT TESTED
Multi-model reliability   NOT TESTED
Regression stability      NOT TESTED

Verify with real agents:
https://scovant.com/scan?utm_source=scovant-core&utm_medium=cli&utm_campaign=oss
```

The headline is **Technical Readiness** (`readiness.technical` — the open
R2 model, see [`docs/r2.md`](docs/r2.md)): `--min-score` and
`--require-canonical` gate on it, and a scan whose Technical Readiness has
no number fails `--min-score`. It is Core's only rating: the five-domain
Static Signal Score that 0.18.0 deprecated was removed in 0.19.0 (see the
[CHANGELOG](CHANGELOG.md) for the migration notes).

## What Core checks

68 checks total: 50 readiness checks across 5 categories (7 experimental)
plus 18 unscored security checks. The 50 readiness checks (7 of them
experimental — they run and report in their own block; see below):

- **Access & Discovery** (11) — HTTPS reachability, robots.txt, AI crawler
  policy, training-vs-search crawler separation, sitemap availability and
  freshness, canonical URLs, indexability, llms.txt, Content-Signal, and
  (experimental) llms.txt utility (useful links, no misplaced policy text,
  not a template)
- **Machine Understanding** (12) — JSON-LD parseability, Organization and
  WebSite/WebPage entities, Product/Offer structured data, product
  identifier count, breadcrumbs, metadata quality, heading structure,
  language declaration, image alt coverage, and (experimental) visible-vs-
  structured price consistency
- **Agent Interfaces** (9) — MCP discovery and server-declaration quality,
  WebMCP static presence and (experimental) tool-declaration quality,
  OpenAPI discovery, OAuth authorization-server and protected-resource
  metadata, and (experimental) UCP profile validity and agent discovery
  surface presence (A2A cards, AI-plugin manifests, agents.json, Agent
  Skills)
- **Trust & Commerce** (7) — contact/support, shipping, returns/refund,
  privacy, terms, `security.txt`, and pricing discoverability
- **Operability & Efficiency** (11) — server-rendered core content,
  redirect-chain complexity, cache validators, broken machine-consumable
  endpoints, agent parse cost, form/control labels, unknown paths return a
  real 404, rate-limiting (429) is signalled with `Retry-After`, bot-
  challenge pages are not served as HTTP 200, and (experimental,
  `--experimental` only) machine-reference integrity and discovery linkage

### Experimental checks

Seven checks ship `experimental`: they evaluate and appear in every report,
in their own block, until they have been calibrated against real-world
traffic and promoted. Like every check, they never affect Technical
Readiness.

| ID | Title |
|---|---|
| `CORE-ACCESS-011` | llms.txt utility |
| `CORE-INTERFACE-004` | WebMCP tool declaration quality |
| `CORE-INTERFACE-008` | UCP profile validity |
| `CORE-INTERFACE-009` | Agent discovery surface presence |
| `CORE-MACHINE-012` | Visible vs. structured price |
| `CORE-OPERABILITY-007` | Machine reference integrity |
| `CORE-OPERABILITY-011` | Discovery linkage |

`CORE-OPERABILITY-007` is additionally gated on the `--experimental` flag
itself — it is the only check whose gatherer (`reference_integrity`) makes
any network request at all beyond the ordinary scan, so a plain `scovant
scan` never resolves an external package registry or DNS name for it. The
other six evaluate on every scan, reported in their own block.

See [`docs/checks.md`](docs/checks.md) for the full catalog — every check's
id, applicable profile, and why it matters — generated straight
from the check registry so it can never drift from what the package
actually runs.

### Agentic Security & Trust (passive)

16 more checks (`CORE-SECURITY-001..016`, 8 of them experimental) report
passive, declared/observed security signals — HTTP security headers, HSTS,
CSP, cookie flags, `security.txt` validity, credential-like values or
internal-network references exposed in machine-facing surfaces, privileged
endpoints advertised to agents, and prompt-injection-shaped content in
`llms.txt`/MCP tool descriptions. They are never rated and never change
Technical Readiness — every scan reports the same
`PASSIVE SIGNALS ONLY` banner and the same four `NOT TESTED` lines
(observed authorization, verified agent identity, prompt-injection
resilience, tool invocation safety). Full boundary, verification-mode
semantics, and standards mapping: [`docs/security.md`](docs/security.md#agentic-security--trust-passive-checks).

## What Core does NOT test

Core is a static, unauthenticated scanner. It never opens a browser and
never observes real traffic, so it cannot tell you:

- Observed WAF/bot-firewall access
- Real agent task completion
- MCP tool execution
- WebMCP state parity (runtime, not just static presence)
- Multi-model reliability
- Regression stability over time

## Standards

Scovant Core maps its checks to **AgentReady v1.0** (<https://agentready.org/>, MIT), the open
baseline standard for agent-facing website signals. AgentReady defines requirements, not weights;
Core reports which requirements its checks measure (`metrics.standards.agentready_v1` in every
report) and never reproduces any third-party score. Mapping: `docs/standards/agentready.md`.

## Core vs Cloud

Scovant Core measures passive, machine-facing signals. Scovant Cloud verifies how real agents actually behave, across providers, browser runtimes, security layers and time.

See [`docs/core-vs-cloud.md`](docs/core-vs-cloud.md) for the full capability
matrix and concrete "Core can determine / cannot determine / Cloud can
test" examples, and [scovant.com/open-source](https://scovant.com/open-source)
for the same comparison as a live, always-current page.

Which readiness rules read the same signal as each check:
[`docs/crosswalk.md`](docs/crosswalk.md).

## GitHub Action

```yaml
- uses: Scovant/scovant-core@v0
  id: scan
  with:
    url: https://staging.example.com
    min-core-score: '70'     # Technical Readiness rating; no number fails too
    fail-on: fail
    allow-private-networks: 'true'
    trusted-target: 'true'
```

`@v0` is the moving major tag while the package is pre-1.0 — see
[`docs/releasing.md`](docs/releasing.md); pin `@v0.19.0` instead for an
exact, never-moving version. `allow-private-networks` is what makes this
example work against a private CI runner scanning its own not-yet-public
staging host — see "Free boundary" below.

The action installs itself from the checked-out ref (`pip install
"$GITHUB_ACTION_PATH"` — no PyPI dependency, no version skew between the
tag you pinned and the code that runs), scans one URL, writes a Markdown
summary to the job's Step Summary, uploads the full report (HTML by
default) as a workflow artifact named `scovant-core-report`, and fails the
step when the gate you configured trips.

### Inputs

| Input | Description | Default |
|---|---|---|
| `url` | Entry URL to scan | *(required)* |
| `profile` | `auto`\|`content`\|`commerce`\|`saas`\|`api` | `auto` |
| `min-core-score` | Fail the step below this Technical Readiness rating, or when it has no number (empty = no gate) | `''` |
| `fail-on` | `never`\|`fail`\|`warn` | `never` |
| `experimental` | Run the experimental checks too (scope `CUSTOM`) | `false` |
| `timeout` | Total scan budget in seconds | `60` |
| `report-format` | `html`\|`markdown` — the uploaded artifact | `html` |
| `allow-private-networks` | Allow private/loopback targets (your own staging) | `false` |
| `trusted-target` | Required (`true`) when `allow-private-networks` is `true`; never honoured for pull requests from forks | `false` |
| `require-canonical` | Fail the step unless Technical Readiness status is `OK` | `false` |
| `fail-on-security` | `never`\|`critical`\|`high`\|`medium` — fail the step on a FAILed security finding at or above this severity (a WARN never gates) | `never` |

### Outputs

| Output | Description |
|---|---|
| `score` | Technical Readiness rating (0-100, empty when it has no number or was not computed) |
| `grade` | Technical Readiness grade (empty when there is no grade, e.g. `DEGRADED`) |
| `pass_count` | Number of PASS findings |
| `warn_count` | Number of WARN findings |
| `fail_count` | Number of FAIL findings |
| `report_path` | Path to the uploaded report file on the runner |
| `coverage` | Technical Readiness evidence coverage ratio (0.0-1.0; empty when not computed) |
| `score_status` | Technical Readiness status: `OK`\|`DEGRADED`\|`INSUFFICIENT_EVIDENCE` (empty when not computed) |
| `scan_scope` | Check selection scope: `CANONICAL`\|`CUSTOM` |
| `error_count` | Number of checks that errored |
| `security_critical` | Number of `critical`-severity findings in the Agentic Security & Trust section (FAIL + WARN) |
| `security_high` | Number of `high`-severity findings in the Agentic Security & Trust section (FAIL + WARN) |
| `security_fail_count` | Number of FAIL findings in the Agentic Security & Trust section |
| `security_warn_count` | Number of WARN findings in the Agentic Security & Trust section |
| `model_id` | Technical Readiness model id, e.g. `R2.1@67df90af85b0` (empty when not computed) |

`pass_count`/`warn_count`/`fail_count` are readiness-only (the Agentic
Security & Trust section is never scored — see `Report.security.scored` —
so a security finding never changes what those three numbers mean); the
`security_*` outputs above make the security population visible on its own.

### Scanning private networks

> **WARNING:** Never derive the target URL from untrusted PR input when
> `allow-private-networks: true`. On a self-hosted runner that combination
> turns the scanner into an internal-network probe.

`allow-private-networks: true` is honoured only together with
`trusted-target: true`, and never for pull requests from forks — the
action exits 2 otherwise. A pull request whose head repository is unknown
(for example, a deleted fork) is treated as a fork. Take the URL from a
repository variable, not from the event:

    with:
      url: ${{ vars.STAGING_URL }}
      allow-private-networks: 'true'
      trusted-target: 'true'

### Free boundary

One URL, one run, no state, no comparison across runs, no schedule, no
outgoing notification. For history, regression detection, real-agent
simulation, and CI-triggered re-scans across your whole site, see
[Scovant Monitor](https://scovant.com/pricing?utm_source=scovant-core&utm_medium=github&utm_campaign=oss).

## JSON, Markdown and HTML output

```bash
scovant scan https://example.com --format json
```

produces a versioned JSON report (`schema_version`, `core_version`,
`ruleset_version`, Technical Readiness in `readiness.technical` with its
per-rule `technical_outcomes`, every finding with its own evidence) suitable for CI pipelines, matching the published
[`docs/report.schema.json`](docs/report.schema.json). Combine with
`--min-score`/`--fail-on` for a CI-friendly exit code, or `--output
report.json` to write it to a file. `--fail-on-security
never|critical|high|medium` gates independently on the Agentic Security &
Trust section (exit code `6` — a WARN never gates, only a FAIL at or above
the given severity; that section is never scored, so this flag is separate
from `--fail-on`/`--min-score`).

```bash
scovant scan https://example.com --format markdown --output report.md
```

renders the same report as GitHub-flavored Markdown — the Technical
Readiness headline, findings grouped by status with collapsible evidence — ready to
paste into a pull request or issue.

```bash
scovant scan https://example.com --format html --output report.html
```

renders the same report as one self-contained HTML document (inline CSS,
no external resources, no JavaScript) — open it directly in a browser or
attach it as a CI artifact.

## MCP

```bash
pip install "scovant-core[mcp]"
scovant mcp
```

starts a stdio MCP server so an agent host (Claude Desktop, Cursor, or any
MCP client) can run Scovant Core's passive scan as a tool call instead of a
shell command. Claude Desktop (`claude_desktop_config.json`) and Cursor
(`.cursor/mcp.json` or Settings → MCP) both take the same config:

```json
{
  "mcpServers": {
    "scovant-core": {
      "command": "scovant",
      "args": ["mcp"]
    }
  }
}
```

| Tool | Description |
| --- | --- |
| `scan_site` | Run all 68 checks against one public URL, return the JSON report |
| `get_core_score` | Run the scan, return the Technical Readiness summary |
| `list_checks` | List every check with category, profiles, experimental flag |
| `explain_check` | Explain one check: why it matters, limitations, Cloud extension, references |
| `get_finding` | Return one finding from a scan this server already ran (no re-scan) |

Every tool is passive/static only — the same evidence gathering `scovant
scan` does (public documents: robots.txt, sitemaps, llms.txt, JSON-LD,
MCP/OAuth/UCP discovery files), never a browser, never an MCP `tools/call`,
never authentication. The server runs one scan at a time and keeps only the
last 8 reports in memory for `get_finding` to read back.

### Claude Code plugin

```
/plugin marketplace add Scovant/scovant-core
/plugin install scovant@scovant
```

See [`claude-plugin/README.md`](claude-plugin/README.md) for the bundled
skills and their MCP wiring.

## Methodology

Full scoring methodology, the status/confidence model, the coverage floor,
and rule versioning: [`docs/methodology.md`](docs/methodology.md).

## Security

Scovant Core fetches arbitrary public URLs; see
[`docs/security.md`](docs/security.md) for the network safety model
(scheme allow-list, SSRF guard, redirect/size/timeout limits). Scanning your
own private/staging infrastructure needs an explicit opt-in — see
[`docs/security.md` § Private-network targets (opt-in)](docs/security.md#private-network-targets-opt-in).
To report a vulnerability, see [`SECURITY.md`](SECURITY.md) or email
`security@scovant.com`.

### Community contributions (opt-in)

`--contribute` is off by default. When you pass it, Core sends ONE HTTPS
POST, after the scan finishes, to a single fixed Scovant endpoint — a
measurement of the PUBLIC site you just scanned, for the aggregate,
opt-in community index at
[scovant.com/research](https://scovant.com/research):

```bash
scovant scan https://example.com --contribute
```

What is sent (exactly these fields, nothing else):

- `domain` — the scanned site's hostname (no path, no query string)
- `timestamp` — when the scan completed
- `core_version`, `ruleset_version`, `ruleset_digest` — which Core build and
  checklist produced the result
- `profile`, `profile_confidence`, `profile_detector_version` — the resolved
  site profile (`content`/`commerce`/`saas`/`api`) and how it was detected
- `experimental` — whether experimental checks were included
- `check_statuses` — each check id's status (`PASS`/`WARN`/`FAIL`/`N/A`/`ERROR`)
- `scan_scope` — `CANONICAL` (only canonical scans are sent)
- `technical` — Technical Readiness: rating, grade, coverage, status and
  model id (only a scan whose Technical Readiness status is `OK` is sent)
- `error_count` — how many checks errored

What is never sent: no page HTML, no evidence, no URLs beyond the domain
itself, no IP address, no filesystem paths, no CLI flags beyond what's
listed above. The request carries no cookies and never follows a redirect.

A contribution is a client-provided measurement, and it is treated as one:
unverified discovery signals only.
Client-provided scores are never incorporated directly into authoritative Scovant research datasets
— a contributed domain is independently re-evaluated by Scovant-controlled
infrastructure before anything about it is published.

`--contribute` is refused (exit 2) together with `--allow-private-networks`
— only a target Core itself already confirmed public gets contributed. A
contribution never changes the CLI's exit code and never fails the scan;
one line on stderr (`contributed: <domain> (accepted|deduplicated|failed:
<reason>)`) reports the outcome. See
[`docs/security.md` § Outbound requests](docs/security.md#outbound-requests)
for the full network contract.

## Contributing

See [`CONTRIBUTING.md`](CONTRIBUTING.md). For how releases are made,
verified, and pinned, see [`docs/releasing.md`](docs/releasing.md).

## License

Apache-2.0. See `LICENSE`.
