Scovant Core — https://example.com/step4
Profile: commerce (requested auto, confidence 85%)
Core version: 0.1.0
1 PASS, 23 WARN, 8 FAIL, 13 N/A, 0 ERROR (45 checks)
Score
35 / 100
Grade: F · Coverage: 100%
Categories
| Category | Weight | Score | Evaluated / applicable |
|---|---|---|---|
| Access & Discovery | 25 | 17 | 23 / 23 |
| Machine Understanding | 25 | 41 | 22 / 22 |
| Agent Interfaces | 20 | n/a | 0 / 0 |
| Trust & Commerce | 15 | 42 | 12 / 12 |
| Operability & Efficiency | 15 | 50 | 13 / 13 |
Findings
| Status | ID | Title | Severity | Summary |
|---|---|---|---|---|
| FAIL | CORE-ACCESS-002 | robots.txt availability and syntax | medium | robots.txt disallows all crawlers from the entire site. |
| FAIL | CORE-ACCESS-003 | AI search crawler policy | high | robots.txt declares every major search and answer-engine crawler as disallowed. |
| FAIL | CORE-ACCESS-005 | Sitemap availability | medium | No sitemap was found. |
| FAIL | CORE-ACCESS-007 | Canonical URL integrity | medium | The canonical URL points off-host. |
| FAIL | CORE-ACCESS-008 | Indexability | high | The entry page declares noindex. |
| FAIL | CORE-MACHINE-005 | Offer price, currency, and availability | high | The Product entity has no Offer with price, currency, or availability. |
| FAIL | CORE-OPERABILITY-004 | Broken machine-consumable endpoints | high | 2 of 2 machine-consumable reference(s) do not resolve. |
| FAIL | CORE-TRUST-004 | Privacy policy discoverability | high | No privacy policy page was found linked from the entry page. |
| WARN | CORE-ACCESS-001 | HTTPS reachability | low | The entry URL redirects 4 times before settling. |
| WARN | CORE-ACCESS-004 | Training vs. search crawler separation | low | Search/retrieval crawlers are restricted together with training crawlers — intent unclear. |
| WARN | CORE-ACCESS-009 | llms.txt presence and integrity | low | llms.txt references 1 link(s) that do not resolve. |
| WARN | CORE-ACCESS-010 | Content-Signal declaration | low | Content-Signal is declared but has syntax error(s). |
| WARN | CORE-MACHINE-001 | JSON-LD parseability | medium | Some JSON-LD blocks on the sampled pages fail to parse as valid JSON. |
| WARN | CORE-MACHINE-002 | Organization entity | medium | No Organization entity was found on the sampled pages. |
| WARN | CORE-MACHINE-003 | WebSite/WebPage entity | low | No WebSite or WebPage entity was found on the sampled pages. |
| WARN | CORE-MACHINE-004 | Product structured data | high | A Product entity exists but carries no identifiers. |
| WARN | CORE-MACHINE-006 | Product identifier count | medium | Product entities declare no stable identifiers. |
| WARN | CORE-MACHINE-008 | Metadata quality | low | The entry page's metadata has issues: og:title, og:description. |
| WARN | CORE-MACHINE-009 | Heading structure | low | The entry page's heading structure has issues: multiple H1. |
| WARN | CORE-MACHINE-010 | Language declaration | low | The entry page's `lang` attribute is absent or invalid. |
| WARN | CORE-MACHINE-011 | Image alt coverage | medium | Only 0/1 images have an alt attribute (ratio 0%). |
| WARN | CORE-OPERABILITY-001 | Server-rendered core content | medium | The entry page's static HTML carries only 148 chars of visible text (< 200). |
| WARN | CORE-OPERABILITY-002 | Redirect chain complexity | low | The entry URL redirects 4 times before settling. |
| WARN | CORE-OPERABILITY-003 | Cache validators | info | The entry response carries no cache validator (ETag, Last-Modified, or Cache-Control). |
| WARN | CORE-OPERABILITY-006 | Form/control labels | medium | 1 control(s) across 1 form(s) lack a label or accessible name. |
| WARN | CORE-TRUST-001 | Contact/support discoverability | low | No contact or support link found on the entry page. |
| WARN | CORE-TRUST-002 | Shipping policy discoverability | medium | No shipping policy page was found linked from the entry page. |
| WARN | CORE-TRUST-003 | Returns/refund policy discoverability | medium | No returns/refund policy page was found linked from the entry page. |
| WARN | CORE-TRUST-005 | Terms/conditions discoverability | medium | No terms/conditions page was found linked from the entry page. |
| WARN | CORE-TRUST-006 | security.txt discoverability | medium | security.txt was found but has no Contact field. |
| PASS | CORE-OPERABILITY-005 | Agent parse cost | info | The entry page's estimated parse cost is ~37 tokens (low). |
| N/A | CORE-ACCESS-006 | Sitemap freshness | info | No valid sitemap to evaluate freshness for. |
| N/A | CORE-INTERFACE-001 | MCP discovery presence | info | No MCP discovery file is published. |
| N/A | CORE-INTERFACE-002 | MCP server declaration quality | info | No MCP server is declared. |
| N/A | CORE-INTERFACE-003 | WebMCP static presence | info | No static WebMCP marker was found on the sampled pages. |
| N/A | CORE-INTERFACE-005 | OpenAPI discovery | info | Not applicable to the commerce profile. |
| N/A | CORE-INTERFACE-006 | OAuth authorization-server metadata | info | Not applicable to the commerce profile. |
| N/A | CORE-INTERFACE-007 | OAuth protected-resource metadata | info | Not applicable to the commerce profile. |
| N/A | CORE-MACHINE-007 | Breadcrumbs | info | Only the entry page was sampled; there are no additional pages to check for breadcrumbs. |
| N/A | CORE-TRUST-007 | Pricing discoverability | info | No pricing signal (a pricing page or a structured product price) was found. |
Experimental (not scored)
CORE-INTERFACE-008(WARN) — A UCP profile is published but fails validation.
N/A: CORE-INTERFACE-004, CORE-INTERFACE-009, CORE-MACHINE-012, CORE-OPERABILITY-007
Evidence
CORE-ACCESS-001 — HTTPS reachability
{
"final_url": "https://example.com/step4",
"input_url": "https://example.com/",
"redirect_chain": [
"https://example.com/",
"https://example.com/step1",
"https://example.com/step2",
"https://example.com/step3"
],
"redirect_count": 4,
"status": 200
}
CORE-ACCESS-002 — robots.txt availability and syntax
{
"error": null,
"resource": "https://example.com/robots.txt",
"served_as_html": false,
"sha256": "901c28336f0ee2ceb8b00545721bc3a75c09d539b54d031ab78f34d43f7e5294",
"sitemap_count": 0,
"status": 200,
"unknown_directives": []
}
CORE-ACCESS-003 — AI search crawler policy
{
"declared_policy": {
"Bingbot": false,
"Claude-SearchBot": false,
"Googlebot": false,
"OAI-SearchBot": false,
"PerplexityBot": false
},
"http_status": 200,
"resource": "https://example.com/robots.txt",
"robots_present": true
}
CORE-ACCESS-004 — Training vs. search crawler separation
{
"explicit_separation": false,
"http_status": 200,
"resource": "https://example.com/robots.txt",
"search_blocked": [
"OAI-SearchBot",
"Claude-SearchBot",
"PerplexityBot",
"Googlebot",
"Bingbot"
],
"training_blocked": [
"GPTBot",
"ClaudeBot",
"Google-Extended",
"CCBot",
"Applebot-Extended"
]
}
CORE-ACCESS-005 — Sitemap availability
{
"entry_count": 0,
"exists": false,
"kind": null,
"parse_error": null,
"probe_error": null,
"probe_status": 404,
"served_as_html": false,
"url": null,
"valid": false
}
CORE-ACCESS-006 — Sitemap freshness
{
"url": null
}
CORE-ACCESS-007 — Canonical URL integrity
{
"canonical_url": "https://example.org/elsewhere",
"entry_url": "https://example.com/step4"
}
CORE-ACCESS-008 — Indexability
{
"robots_meta": "noindex",
"x_robots_tag": null
}
CORE-ACCESS-009 — llms.txt presence and integrity
{
"errors": [],
"references_broken": [
"https://example.com/does-not-exist"
],
"references_checked": 1,
"references_unresolved": [],
"resource": "https://example.com/llms.txt",
"valid": true
}
CORE-ACCESS-010 — Content-Signal declaration
{
"declared": true,
"dimensions": {
"ai-input": "unset",
"ai-train": "unset",
"search": "unset"
},
"syntax_errors": [
"unrecognized value for 'search': 'maybe' (expected yes|no)"
]
}
CORE-INTERFACE-001 — MCP discovery presence
{
"exists": false,
"http_status": 404,
"resource": "https://example.com/.well-known/mcp.json",
"server_card": false,
"valid": false
}
CORE-INTERFACE-003 — WebMCP static presence
{
"pages_scanned": 1,
"pages_with_marker": []
}
CORE-MACHINE-001 — JSON-LD parseability
{
"pages": [
{
"parsed": 1,
"raw": 2,
"url": "https://example.com/step4"
}
],
"parsed_total": 1,
"raw_total": 2
}
CORE-MACHINE-002 — Organization entity
{
"found": false,
"pages_parsed": 1
}
CORE-MACHINE-003 — WebSite/WebPage entity
{
"found": false,
"pages_parsed": 1
}
CORE-MACHINE-004 — Product structured data
{
"has_identifiers": false,
"pages_parsed": 1,
"product_pages": [
"https://example.com/step4"
]
}
CORE-MACHINE-005 — Offer price, currency, and availability
{
"availability": null,
"currency": null,
"missing": [
"price",
"priceCurrency",
"availability"
],
"price": null,
"url": "https://example.com/step4"
}
CORE-MACHINE-006 — Product identifier count
{
"count": 0,
"identifier_keys": [],
"pages_parsed": 1
}
CORE-MACHINE-008 — Metadata quality
{
"entry_url": "https://example.com/step4",
"issues": [
"og:title",
"og:description"
],
"meta_description": "A synthetic fixture site with several broken agent-readiness signals.",
"missing": [
"og:title",
"og:description"
],
"title": "Example Bad Shop"
}
CORE-MACHINE-009 — Heading structure
{
"entry_url": "https://example.com/step4",
"h1_count": 2,
"heading_count": 2,
"issues": [
"multiple H1"
],
"levels": [
"h1",
"h1"
]
}
CORE-MACHINE-010 — Language declaration
{
"entry_url": "https://example.com/step4",
"html_lang": null
}
CORE-MACHINE-011 — Image alt coverage
{
"covered": 0,
"empty_alt": 0,
"pages_parsed": 1,
"ratio": 0.0,
"total": 1,
"with_alt": 0
}
CORE-OPERABILITY-001 — Server-rendered core content
{
"entry_url": "https://example.com/step4",
"spa_shell_marker": false,
"visible_text_chars": 148
}
CORE-OPERABILITY-002 — Redirect chain complexity
{
"redirect_chain": [
"https://example.com/",
"https://example.com/step1",
"https://example.com/step2",
"https://example.com/step3"
],
"redirect_count": 4
}
CORE-OPERABILITY-003 — Cache validators
{
"cache_control": null,
"etag": null,
"last_modified": null
}
CORE-OPERABILITY-004 — Broken machine-consumable endpoints
{
"broken": [
{
"source": "canonical",
"status": 404,
"url": "https://example.org/elsewhere"
},
{
"source": "llms",
"status": 404,
"url": "https://example.com/does-not-exist"
}
],
"broken_count": 2,
"inconclusive": [],
"refs_checked": 2,
"refs_resolved": 2,
"unresolved": []
}
CORE-OPERABILITY-005 — Agent parse cost
{
"dom_nodes": 16,
"estimated_tokens": 37,
"html_bytes": 725,
"level": "LOW",
"link_count": 0,
"script_bytes": 88,
"script_ratio": 0.12137931034482759,
"structured_bytes": 88,
"text_chars": 148,
"token_chars_ratio": 4
}
CORE-OPERABILITY-006 — Form/control labels
{
"totals": {
"forms": 1,
"inputs": 1,
"unlabeled_inputs": 1,
"unlabeled_selects": 0,
"unnamed_buttons": 0
}
}
CORE-TRUST-001 — Contact/support discoverability
{
"contact_url": null,
"kind": null
}
CORE-TRUST-006 — security.txt discoverability
{
"contact": false,
"expires": "2020-01-01T00:00:00Z",
"expires_valid": false,
"found_url": "https://example.com/.well-known/security.txt",
"status": 200
}
Remediation
CORE-ACCESS-001— Collapse the redirect chain to at most one hop.CORE-ACCESS-002— Scope Disallow rules to specific paths instead of blocking `/` for `*`.CORE-ACCESS-003— Allow search/retrieval crawlers (e.g. OAI-SearchBot, Claude-SearchBot, PerplexityBot) in robots.txt while keeping any training restrictions separate.CORE-ACCESS-004— Restrict only the training-purpose crawler tokens; leave search/retrieval crawlers allowed.CORE-ACCESS-005— Publish /sitemap.xml and declare it in robots.txt.CORE-ACCESS-007— Point the canonical tag at a URL on this site's own host.CORE-ACCESS-008— Remove the noindex directive from the meta robots tag or X-Robots-Tag header.CORE-ACCESS-009— Remove or fix the broken references in llms.txt.CORE-ACCESS-010— Fix the Content-Signal directive, e.g. "Content-Signal: search=yes, ai-input=yes, ai-train=no".CORE-MACHINE-001— Fix the malformed JSON-LD blocks so every one parses (validate with a JSON linter before publishing).CORE-MACHINE-002— Add JSON-LD with "@type": "Organization" declaring at least name and url.CORE-MACHINE-003— Add JSON-LD with "@type": "WebSite" or "WebPage".CORE-MACHINE-004— Add a stable identifier (sku, gtin, mpn, or brand) to the Product entity.CORE-MACHINE-005— Add an Offer with price, priceCurrency, and availability to the Product entity.CORE-MACHINE-006— Add a stable identifier (sku, gtin, mpn, or brand) to the Product entity.CORE-MACHINE-008— Add the missing metadata tags and give title and description distinct content.CORE-MACHINE-009— Use exactly one H1 per page and avoid skipping heading levels (e.g. h1 to h3).CORE-MACHINE-010— Add a valid `lang` attribute to the `<html>` tag (e.g. `lang="en"`).CORE-MACHINE-011— Add an alt attribute (or an explicit empty alt for decorative images) to every image.CORE-OPERABILITY-001— Server-render (or statically pre-render) the core content so an agent's plain HTTP fetch sees it.CORE-OPERABILITY-002— Collapse the redirect chain to at most one hop.CORE-OPERABILITY-003— Add an ETag or Last-Modified header so agents can issue conditional GETs.CORE-OPERABILITY-004— Fix or remove the broken references (sitemap, llms.txt, OpenAPI spec, policy links) so agents don't hit dead links.CORE-OPERABILITY-006— Add a <label> (or aria-label/aria-labelledby) to every input/select, and a name or accessible text to every button.CORE-TRUST-001— Add a clearly labeled contact or support link (a page or a mailto link) to the entry page.CORE-TRUST-002— Publish a shipping policy page (cost, timing, carriers) and link it from the entry page's nav or footer.CORE-TRUST-003— Publish a returns/refund policy page and link it from the entry page's nav or footer.CORE-TRUST-004— Publish a privacy policy page and link it from the entry page's nav or footer.CORE-TRUST-005— Publish a terms/conditions page and link it from the entry page's nav or footer.CORE-TRUST-006— Add a Contact field (RFC 9116) to security.txt.
Limitations
- Scovant Core evaluates declared and static evidence only; it does not observe real agent traffic.
- Only reachability and syntax are checked; per-agent policy is evaluated by the other CORE-ACCESS checks.
- Only the declared robots.txt policy is evaluated; whether the crawler is actually served is not observed.
- Only the declared robots.txt policy is evaluated.
- Only the first-declared sitemap (or its first child, for a sitemap index) is fetched and validated.
- Only ISO 8601 date-formatted lastmod values are parsed; malformed dates are ignored, not penalised.
- Only the entry page's declared canonical is evaluated.
- Only the entry page's own robots meta tag and X-Robots-Tag header are checked.
- Absence is not penalised; the convention is emerging. Reference checks are HTTP status only.
- Absence is not penalised; the convention is emerging.
- Only `/.well-known/mcp.json` and the two candidate server-card paths are probed; a custom discovery location is not found.
- Only the `mcpServers` object at /.well-known/mcp.json is parsed; a server card is not covered by this check.
- This is a static text scan of already-fetched HTML for a marker string; the browser-side registration was not executed, so a marker's presence does not confirm the tools actually register or work.
- Only a fixed set of conventional paths, plus same-origin entry-page links naming openapi/swagger, are probed.
- Only the conventional /.well-known/oauth-authorization-server path is probed.
- Only the conventional /.well-known/oauth-protected-resource path is probed.
- Only well-formed-JSON parseability is checked; schema.org vocabulary correctness is not validated.
- Only the schema_org nodes on the sampled pages are checked; an Organization declared elsewhere on the site is not evaluated.
- Only the schema_org nodes on the sampled pages are checked.
- Only the sampled pages are checked; a product catalog not represented in the sample is not evaluated.
- Only the first sampled page that exposes a Product entity is checked.
- Only the non-entry pages in the sampled set are checked.
- Only the entry page's metadata is checked.
- Only the entry page's heading outline is checked.
- Only the entry page's `<html lang>` attribute is checked.
- Only the images on the sampled pages are counted.
- Static-HTML signal only: the raw fetched response is inspected, never rendered in a browser, so a site that hydrates real content very quickly may still be flagged here.
- Only the entry URL's own redirect chain is inspected; redirects encountered while fetching other sampled pages are not counted here.
- Only the entry response's own headers are checked; per-page-type validator strategy is not inspected.
- Only the capped set of references collected by the machine_links gatherer are checked; endpoints not linked from any declared document are not found. A declared MCP endpoint is recorded but never judged — Core does not perform the MCP handshake.
- `estimated_tokens` is a character-count estimate (chars / token_chars_ratio), never a real tokenizer count, and only the entry page is measured.
- Only forms on the sampled page set are inspected; a form behind client-side rendering that never appears in the static HTML is not seen.
- Only the entry page's anchors are scanned; a contact method reachable only from a deeper page is not found.
- Only a same-origin link discovered from the entry page's nav/footer/anchors is followed; a shipping policy reachable only from a deeper page is not found.
- Only a same-origin link discovered from the entry page's nav/footer/anchors is followed; a returns policy reachable only from a deeper page is not found.
- Only a same-origin link discovered from the entry page's nav/footer/anchors is followed; a privacy policy reachable only from a deeper page is not found.
- Only a same-origin link discovered from the entry page's nav/footer/anchors is followed; terms reachable only from a deeper page are not found.
- Only the conventional /.well-known/security.txt and legacy /security.txt paths are probed.
- Pricing-page discovery follows only a same-origin link from the entry page; the structured-price fallback checks only the sampled pages.
Not tested by Scovant Core
- Observed WAF access
- Real agent tasks
- MCP tool execution
- WebMCP state parity
- Multi-model reliability
- Regression stability
Core vs Cloud
| Capability | Core | Cloud |
|---|---|---|
| HTTP reachability | ✅ | ✅ |
| robots.txt | ✅ | ✅ |
| Sitemap | ✅ | ✅ |
| llms.txt | ✅ | ✅ |
| Structured data (JSON-LD) | ✅ | ✅ |
| Product/Offer data | ✅ | ✅ |
| Static crawler policy | ✅ | ✅ |
| Content-Signal | ✅ | ✅ |
| MCP discovery | ✅ | ✅ |
| WebMCP static presence | ✅ | ✅ |
| OpenAPI presence | ✅ | ✅ |
| OAuth authorization-server / protected-resource metadata | ✅ | ✅ |
| UCP profile validity | ✅ (experimental) | ✅ |
| Agent discovery surface (A2A cards, AI-plugin, agents.json, Agent Skills) | ✅ (experimental) | ✅ |
| Core Score | ✅ | — |
| Cloud's full compatibility score | ❌ | ✅ |
| Cloud's full production ruleset | ❌ | ✅ |
| Observed WAF/bot-firewall behavior | ❌ | ✅ |
| Real crawler network access | ❌ | ✅ |
| Browser-based agent simulation | ❌ | ✅ |
| Multi-model execution | ❌ | ✅ |
| MCP tool invocation | ❌ | ✅ |
| WebMCP tool execution/state parity | ❌ | ✅ |
| Tool/UI parity checking | ❌ | ✅ |
| Checkout/task completion | ❌ | ✅ |
| CAPTCHA/challenge behavior | ❌ | ✅ |
| Verified-agent access | ❌ | ✅ |
| Failure attribution | ❌ | ✅ |
| Temporal stability / regressions | ❌ | ✅ |
| Scheduled monitoring | ❌ | ✅ |
| Alerts/webhooks | ❌ | ✅ |
| Hosted, shareable reports | ❌ | ✅ |
| Contextual fix plan | ❌ | ✅ |
Core measures what a site declares. Cloud measures what real agents actually experience.
Methodology
Category weights are 25/25/20/15/15. Each check contributes PASS = 1, WARN = 0.5, FAIL = 0 to its category; ERROR lowers coverage (a document that could not be read counts against evidence), and N/A is excluded entirely. A category scores INSUFFICIENT_EVIDENCE below a 60% coverage floor of its applicable weight. Full reference: docs/methodology.md.
Provenance
- scan_id
- local-golden
- started_at
- 2026-09-04T00:00:00Z
- completed_at
- 2026-09-04T00:00:00Z
- schema_version
- 1.0
- core_version
- 0.1.0
- ruleset_version
- 2026.09
- ruleset_digest
- d83df799eaa7
- python
- <runtime>
- platform
- <runtime>
- user_agent
- ScovantCore/0.1.0 (+https://github.com/Scovant/scovant-core)
- timeout
- 60.0
- max_pages
- 5
- network_mode
- fixture
- experimental
- False
- allow_private_networks
- False