Metadata-Version: 2.4
Name: wcp-conformance
Version: 0.1.0
Summary: Conformance test suite for the Workstation Capability Protocol (WCP)
Author: The Workstation Capability Protocol contributors
License: Apache-2.0
Project-URL: Homepage, https://github.com/kchemorion/WorkstationCapabilityProtocol
Project-URL: Repository, https://github.com/kchemorion/WorkstationCapabilityProtocol
Project-URL: Documentation, https://github.com/kchemorion/WorkstationCapabilityProtocol/blob/main/SPEC.md
Project-URL: Issues, https://github.com/kchemorion/WorkstationCapabilityProtocol/issues
Keywords: mcp,model-context-protocol,conformance,wcp,workstation,standard
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp>=1.2
Requires-Dist: httpx>=0.27
Requires-Dist: jsonschema>=4.21
Requires-Dist: pyjwt>=2.8
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Dynamic: license-file

# wcp-conformance

A standalone conformance test suite for the Workstation Capability
Protocol (WCP). It certifies **any** WCP endpoint — the `wcpd` reference
server or a third-party implementation — against the published contract:
the manifest schema (`schemas/manifest.schema.json`), the per-operation
schemas (`schemas/ops/<op.id>.json`), and the MCP binding described in the
WCP spec. This package depends only on that published standard; it never
imports or relies on `wcpd` internals.

The suite checks two things:

- **Core conformance** — the mandatory baseline every WCP server must
  meet: a valid manifest and the required `wcp.system` operations
  (e.g. `describe`).
- **Per-family conformance** — for each capability family the server
  advertises (`wcp.fs`, `wcp.exec`, `wcp.window`, ...), whether its
  operations round-trip against their schemas and enforce scopes,
  sandboxing, and error codes correctly. A server that only implements a
  subset of families (e.g. a headless container with just `fs` + `exec`)
  can still be certified for exactly what it implements.

Results are structured as a list of `CheckResult`s rolled up into a
`ConformanceReport` (see `src/wcp_conformance/results.py`), which can be
rendered as human-readable text (`report.summary()`) or serialized to
JSON (`report.as_dict()`).

## Install

This package has its own virtualenv, independent of `reference/wcpd`.

```bash
cd conformance
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"
```

## Test

```bash
cd conformance
.venv/bin/python -m pytest -q
```

## Usage

```bash
wcp-conformance --help
```

The CLI needs a bearer token to call the target server. Exactly one of
two token modes must be supplied:

1. **Pre-minted tokens** (works against any WCP server, including
   third-party implementations). The operator mints tokens out of band
   and passes them in:

   ```bash
   wcp-conformance \
     --endpoint https://workstation.example.com/mcp \
     --token-admin "$ADMIN_TOKEN" \
     --token-none "$ZERO_SCOPE_TOKEN" \
     --token-star "$STAR_SCOPE_TOKEN" \
     --json report.json
   ```

   - `--token-admin`: a full-scope token, used to exercise normal
     operation of every advertised family.
   - `--token-none`: a zero-scope token, used to verify the server
     correctly denies unauthorized calls.
   - `--token-star`: a token scoped to the wildcard `*` (but not the
     restricted `wcp:input:synthesize` scope), used to verify wildcard
     scopes don't implicitly grant input synthesis.

2. **Local HS256 minting** (convenient against reference/HS256-based
   servers such as `wcpd`). The suite mints its own tokens with
   `pyjwt`, given the server's shared secret:

   ```bash
   wcp-conformance \
     --endpoint http://localhost:8977/mcp \
     --jwt-secret "$WCP_JWT_SECRET" \
     --workstation-id dev-ws \
     --json report.json
   ```

By default the suite only ever invokes operations the target's manifest
marks `read_only`; it never performs a mutating action expected to
succeed. `--include-mutating` and `--sandbox-path PATH` (default `.`)
are available for exercising more of the surface — see
[`docs/certifying.md`](docs/certifying.md) for details, including a
worked example with `--include-mutating --sandbox-path /tmp/wcp-probe`.

The CLI prints `report.summary()` to stdout (unless `--quiet`),
optionally writes a machine-readable report to the path given by
`--json`, and exits `0` if `report.ok` is true, `1` if a required check
failed, or `2` on a usage error (e.g. no token source supplied).

For the full certifying guide — both token modes, what "read-only by
default" means, and how to read `core_conformant`/`family_verdicts` in
the report — see [`docs/certifying.md`](docs/certifying.md).
