Metadata-Version: 2.4
Name: crewscore
Version: 0.6.0
Summary: Find the guardrails your AI agent prompt forgot: which of eight failure modes - prompt injection, hallucination, runaway cost, missing human approval and more - the text never guards against. Offline, deterministic, explainable per-rule findings, with a CI gate.
Project-URL: Homepage, https://crewscore.ai
Project-URL: Repository, https://github.com/shmindmaster/crewscore
Project-URL: Issues, https://github.com/shmindmaster/crewscore/issues
Project-URL: Documentation, https://github.com/shmindmaster/crewscore#readme
Author-email: Sarosh Hussain <sarosh@pendoah.ai>
License-Expression: MIT
License-File: LICENSE
Keywords: ai-agents,ci-cd,crewscore,governance,guardrails,linter,llm,prompt-engineering,safety,static-analysis
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Requires-Dist: click>=8.0
Requires-Dist: rich>=13.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: pyyaml>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

<div align="center">

# CrewScore

### Find the safety rules your AI agent prompt forgot.

CrewScore reads your system prompt and shows you which of **23 safety controls
it never states** — injection defense, human approval, cost limits, stop
conditions.

**We scanned 356 real agent prompts: 83 production prompts and 273
general-purpose prompts. Among the production subset, median coverage was 14
of 100.**
[See the numbers →](docs/validation-corpus.md)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://python.org)
[![PyPI](https://img.shields.io/badge/PyPI-crewscore-blue.svg)](https://pypi.org/project/crewscore/)
[![GitHub Action](https://img.shields.io/badge/Action-shmindmaster%2Fcrewscore%40v2-blue.svg)](https://github.com/marketplace/actions/crewscore)

<br/>

> Example result: **8 of 23 written guardrails found** · 15 may be missing.
> CrewScore checks whether controls are written down, not whether an agent obeys them.

**Try it live, no install:** [crewscore.ai](https://crewscore.ai)

</div>

```bash
pip install crewscore
crewscore scan .
```

Deterministic regex over prompt text. **Offline, no API key, no LLM.**

---

## Read this first

**CrewScore is a checklist, not a benchmark.** The number is the share of **23
published controls** your prompt states — nothing about whether they are well
specified, mutually consistent, or obeyed at runtime.

| What the prompt does | What it scores |
| --- | --- |
| Nothing written down | **0** |
| One control in each of the 8 dimensions | **36** |
| All 23 controls | **100** |
| One control restated five different ways | same as stating it once |

So a **low score is actionable** — you probably have not written down an
injection policy, a human gate, or a safe-stop rule, and those are worth
writing. A **high score means the text is present**, not that the agent obeys
it. Don't rank prompts, teams, or vendors by this number, and don't treat a
threshold as a safety bar. Prefer the findings to the total.

Three dimensions — **Cost**, **Compliance**, **Audit** — ship with known-poor
construct validity and say so. Compliance is keyword detection; naming a
regulation is not complying with it.

📄 **[The validation study →](docs/validation.md)** — including the arithmetic
showing our own scale was broken through `0.1.0`, which we published before
fixing.

📊 **[Measured against 356 real prompts →](docs/validation-corpus.md)** —
Cliff's δ = 0.672 separating production agent prompts from general-purpose
ones, generated by a committed harness rather than typed by hand.

---

## Usage

```bash
crewscore scan .                          # find and score every agent prompt
crewscore init .                          # create a prompt-free regression baseline + PR workflow
crewscore scan . --fail-on-regression --baseline .crewscore-baseline.json
crewscore scan . --require human_gate.approval_required
crewscore test --prompt-file ./prompt.md  # score one file
crewscore fix  --prompt-file ./prompt.md --plan   # what's missing, no writes
crewscore rules --concepts                # the 23 controls, and the rules behind them
```

**[Full CLI reference →](docs/cli.md)** · **[How scoring works →](docs/scoring.md)**

---

## CI

```yaml
- uses: shmindmaster/crewscore@v2
  with:
    scan-path: "."
    # Report-only by default. Protect controls explicitly instead of treating
    # the coverage average as a safety bar:
    required-controls: "human_gate.approval_required,safe_stop.stop_condition"
    sarif: "crewscore.sarif"
```

Posts a sticky PR comment with the open rule findings. Guard downstream steps
on the `scored` output, not on `score` — an empty score casts to `0`.

**[Action inputs, outputs, and the CLI variant →](docs/ci.md)**

---

## Two artifacts, two rulesets

CrewScore judges two kinds of file, and tells you which it thinks it is looking
at. Detection is by filename and path — never by sniffing content.

| Artifact | Examples | Judged on |
|----------|----------|-----------|
| **Coding-agent config** | `AGENTS.md`, `CLAUDE.md`, `.cursorrules` | Configuration smells |
| **Agent system prompt** | `system-prompt.md`, anything under `prompts/` or `agents/` | The 8 governance dimensions |

A file saying *"always use pnpm"* is telling a coding agent how to work in your
repo. It has no reason to contain HIPAA language, and scoring it against that
is a category error.

We know the size of that error because we measured it: against the 100
most-starred repos with an `AGENTS.md`
([arXiv:2606.15828](https://arxiv.org/abs/2606.15828)), the governance ruleset
put **all 100 in the worst tier**. A scale the entire population fails carries
no information. So config files get a smell verdict instead — and in `--json`,
no governance grade at all.

```bash
crewscore test --prompt-file AGENTS.md
# -> CONFIG: NO SMELLS DETECTED   (not "0/100 CRITICAL GAPS")
```

---

## Configuration smells

Problems in the *shape* of an instruction file rather than its content, from a
published catalog —
[*Configuration Smells in AGENTS.md Files*](https://arxiv.org/abs/2606.15828)
(dos Santos et al., 2026), which found **91 of 100** popular projects carried
at least one.

| Smell | Heuristic | Found in |
|-------|-----------|----------|
| **Context Bloat** | ≥ 200 lines | 42% of studied projects |
| **Lint Leakage** | Style rules a configured linter already enforces | 62% |
| **Init Fossilization** | Tracked by git with exactly one commit | 24% |

The paper's other three smells need an LLM to detect. We would rather ship
three honest detectors than six approximate ones. Lint Leakage is an
approximation of the paper's detector and says so in its output; Init
Fossilization cannot tell "never needed revising" from "never got revised."

**Smells never change the score.** Folding them in would silently change what
every existing `--threshold` means.

---

## Development

```bash
git clone https://github.com/shmindmaster/crewscore.git
cd crewscore
pip install -e ".[dev]"
pytest
```

[AGENTS.md](AGENTS.md) has contributor operating notes.
[CONTRIBUTING.md](CONTRIBUTING.md) covers adding a rule — note that adding a
synonym for a control that is already covered changes no score, by design.

---

## Docs

| | |
|---|---|
| [Validation](docs/validation.md) | What the number does and does not measure |
| [Corpus validation](docs/validation-corpus.md) | Generated result over 356 real prompts |
| [Scoring](docs/scoring.md) | Formula, dimensions, provenance, charter |
| [CLI](docs/cli.md) | Every command and flag |
| [CI](docs/ci.md) | Action and CLI integration |
| [Policies and SARIF](docs/policies.md) | Regression and required-control CI without score gating |
| [Architecture](docs/architecture.md) | One catalog powering CLI, Action, and browser |
| [Scoring governance](docs/scoring-governance.md) | How public rules and validation change |
| [Roadmap](docs/roadmap.md) | Available work and deliberately deferred capabilities |
| [Security](SECURITY.md) | Private vulnerability reporting |
| [Comparison](docs/comparison.md) | Other tools, and what to use after this one |
| [CHANGELOG](CHANGELOG.md) | Including every scoring change and its measured delta |

---

## What this is not

Live adversarial red-teaming · runtime tool-gate enforcement · a security or
compliance certification · proof the model will obey the text.

**Roadmap:** framework adapters that extract prompts from LangGraph / CrewAI /
AutoGen graphs; optional live adversarial testing (post-traction, not the
default path).

MIT licensed.
