Metadata-Version: 2.5
Name: doodle-lint
Version: 1.2.0
Summary: Measures whether your agent skill actually fires, plus static rules for SKILL.md.
Project-URL: Homepage, https://github.com/krishyaid-coder/doodle
Project-URL: Issues, https://github.com/krishyaid-coder/doodle/issues
Author: doodle contributors
License: MIT
License-File: LICENSE
Keywords: agent,anthropic,claude,linter,skills
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Requires-Dist: pyspellchecker>=0.7
Requires-Dist: pyyaml>=6.0
Requires-Dist: tomli>=2.0; python_version < '3.11'
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: eval
Requires-Dist: anthropic>=0.30.0; extra == 'eval'
Provides-Extra: semantic
Requires-Dist: fastembed>=0.3; extra == 'semantic'
Description-Content-Type: text/markdown

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="./docs/assets/logo-banner-dark.svg">
    <img src="./docs/assets/logo-banner-light.svg" alt="doodle" width="480"/>
  </picture>
</p>

<p align="center">
  Measures whether your agent skill actually fires. Plus 16 static rules for the obvious stuff.
</p>

<p align="center">
  <a href="./LICENSE"><img src="https://img.shields.io/badge/license-MIT-1a1a1a?style=flat-square" alt="MIT"/></a>
  <a href="./RULES.md"><img src="https://img.shields.io/badge/rules-16-1a1a1a?style=flat-square" alt="16 rules"/></a>
  <a href="https://open-vsx.org/extension/krishyaid-coder/doodle-lint"><img src="https://img.shields.io/badge/vscode-open%20vsx-1a1a1a?style=flat-square" alt="Open VSX"/></a>
  <a href="./docs/TRIGGER_SURFACE_REPORT.md"><img src="https://img.shields.io/badge/trigger%20surface-24%25%20mean-E5484D?style=flat-square" alt="Trigger Surface Report"/></a>
  <a href="./docs/QUALITY_REPORT.md"><img src="https://img.shields.io/badge/quality%20report-200%20skills-1a1a1a?style=flat-square" alt="Quality Report"/></a>
</p>

---

## Overview

`SKILL.md` is the [Agent Skills open standard](https://agentskills.io), the format Claude Code, VS Code, Codex, Cursor, Gemini CLI and 30-odd other tools read to extend an agent with custom capabilities.

Most tooling for it checks whether a skill file is *well-formed*. doodle does that too, but the more interesting question is whether the skill will ever be **selected**. Agents pick skills by matching the user's request against skill descriptions, so a description can be perfectly formed and still lose every match:

```
description: Performs heuristic evaluation of committed changesets.
user types:  "check my code"
overlap:     0%
```

Nothing there is wrong. It just never runs, and a conventional linter reports the file as clean.

doodle measures that gap two ways:

- **`doodle surface`**: offline. Compares your description against realistic user phrasings and names the words you're missing. No API key.
- **`doodle eval`**: empirical. Runs prompts through the model and reports how often the skill actually fires.

Across 77 published skills, descriptions shared a mean of 24% of their vocabulary with realistic requests, and description *length* explained under 8% of the variance. See the [Trigger Surface Report](./docs/TRIGGER_SURFACE_REPORT.md).

Plus 16 static rules, each citing the authoring guide or a documented community issue, for the conventional checks.

## Install

```bash
pip install doodle-lint
```

Requires Python 3.10 or newer. Runtime dependencies: `PyYAML`, `pyspellchecker`, and `tomli` on Python 3.10. PyPI publication is pending; once complete, `pip install doodle-lint` will work.

Verify:

```bash
doodle --version
doodle --list-rules
```

## Usage

```bash
# Scaffold a new skill that passes the linter out of the box
doodle init my-skill --eval          # creates ./my-skill/{SKILL.md,eval.yaml}

# Measure trigger surface: does the description use the words users type?
doodle surface path/to/SKILL.md

# Same analysis with embedding similarity (catches paraphrases lexical overlap misses)
# Requires: pip install "doodle-lint[semantic]"
doodle surface path/to/SKILL.md --semantic

# Lint a single skill
doodle path/to/SKILL.md

# Lint a directory recursively
doodle ./skills

# JSON output for CI
doodle ./skills --format=json

# SARIF output for GitHub code scanning
doodle ./skills --format=sarif > doodle.sarif

# Apply auto-fixes for safe rules
doodle ./skills --fix

# Promote info to warning, warning to error
doodle --strict ./skills

# Disable a specific rule
doodle --ignore=body/emoji ./skills

# Explain a rule
doodle --explain desc/vague-trigger
```

Exit codes: `0` clean, `1` warnings, `2` errors, `3` tool error.

### Phase 2: trigger-accuracy eval

```bash
# Requires: pip install ".[eval]", npm install -g promptfoo, ANTHROPIC_API_KEY
doodle eval --generate path/to/SKILL.md    # Draft a starter eval.yaml via Claude
doodle eval path/to/SKILL.md               # Run the eval, report the score
doodle eval path/to/SKILL.md --dry-run     # Preview the Promptfoo config
```

Full workflow: [docs/EVAL.md](./docs/EVAL.md).

## VS Code / Cursor extension

The extension is published on the [Open VSX Registry](https://open-vsx.org/extension/krishyaid-coder/doodle-lint) and works in Cursor, VSCodium, Windsurf, and VS Code with Open VSX enabled.

Features: real-time diagnostics, status bar with finding counts, hover messages linking to the rule catalog, quick-fix code actions for fixable rules, and a `doodle eval` bridge command.

Install from your editor's Extensions panel (search `doodle`), or via CLI:

```bash
# Cursor, VSCodium, Windsurf (Open VSX is the default gallery)
code --install-extension krishyaid-coder.doodle-lint

# Vanilla VS Code: install the VSIX from the latest GitHub Release
curl -L -o /tmp/doodle.vsix \
  https://github.com/krishyaid-coder/doodle/releases/latest/download/doodle-lint-0.2.0.vsix
code --install-extension /tmp/doodle.vsix
```

The extension shells out to the CLI installed above. Upgrading the CLI upgrades the extension's rules automatically.

Full extension docs: [vscode/README.md](./vscode/README.md).

## Pre-commit hook

If your project uses [pre-commit](https://pre-commit.com/), add doodle in four lines:

```yaml
# .pre-commit-config.yaml
repos:
  - repo: https://github.com/krishyaid-coder/doodle
    rev: v1.0.0
    hooks:
      - id: doodle
```

Then:

```bash
pre-commit install
git commit -m "test"    # doodle runs automatically on any changed SKILL.md files
```

Two hooks are provided:

- `doodle`: runs by default on every commit, blocks commits that produce warnings or errors
- `doodle-fix`: runs manually via `pre-commit run doodle-fix --all-files`, applies safe auto-fixes and stages the results

## CI (GitHub Action)

```yaml
- uses: krishyaid-coder/doodle@v0
  with:
    path: ./skills
    strict: true
    fail-on: warning   # warning | error | never
```

## Rules

Each rule cites either Anthropic's authoring documentation or a documented community issue. Full spec with examples, in-sample frequency, and citations: [RULES.md](./RULES.md).

## Starter templates

Category-specific templates for common skill types are shipped in [`eval-suites/`](./eval-suites/) and wired into `doodle init`:

```bash
doodle init --list-templates
doodle init my-reviewer --template=code-reviewer --eval
```

The scaffold pairs a tuned SKILL.md description (passes the linter out of the box) with a matching `eval.yaml` containing category-appropriate `should_fire` and `should_not_fire` prompts. Available templates:

| Template | For skills that... |
| --- | --- |
| `code-reviewer` | Review diffs, PRs, staged changes for correctness and security |
| `refactorer` | Restructure code without changing observable behavior |
| `sql-generator` | Write SQL from natural-language analytics questions |
| `docs-writer` | Produce READMEs, API references, and docstrings |
| `test-writer` | Author unit, integration, and end-to-end tests |
| `security-auditor` | Audit code for OWASP issues, injection risks, and CVEs |
| `debugger` | Investigate errors and find root causes |
| `data-engineer` | Design ETL pipelines, dbt models, Airflow DAGs |
| `api-designer` | Design REST endpoints, GraphQL schemas, service contracts |
| `skill-creator` | Author or improve Claude skills |

Full contributor guide for new templates: [eval-suites/README.md](./eval-suites/README.md).

## Quality badge

Skill authors can advertise the doodle grade of their `SKILL.md` in the skill's own README:

```bash
doodle badge path/to/SKILL.md
```

Prints a shields.io-backed markdown snippet, for example:

```markdown
[![doodle A](https://img.shields.io/badge/doodle-A-green?style=flat-square)](https://github.com/krishyaid-coder/doodle)
```

Grade rubric:

| Grade | Meaning                                                            |
| ----- | ------------------------------------------------------------------ |
| A+    | Zero findings                                                      |
| A     | Info-level findings only                                           |
| B     | 1 or 2 warnings, no errors                                         |
| C     | 3 or 4 warnings, no errors                                         |
| D     | 5 or more warnings, no errors                                      |
| F     | Any error (skill will misload or fail to trigger)                  |

The badge reflects the same rules `doodle lint` would apply today, respecting your `.doodle.toml` overrides. Other formats:

```bash
doodle badge SKILL.md --format=url    # just the shields.io URL
doodle badge SKILL.md --format=text   # just the grade letter
doodle badge SKILL.md --format=json   # structured, includes finding counts
doodle badge SKILL.md --link=https://github.com/you/your-skills   # override the badge target
```

| Rule                        | Severity                 | Fixable | Description                                            |
| --------------------------- | ------------------------ | ------- | ------------------------------------------------------ |
| `desc/too-long`             | warning                  | no      | Description longer than 250 characters                 |
| `desc/too-short`            | warning                  | no      | Description shorter than 60 characters or missing      |
| `desc/no-trigger-phrase`    | warning                  | no      | No explicit "Use when" or "Trigger with" phrasing      |
| `desc/vague-trigger`        | warning                  | no      | Trigger overlaps Claude's default behavior             |
| `desc/weak-trigger-surface` | info                     | no      | Description doesn't use the vocabulary users type      |
| `desc/typo`                 | info (off by default)    | no      | Description contains a likely misspelling              |
| `body/too-long`             | warning                  | no      | Body longer than 500 lines                             |
| `body/way-too-long`         | error                    | no      | Body longer than 1500 lines                            |
| `body/absolute-user-path`   | warning                  | no      | Contains `/Users/`, `/home/`, or `~/` outside fences   |
| `body/emoji`                | info (off by default)    | yes     | Emoji in body. Enable via `--strict` or config         |
| `fm/name-mismatch-dir`      | warning                  | no      | Frontmatter `name` doesn't match parent directory      |
| `fm/missing-allowed-tools`  | warning                  | no      | Extended dialect uses tools but omits scoping          |
| `fm/unknown-field`          | info                     | no      | Anthropic dialect has non-standard field               |
| `hygiene/desc-blank-lines`  | info                     | yes     | Description contains embedded blank lines              |
| `hygiene/trailing-whitespace` | info                   | yes     | Line has trailing whitespace                           |
| `hygiene/final-newline`     | info                     | yes     | File does not end with a newline                       |

## Architecture

One small Python package. Data flow: files → parser → rule registry → severity gate → formatter.

```mermaid
flowchart LR
    A[SKILL.md files] -->|read| B[Parser]
    B -->|ParsedSkill| C[Rule registry]
    Cfg[.doodle.toml] -->|custom rules<br/>+ overrides| C
    C -->|Finding stream| D[Severity gate]
    D -->|filtered| E{Formatter}
    E -->|text| F[stdout]
    E -->|json / sarif| G[CI / dashboard]
```

Components:

- `parser`: splits frontmatter from body and auto-detects the dialect.
- `rule registry`: runs built-in and custom rules, applies severity overrides, filters by dialect and per-path globs.
- `custom rules`: pattern and frontmatter-required rules loaded from `.doodle.toml`.
- `formatter`: renders findings as colored text, JSON, or SARIF.

Full component reference, sequence diagrams, and extension points: [docs/ARCHITECTURE.md](./docs/ARCHITECTURE.md).

## Dialects

Two `SKILL.md` dialects exist in the wild. doodle auto-detects.

- **anthropic**: minimal frontmatter (`name`, `description`, optional `license`). Used by [`anthropics/skills`](https://github.com/anthropics/skills).
- **extended**: community schema with `version`, `author`, `tags`, `allowed-tools`. Used by [`alirezarezvani/claude-skills`](https://github.com/alirezarezvani/claude-skills), [`jeremylongshore/claude-code-plugins-plus-skills`](https://github.com/jeremylongshore/claude-code-plugins-plus-skills), and [`DietrichGebert/ponytail`](https://github.com/DietrichGebert/ponytail).

Some rules apply to both. Others are dialect-scoped.

## Custom rules (teams and enterprise)

Drop a `.doodle.toml` in your project root. No Python required for regex-based rules or frontmatter requirements.

```toml
[options]
dialect = "extended"
fail-on = "warning"
# Opt-in: desc/weak-trigger-surface uses embeddings (needs doodle-lint[semantic]).
# Default is lexical: fast, offline, right for pre-commit/CI.
# semantic = true

[severity]
"body/emoji" = "off"
"body/too-long" = "info"

[[paths]]
glob = "**/experiments/**/SKILL.md"
disabled = ["desc/vague-trigger", "body/too-long"]

[[rules]]
id = "acme/no-customer-pii"
kind = "pattern"
pattern = "(?i)\\bcustomer_[a-z]+\\b"
applies-to = "body"
severity = "error"
message = "Customer PII tokens are not allowed in skills."
suggestion = "Use 'user_<role>' instead."

[[rules]]
id = "acme/require-team-tag"
kind = "frontmatter-required"
fields = ["team", "data-classification"]
severity = "error"
message = "Internal skills must declare team and data-classification."
```

Config is discovered by walking up the directory tree. Force a path with `--config`. For rules requiring Python logic, see [docs/EXTENDING.md](./docs/EXTENDING.md).

## Roadmap

| Version         | Feature                                                                                      | Status    |
| --------------- | -------------------------------------------------------------------------------------------- | --------- |
| v0.1            | Static linter, 12 rules, CLI, GitHub Action                                                  | shipped   |
| v0.2            | `.doodle.toml` config, custom rules, per-path overrides, severity overrides                  | shipped   |
| v0.3            | `--fix`, SARIF output, parse-error suggestions                                               | shipped   |
| v0.4            | `doodle eval` trigger-accuracy harness, `--generate` starter eval suites                     | shipped   |
| vscode 0.2      | VS Code extension on Open VSX Registry                                                       | shipped   |
| v0.5            | `doodle init` skill scaffold that passes the linter out of the box                           | shipped   |
| v0.6            | `desc/typo` spelling check backed by pyspellchecker, with a curated allowlist for AI vocabulary | shipped   |
| v0.7            | `doodle badge` quality badges (shields.io-backed, A+/A/B/C/D/F rubric)                       | shipped   |
| v0.8            | Two new hygiene rules with auto-fix (trailing-whitespace, final-newline)                     | shipped   |
| Docs (July)     | Quality Report refreshed to 200 skills across 6 repos                                        | shipped   |
| v0.9            | Community eval-suite library: 10 category templates (code-reviewer, refactorer, sql-generator, docs-writer, test-writer, security-auditor, debugger, data-engineer, api-designer, skill-creator) | shipped   |
| **v1.0**        | **Stable release. PyPI published. Pre-commit hook integration.**                             | shipped   |
| v1.1            | `doodle surface`: offline trigger-surface analysis, plus the Trigger Surface Report on 77 published skills | shipped |
| Phase 4         | Managed scanner and quality-badge dashboard for marketplace operators (paid service)         | exploring, waits for demand |
| Phase 5         | LSP extraction for Neovim, Zed, JetBrains via the same rule engine                           | if demand |

## Validation

I ran doodle against **200 published `SKILL.md` files** across six repositories: [DietrichGebert/ponytail](https://github.com/DietrichGebert/ponytail), [anthropics/skills](https://github.com/anthropics/skills), [obra/superpowers](https://github.com/obra/superpowers), [vercel-labs/skills](https://github.com/vercel-labs/skills), [alirezarezvani/claude-skills](https://github.com/alirezarezvani/claude-skills), and [jeremylongshore/claude-code-plugins-plus-skills](https://github.com/jeremylongshore/claude-code-plugins-plus-skills). 81 percent of the sample had at least one quality finding. The rate is statistically identical to the June edition's 62-skill sample (82 percent), confirming the signal is not sampling artifact.

Standout results: obra/superpowers is 86 percent clean; anthropic first-party is 6 percent clean (1 of 18); ponytail is 0 for 6.

Full methodology, per-repository breakdown, and raw data: [docs/QUALITY_REPORT.md](./docs/QUALITY_REPORT.md).

## Documentation

- [Trigger Surface Report](./docs/TRIGGER_SURFACE_REPORT.md): do published descriptions use the words users type? Methodology, raw data, and honest limitations
- [Quality Report](./docs/QUALITY_REPORT.md): static-rule findings across 200 published skills, with methodology and raw data
- [Architecture](./docs/ARCHITECTURE.md): diagrams, components, extension points, trade-offs
- [Rule spec](./RULES.md): every rule with citation, example, and in-sample frequency
- [Extending](./docs/EXTENDING.md): add a rule in Python or via `.doodle.toml`
- [Eval guide](./docs/EVAL.md): Phase 2 trigger-accuracy workflow
- [VS Code extension](./vscode/README.md): editor integration reference
- [Starter templates](./eval-suites/README.md): community eval-suite library, one entry per skill category
- [Why doodle](./docs/WHY.md): impact argument, honest risk assessment
- [Contributing](./CONTRIBUTING.md): ground rules and PR checklist

## License

[MIT](./LICENSE). The CLI and rule-set are MIT-licensed permanently. Any future hosted services will be a separate repository under a separate license.
