Metadata-Version: 2.4
Name: context-lint
Version: 0.1.0
Summary: ESLint for your LLM prompts — catch context bugs before they silently kill your output quality.
Author: context-lint contributors
License: MIT
Project-URL: Homepage, https://github.com/your-org/context-lint
Project-URL: Repository, https://github.com/your-org/context-lint
Project-URL: Issues, https://github.com/your-org/context-lint/issues
Keywords: llm,prompt,linter,prompt-engineering,anthropic,claude,ai
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: anthropic>=0.40
Provides-Extra: rich
Requires-Dist: rich>=13.0; extra == "rich"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: rich>=13.0; extra == "dev"

![demo](demo.gif)
# context-lint

**ESLint for your LLM prompts.** Catch the context bugs that silently wreck your output quality — buried instructions, token bloat, contradictions, ALL-CAPS shouting — and optionally auto-fix them with Claude.

```text
context-lint v0.1.0 — scanning: prompt.txt

  WARN   AGGRESSIVE_FORMATTING  ALL-CAPS commands found: "ALWAYS" (2 occurrences); excessive exclamation marks (1 run(s)); forceful phrases: "YOU MUST"
         → Use normal sentence case and plain phrasing. Models respond worse to aggressive formatting and shouting.

  WARN   CONTRADICTION          Conflicting instructions (length: concise vs thorough)
         → Keep the more specific, actionable instruction and remove the conflicting one.
         └ "concise" vs "thorough"

  WARN   ROLE_BLEED             User turn opens with a system-style instruction ("you are")
         → Move persistent role/behavior instructions into the system prompt; keep the user turn as the actual input.
         └ "You are an expert assistant. ALWAYS be concise in your answ…"

  INFO   REDUNDANT_COT          Explicit chain-of-thought instruction detected
         → Remove the step-by-step phrase. Modern reasoning models reason internally without being told to.
         └ "Think step by step"

────────────────────────────────────────────────────────
  4 issues found  (3 warnings, 1 info)  →  Score: 61/100

  Tip: Run with --fix to get a rewritten prompt with all issues resolved.
```

## The problem

Prompts fail in ways that never throw an error. The model just quietly does worse, and you have no idea why. The usual suspects are well-documented but easy to miss by eye:

- **Critical instructions buried in the middle.** LLM attention is U-shaped — content in the middle of a long context is the most likely to be ignored.
- **Token bloat.** Past a few thousand tokens, instruction-following measurably degrades.
- **ALL-CAPS and "!!!".** Shouting at the model tends to make outputs *more* erratic, not more obedient.
- **Contradictions.** "Be concise" and "be comprehensive" in the same prompt force the model to guess.
- **Role bleed.** Stuffing persistent role instructions into a user turn (or a question into the system prompt) blurs a boundary the model relies on.

`context-lint` is a static linter for all of this. No prompt actually runs — it reads your prompt, flags the issues with a fix and a reason for each, and gives you a 0–100 quality score. With `--fix`, it sends the prompt and the findings to Claude and streams back a rewritten version, then re-lints to prove the issues are gone.

## Install

```bash
pip install context-lint
```

For nicer colored output, install with the optional `rich` extra (the tool falls back to plain ANSI without it):

```bash
pip install "context-lint[rich]"
```

## Quickstart

```bash
# Lint a file
context-lint prompt.txt

# Lint from stdin
echo "You are a bot. Be concise but thorough." | context-lint -

# Lint several files at once
context-lint prompts/*.txt

# Auto-fix with Claude (requires an API key)
export ANTHROPIC_API_KEY=sk-...
context-lint prompt.txt --fix

# Machine-readable output for CI
context-lint prompt.txt --json
```

## The rules

Every finding carries a `rule_id`, a severity, a one-line fix, and a `why` (shown with `--verbose`, and always handed to the `--fix` rewriter).

| Rule | Severity | Catches |
| --- | --- | --- |
| `LOST_IN_MIDDLE` | WARN | Instruction-like sentences sitting in the middle 40% of the context, where models attend to them least reliably. |
| `TOKEN_BUDGET` | WARN / ERROR | Prompts over ~3,000 estimated tokens (WARN) or ~6,000 (ERROR), where reasoning quality drops. |
| `AGGRESSIVE_FORMATTING` | WARN | Repeated ALL-CAPS commands, runs of `!!!`, and shouty phrases like "YOU MUST". Technical acronyms (JSON, API, …) are allow-listed. |
| `CONTRADICTION` | WARN | Opposing directives — concise vs thorough, formal vs casual, or "always X" alongside "never X". |
| `ROLE_BLEED` | WARN | A user turn that opens with role-setting ("You are…", "Act as…"), or a system prompt that contains an actual user question. |
| `REDUNDANT_COT` | INFO | Explicit "think step by step" phrasing, which is redundant on models that already reason internally. |
| `VAGUE_OUTPUT_FORMAT` | INFO | A named output format (JSON, table, list, …) with no concrete example to anchor it. |
| `CONTEXT_ROT_RISK` | WARN | Long context (>1,500 tokens) with no summarize / filter / extract strategy to keep it manageable. |

Run a single rule with `--rule`, or raise the floor with `--min-severity`:

```bash
context-lint prompt.txt --rule TOKEN_BUDGET
context-lint prompt.txt --min-severity warn   # hide INFO findings
```

## `--fix`: rewrite with Claude

`--fix` runs the linter, shows the findings, then asks Claude to rewrite the prompt fixing every one — preserving your original intent — and streams the result back. It finishes by re-linting the rewrite so you can see the issues are actually resolved.

**Before** — `Score: 61/100`

```text
You are an expert assistant. ALWAYS be concise in your answers, but ALWAYS be
thorough and detailed too. YOU MUST get this right!!! Think step by step before
responding to the user's question about their account.
```

**After** `context-lint prompt.txt --fix` — `Score: 100/100`

```text
Answer the user's question about their account. Keep the response clear and concise.

CHANGES MADE:
• [ROLE_BLEED] — removed the "You are…" role-setting from the user turn; this is a request, not a system prompt.
• [AGGRESSIVE_FORMATTING] — converted ALL-CAPS to sentence case and dropped the "!!!".
• [CONTRADICTION] — kept "concise" and removed the conflicting "thorough and detailed".
• [REDUNDANT_COT] — removed "Think step by step"; the model reasons internally.
```

Write only the rewritten prompt to a file (the report still prints to the terminal):

```bash
context-lint prompt.txt --fix --output prompt.fixed.txt
```

`--fix` needs `ANTHROPIC_API_KEY` in the environment. If it's missing, `context-lint` prints a clear message and exits `3` without ever prompting you for it. The rewrite uses Claude Sonnet.

## Input formats

Formats are auto-detected — no flags needed. Content sniffing wins over the file extension.

- **Plain text** (`.txt`, `.md`, `.prompt`) — treated as a single user message.
- **JSON** (`.json`) — either a bare messages array `[{"role": ..., "content": ...}]` or an object `{"system": "...", "messages": [...]}`. Block-style content (`[{"type": "text", "text": "…"}]`) is flattened automatically.
- **XML-style** — `<system>…</system><user>…</user>` tags (also `<assistant>` / `<developer>`).

## CI integration

### GitHub Actions

```yaml
name: prompt-lint
on: [push, pull_request]

jobs:
  context-lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install context-lint
      - run: context-lint prompts/*.txt
```

The job fails on any warning (exit `1`) or error (exit `2`). To gate only on errors, add `--min-severity error`.

### pre-commit

Add to your `.pre-commit-config.yaml`:

```yaml
repos:
  - repo: https://github.com/your-org/context-lint
    rev: v0.1.0
    hooks:
      - id: context-lint
```

The hook lints every staged `.txt`, `.md`, `.json`, and `.prompt` file and blocks the commit if any fails.

## Exit codes

| Code | Meaning |
| --- | --- |
| `0` | Clean, or info-only findings |
| `1` | One or more warnings |
| `2` | One or more errors (or a usage / parse error) |
| `3` | `--fix` requested but `ANTHROPIC_API_KEY` is not set |

## JSON output

`--json` emits a single object for one file, or a `results` array for several. Each issue includes its `rule_id`, `severity`, `message`, `fix`, `why`, and `snippet`:

```json
{
  "version": "0.1.0",
  "source": "prompt.txt",
  "score": 61,
  "issues": [
    {
      "rule_id": "ROLE_BLEED",
      "severity": "WARN",
      "message": "User turn opens with a system-style instruction (\"you are\")",
      "fix": "Move persistent role/behavior instructions into the system prompt; keep the user turn as the actual input.",
      "why": "Mixing system-level instructions into user turns confuses model behavior...",
      "snippet": "You are an expert assistant. ALWAYS be concise..."
    }
  ],
  "summary": { "errors": 0, "warnings": 3, "info": 1, "total": 4 },
  "exit_code": 1
}
```

## How the score works

The score starts at 100 and deducts a penalty per finding: **ERROR −15, WARN −12, INFO −3**, floored at 0. A clean prompt scores 100. So three warnings plus one info is `100 − (12 + 12 + 12 + 3) = 61`. Filters like `--min-severity` change the score, because the score always reflects exactly what's shown.

## Extending it

Rules are self-registering. To add one: create a module under `context_lint/rules/`, subclass `BaseRule`, implement `check(prompt) -> list[LintIssue]`, and append the class to the `RULES` list in `context_lint/rules/__init__.py`. The runner has zero per-rule hardcoding, so that's the whole change.

## Roadmap

- More rules: few-shot imbalance, delimiter consistency, injected-instruction detection.
- Token counting via real tokenizers (currently a fast `words × 1.33` estimate).
- Config file for per-project thresholds and rule toggles.
- Editor integrations (LSP) for inline linting as you write prompts.

## Development

```bash
git clone https://github.com/your-org/context-lint
cd context-lint
pip install -e ".[dev]"
pytest
```

## License

MIT.
