Metadata-Version: 2.4
Name: skeptic-cli
Version: 0.5.0
Summary: Independent engineering quality gate for AI-generated and human-written Python code.
Author-email: Hamza Shaikh <hamzashaikhm123@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Hamza Shaikh
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/HamzaShaikh17/Skeptic
Project-URL: Repository, https://github.com/HamzaShaikh17/Skeptic
Project-URL: Issues, https://github.com/HamzaShaikh17/Skeptic/issues
Project-URL: Documentation, https://github.com/HamzaShaikh17/Skeptic/blob/master/docs/USAGE.md
Keywords: code-quality,static-analysis,security,ci-cd,ai-agents,mcp,linting,skeptic
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: pydantic>=2.6
Requires-Dist: rich>=13.7
Requires-Dist: ruff>=0.4
Requires-Dist: bandit>=1.7
Requires-Dist: pyright>=1.1
Requires-Dist: pytest>=8.0
Requires-Dist: pytest-cov>=5.0
Requires-Dist: pytest-json-report>=1.5
Requires-Dist: pip-audit>=2.7
Requires-Dist: pyyaml>=6.0
Requires-Dist: mcp>=1.2
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: anyio>=4.0; extra == "dev"
Provides-Extra: narration
Requires-Dist: google-genai>=1.0; extra == "narration"
Requires-Dist: python-dotenv>=1.0; extra == "narration"
Provides-Extra: verify
Requires-Dist: google-genai>=1.0; extra == "verify"
Requires-Dist: python-dotenv>=1.0; extra == "verify"
Requires-Dist: httpx>=0.27; extra == "verify"
Dynamic: license-file

# Skeptic

An independent engineering quality gate for AI-generated (and human-written) Python
code. Skeptic doesn't take a change's word for it — it orchestrates Ruff, Pyright,
Bandit, pytest/coverage, and pip-audit into one pass/fail verdict, with evidence
attached to every failure, so an AI coding agent (or a human) can be required to
satisfy it before a change counts as done.

## Status

Phase 1 CLI MVP (`v0.3`), Phase 2's MCP server, and Phase 3 complete through
milestone 8 (LLM verifier/pricing/billing, milestone 9, not built). All five
deterministic adapters (Ruff, Pyright, Bandit, pytest, pip-audit) plus
structural SOLID checks, complexity, Change Risk Score, and AI provenance
tagging are wired and unit-tested against fixture repos; LLM-backed
narration and verification are live-tested against the real Gemini API. See
`Plan.md` for the full build plan and current milestone.

## Install

```bash
pip install -e .
```

This installs `skeptic` as a console command, backed by Click. Verified against
a clean `python -m venv` + `pip install -e .` with no other setup. Not yet on
PyPI — install from a local clone until it is.

## Prerequisites

**Install and activate the target repo's own dependencies before running `skeptic check`.**
`skeptic` shells out to `pyright` (and `pytest`) using whatever Python environment
is currently active — it does not install the target repo's dependencies for you.
If they aren't installed, Pyright can't resolve most imports and will report a
flood of `reportMissingImports` errors that have nothing to do with real type
safety, and `pytest` will fail to collect tests at all. Since the default gate
is zero-tolerance (`types_max_errors: 0`), this alone is enough to fail every
real repo. The fix: `cd` into the target repo, activate its venv (or otherwise
make sure its dependencies are installed in the active environment — e.g.
`pip install -r requirements.txt`, `uv sync`, `poetry install`), *then* run
`skeptic check`. Ruff, Bandit, and pip-audit don't need this — they analyze
source/dependency manifests directly rather than resolving imports.

## Usage

```bash
# Generate a starter config in your repo
skeptic init /path/to/your/repo

# Run the gate
skeptic check /path/to/your/repo

# Machine-readable output (for CI or an agent to parse)
skeptic check /path/to/your/repo --json
```

### Example

Given a repo with an unused import, a real type error, and a vulnerable pinned
dependency:

```
$ skeptic check .

Engineering Gate: FAIL

+----------------------------------------------------------------------------+
| Rule              | Status | Detail                                        |
|-------------------+--------+-----------------------------------------------|
| lint              | FAIL   | 1 lint findings (max allowed: 0)              |
| types             | FAIL   | 1 type errors (max allowed: 0)                |
| security_critical | PASS   | 0 critical security findings (max allowed: 0) |
| security_high     | PASS   | 0 high security findings (max allowed: 0)     |
| coverage          | PASS   | 88.9% coverage (min required: 0.0%)           |
| dependencies      | FAIL   | 12 critical/high CVEs (max allowed: 0)        |
+----------------------------------------------------------------------------+

lint findings:
  app.py:1  `os` imported but unused (F401)

types findings:
  app.py:14  Argument of type "Literal['not a number']" cannot be assigned to
  parameter "a" of type "int" in function "add" (reportArgumentType)
...
```

Exit code `1`. Fix the issues (or explicitly relax `skeptic.yaml`) and the same
command exits `0` with `Engineering Gate: PASS`.

## Configuration

Edit `skeptic.yaml` in your repo root:

```yaml
lint:
  max_errors: 0
types:
  max_errors: 0
security:
  max_critical: 0
  max_high: 0
coverage:
  min_percent: 80
dependencies:
  max_critical_cves: 0

# Optional, unset by default - see "Architecture findings" below. Neither
# blocks the gate until you uncomment it.
# architecture:
#   max_findings: 0
# ai_review:
#   max_risk_label: MEDIUM   # LOW | MEDIUM | HIGH
```

No `skeptic.yaml`? Defaults are strict (zero tolerance on everything) for
lint/types/security/dependencies — `architecture` and `ai_review` are the
two exceptions: they stay off until you explicitly configure them (see
below for why).

For the full walkthrough — reading output, `--json`/CI integration, what
each check actually does, troubleshooting — see [`docs/USAGE.md`](docs/USAGE.md).

## Architecture findings (SOLID + complexity) + narration

`skeptic check` also runs four deterministic, LLM-free structural checks —
SRP, ISP, DIP (`tool="solid"`), and McCabe cyclomatic complexity
(`tool="complexity"`, functions over 10 flagged by default). They show up
in the table output and `--json` (`solid_findings`/`complexity_findings`)
either way, but whether they can **fail** the gate depends on `skeptic.yaml`:

```
architecture findings (informational - not yet gated):
  app/god_service.py:11  class 'GodService' touches 3 unrelated external
  systems (database, email, http) via: ... (SRP)
```

**Not gated by default, deliberately** — unlike lint/types/security/
dependencies, which default to zero-tolerance. These checks are new and
haven't been broadly triaged the way an established linter has, so turning
every existing repo's gate from PASS to FAIL the moment you upgrade would
be a surprising, unrequested breaking change. Opt in explicitly:

```yaml
architecture:
  max_findings: 0   # gates both solid and complexity findings together
```

### Change Risk Score

Every `skeptic check` run (CLI table, `--json`, and MCP output) includes a
composite LOW/MEDIUM/HIGH label — security + regression (test health) +
architecture + complexity + test confidence, each scored 0-100 and always
shown, not just the label:

```
Change Risk Score: MEDIUM (38.2/100 - security=40.0, regression=0.0, architecture=45.0, complexity=10.0, test_confidence=100.0)
```

The weights and LOW/MEDIUM/HIGH thresholds are a documented first-pass
heuristic (see `src/skeptic/core/risk_score.py`), not an empirically
calibrated model — treat the label as a prioritization signal, not a
certified verdict. Gate on it explicitly if you want it to block:

```yaml
ai_review:
  max_risk_label: MEDIUM   # fails the gate if the label exceeds this
```

### AI provenance (`skeptic provenance`)

```bash
skeptic provenance /path/to/your/repo --since-ref HEAD~20
```

Estimates what share of recent commits landed while an AI agent was
actively using `skeptic-mcp` against this repo, correlating each commit's
timestamp against the local [MCP call log](#call-logging). **This is an
approximation, not a precise record**: `skeptic-mcp`'s tools are read-only
analysis, so the call log records when an agent called them, not which
lines it edited — a commit landing within `--window-minutes` (default 15)
of a logged call is labeled `ai_generated` (pure addition) or
`ai_modified` (touched existing lines); everything else is `human`. No
model identification, just the ratio. Requires no setup — with zero MCP
history for a repo, everything is reported as `human`.

### Plain-language narration (optional, costs an API call)

```bash
skeptic check /path/to/your/repo --narrate
```

Sends each SOLID finding to Gemini for a short "why this matters + how to
fix it" explanation, printed under the finding. The LLM never originates a
finding or changes the verdict — it only narrates one a deterministic check
already produced, and if narration fails (no key, network error, rate
limit) `skeptic check` still runs and reports normally, just without the
narration text.

Requires `GEMINI_API_KEY`:

```bash
pip install -e ".[narration]"   # installs google-genai + python-dotenv
cp .env.example .env            # then fill in GEMINI_API_KEY
```

`.env` is loaded automatically (and is gitignored — never commit it). The
model is `gemini-3.5-flash` by default, overridable via `SKEPTIC_GEMINI_MODEL`
if it gets deprecated later — Gemini model availability shifted twice while
building this feature (see `src/skeptic/narration/gemini_narrator.py`), so
this is a real, not hypothetical, concern.

## Adversarial verifier (`skeptic verify`)

Generates and runs attack test cases against a **running instance you
control** — boundary values, invalid input, injection, auth bypass, IDOR,
concurrency (race conditions), and failure-mode (timeout) probes — and
reports pass/fail per category with the exact request that triggered each
result.

```bash
pip install -e ".[verify]"   # installs google-genai + python-dotenv + httpx
# start your own app locally first, e.g.: uvicorn app.main:app --port 8000

skeptic verify /path/to/your/repo --target http://localhost:8000
```

**Safety, by design, not as an afterthought:**
- **Read-only against your code.** It never writes to the repo path — only
  generates requests and sends them to `--target`.
- **The LLM never executes anything.** Gemini returns structured data
  (method/path/headers/body) via a JSON schema, never code — the only thing
  that ever runs is an HTTP request Skeptic's own code sends. Real
  arbitrary-code-execution risk was a deliberate design decision *not* to
  take on for this milestone.
- **Refuses non-local targets by default.** `--target` must resolve to
  localhost or a private address (`10.x`, `172.16–31.x`, `192.168.x`, link-
  local) or the command exits immediately, before generating anything —
  it's sending real injection/auth-bypass/IDOR payloads, so this shouldn't
  be pointable at a service you don't own by accident. Pass
  `--allow-external` if you're certain the target is yours.

`--diff-ref` (default `HEAD`) focuses attack-case generation on your
uncommitted changes if `path` is a git repo; falls back to general-purpose
REST-API cases otherwise (not a git repo, or the ref doesn't exist) — never
a hard failure. `passed: null` on a result means the heuristic genuinely
can't tell (e.g. every concurrent request to a mutating endpoint succeeding
identically — could be a race condition, could be a correctly-idempotent
endpoint) and a human should look at `detail`; it's never silently coerced
to a pass.

Uses the same `GEMINI_API_KEY`/`.env` as `--narrate` above.

## MCP server (Claude Code / Cursor)

Skeptic's engine is also exposed as an MCP server, so an AI coding
agent can call it mid-task instead of you running `skeptic check` by hand.
Same engine, same adapters, same gate — the CLI and the MCP server are both
thin clients of `skeptic.core`.

### Install

`pip install -e .` (see above) also installs the `skeptic-mcp` console
command, which starts the server over stdio.

### Configure your project

Add this to your project's `.mcp.json` (Claude Code) or `.cursor/mcp.json`
(Cursor) — not Skeptic's own repo, *the repo you want the agent to check*:

```json
{
  "mcpServers": {
    "skeptic": {
      "command": "skeptic-mcp",
      "args": []
    }
  }
}
```

`skeptic-mcp` must resolve on `PATH` in whatever environment your editor
launches subprocesses from — same as any other locally-installed MCP server.
Skeptic's own repo ships this file too (dogfooding: Claude Code sessions
working on Skeptic itself get the tools automatically).

### Tools exposed

| Tool | Signature | Returns |
| --- | --- | --- |
| `skeptic_check` | `(repo_path: str)` | Full pass/fail verdict + evidence for every failing rule + tool statuses. Equivalent to `skeptic check --json`. |
| `skeptic_get_findings` | `(repo_path: str, severity: str \| None)` | Every raw finding across all 5 tools, optionally filtered to one severity (`critical`/`high`/`medium`/`low`) — not limited to findings tied to a failing gate rule. |
| `skeptic_gate_status` | `(repo_path: str)` | Same gate evaluation as `skeptic_check`, without the findings payload — a cheap pass/fail poll. |

The same [Prerequisites](#prerequisites) caveat applies: the target repo's
own dependencies need to be installed/active in the environment the MCP
server runs in, or `types` findings will mostly be import-resolution noise.

### Call logging

Every call to any of the three tools is appended to a local, per-repo,
append-only JSONL log at `~/.skeptic/mcp_logs/<repo-name>-<hash>.jsonl` —
timestamp, session id, tool name, args, and the full result. Nothing reads
this back today; it's the seed of future evidence/provenance work, logged
now because the cost of doing so later (once real usage has already
happened without a record of it) is much higher.

## Architecture

See `Plan.md` and `src/skeptic/core/models.py` for the language-agnostic schema.
Python-specific tool wrappers live in `src/skeptic/adapters/python/` — adding a
new language later means adding a new adapter directory, not rewriting the core.

## Development

```bash
pytest tests/
```
