Metadata-Version: 2.4
Name: verifaied
Version: 0.1.0.dev26
Summary: Upload local pytest coverage to verifAIed for instant feedback
Project-URL: Homepage, https://pypi.org/project/verifaied/
Author: Kyle Richards
License: MIT
Keywords: ai,coverage,llm,pytest,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27.0
Requires-Dist: rich>=13.7.0
Requires-Dist: typer>=0.12.0
Description-Content-Type: text/markdown

# verifaied

CLI for uploading local pytest coverage to verifAIed so an LLM (or you) can see what's untested without waiting for CI.

## Install

```bash
uv tool install verifaied        # or
pipx install verifaied
```

## Configure

Set the API token (mint one from the verifAIed app). The CLI ships
pointing at the hosted verifAIed API (`https://api.verifaied.app`) by
default; point it elsewhere via `VERIFAIED_API_URL` if you're running
the backend locally or self-hosting:

```bash
export VERIFAIED_API_TOKEN=vr_live_...
# Only set this if you're not using the hosted API:
export VERIFAIED_API_URL=http://localhost:8000   # local dev
```

## Use

From inside any git repo you've connected to verifAIed:

```bash
pytest --cov --cov-report=json --junitxml=junit.xml
verifaied upload
```

That's it — `--repo` is optional. The CLI parses `git remote get-url origin`,
looks the repo up under your account via `/installations/me`, and uses the
resulting UUID. You can also be explicit:

```bash
verifaied upload --repo kyle/verifaied        # owner/name slug
verifaied upload --repo <UUID>                # raw UUID, no lookup
```

The CLI reads `coverage.json`, pulls the source for every file it
references from your working tree, and posts everything to
`/repositories/<id>/local-coverage`. The response prints a summary of
untested / partial / failing functions so you (or your LLM) can fix
them on the next iteration.

### What gets uploaded

The CLI only sends files that appear in `coverage.json`'s `"files"`
map — i.e. exactly the files `pytest --cov` instrumented. There is no
directory walk and no glob:

- `.env`, build artefacts, vendored libraries, and anything outside
  your coverage scope are never read.
- Test files themselves are only uploaded if your coverage config
  includes them (e.g. `--cov=tests`).

To audit the exact file list (and total bytes) before anything leaves
your machine, run with `--dry-run`:

```bash
verifaied upload --dry-run
```

This prints the resolved branch / commit / file table and exits without
contacting the backend. Useful when you're about to upload from an
unfamiliar repo, or when narrowing down where an unexpected file is
coming from.

If a sensitive value did make it into a covered source file (a baked-in
API key in a fixture, etc.), open the repo in the verifAIed web UI,
expand **Recently deleted** under the branches grid, and use
**Permanently delete**. That hard-deletes the analysis row and cascades
to the uploaded source — the per-card **Delete** is a soft-delete that
keeps the data around for accidental-deletion recovery.

### Flags

- `--repo / -r <UUID|owner/name>` — repository to upload to. If omitted,
  the CLI auto-detects from the GitHub `origin` remote.
- `--branch / -b <name>` — branch to attach the upload to (default:
  `git branch --show-current`, then `local`)
- `--coverage <path>` — path to coverage.json (default: `./coverage.json`)
- `--junit <path>` — optional JUnit XML for failing-test detail
- `--commit-sha <sha>` — commit sha for display (default: `git rev-parse HEAD`)
- `--root <path>` — root the coverage paths are relative to (default: cwd)
- `--api-url <url>` — backend base URL (overrides `VERIFAIED_API_URL`)
- `--token <token>` — API token (overrides `VERIFAIED_API_TOKEN`)
- `--dry-run` — print the file list and total bytes that would be
  uploaded, then exit without contacting the backend. Skips the token
  requirement so you can audit without configuring auth.

## Agent loop

We've found the most fruitful way to use verifAIed is to put a short
loop into your coding agent's instructions file (`CLAUDE.md`,
`AGENTS.md`, `.cursorrules`, `GEMINI.md`,
`.github/copilot-instructions.md`) so it self-corrects on every change.

The easiest way is to let your agent write that loop into the rules
file for you. Paste the prompt below into your agent — it will inspect
the repo, find the project's real test command, confirm the CLI is
installed and `VERIFAIED_API_TOKEN` is set, and write the loop section
into the matching rules file, tailored to your repo:

```markdown
We've found that adding a short test-coverage loop to your agent's instructions is the most fruitful way to use verifAIed. I want you to write that loop into this repo's agent-instructions file, tailored to how this repo actually runs its tests.

verifAIed is a coverage analysis tool. Its CLI (`verifaied upload`) pushes a local `coverage.json` + `junit.xml` to the server; its MCP server exposes a `fix_branch` tool that returns a single prompt describing every untested function, partial branch, and failing test on the current branch. The agent loop is: run tests → `verifaied upload` → call `fix_branch` → apply the returned prompt → repeat until `fix_branch` returns no findings.

Do the following, in order:

# 1. Detect the project setup

- **Test runner & command**: look at `pyproject.toml` (`[tool.pytest.ini_options]`, `[project.scripts]`), `pytest.ini`, `tox.ini`, `setup.cfg`, `Makefile` targets (`test`, `check`), `justfile`, or a top-level `scripts/` directory. Use the test command the project already uses — don't invent a new one.
- **Coverage flags**: pytest must produce `coverage.json` with branch coverage AND per-test contexts, plus `junit.xml`. If the existing command already does that, reuse it. Otherwise build the command:
  ```
  pytest --cov --cov-branch --cov-context=test --junitxml=junit.xml
  coverage json --show-contexts -o coverage.json
  ```
  The `coverage json --show-contexts` step is non-negotiable — pytest-cov's `--cov-report=json` alone drops the per-test contexts, which makes verifAIed mark the upload "Needs attention".
- **Agent-instructions file**: pick the one that matches the agent you (the model) are. If unsure, fall back to the project's existing convention.
  - Claude Code: `CLAUDE.md`
  - Codex CLI / OpenAI: `AGENTS.md`
  - Cursor: `.cursorrules` or `.cursor/rules/*.mdc`
  - Gemini CLI: `GEMINI.md`
  - GitHub Copilot: `.github/copilot-instructions.md`
  - If none exists in the repo, create the one matching your own agent.

# 2. Verify the CLI is installed and configured

- Run `verifaied --help`. If it exits non-zero, install it: `uv tool install verifaied` (preferred) or `pipx install verifaied`.
- Check `echo $VERIFAIED_API_TOKEN`. If empty, stop and tell the user to mint one from the verifAIed app (Settings → Tokens) and export it: `export VERIFAIED_API_TOKEN=vr_live_...`. Do NOT invent a placeholder and do NOT proceed without it.

# 3. Write the loop into the agent-instructions file

Append (or create) a section titled `## Test coverage loop with verifAIed` containing:

- The exact test command(s) you built in step 1 — so future runs of the agent don't have to re-derive them.
- The exact upload command (`verifaied upload`, plus any flags this repo needs — e.g. `--coverage path/to/coverage.json` if your test command lands the JSON outside the repo root).
- An instruction to call the `fix_branch` tool on the verifAIed MCP server with `scope: everything`, then apply the returned prompt.
- An instruction to repeat from step 1 until `fix_branch` returns no findings.
- One line: "If `VERIFAIED_API_TOKEN` is unset, stop and ask the user — do not skip this loop."

# 4. Confirm with the user

Show the user the diff of the file you edited (or created) and the test command you wrote in. Do NOT run the loop yourself yet — let the user review first.

# Hard rules

- Do NOT change the test command's behavior beyond adding coverage / junit flags.
- Do NOT add new dependencies if pytest-cov and coverage are already transitive deps.
- Do NOT write a fallback that skips the loop when the token is missing — the agent must stop and ask.
- Do NOT enumerate functions yourself or write tests; `fix_branch` returns the prompt that does that.
- Do NOT replace an existing agent-instructions file — append a new section.
```

### Or paste the instructions in by hand

If you'd rather skip the meta-prompt, drop this section into your
agent's rules file as-is. Swap in your repo's actual test command if
it differs from the pytest default:

```markdown
## Test coverage loop with verifAIed

After implementing or modifying any code in this repo, run this loop until verifAIed reports clean:

1. Run the test suite with branch coverage and per-test contexts:
   ```
   pytest --cov --cov-branch --cov-context=test --junitxml=junit.xml
   coverage json --show-contexts -o coverage.json
   ```
2. Upload the results to verifAIed:
   ```
   verifaied upload
   ```
3. Call the `fix_branch` tool on the verifAIed MCP server with `scope: everything`. It returns a single prompt describing every untested function, partial branch, and failing test on this branch.
4. Apply the fixes the prompt describes.
5. Go back to step 1.

Stop when `fix_branch` returns no findings. If `VERIFAIED_API_TOKEN` is not set, stop and ask the user — do not skip this loop.
```
