Metadata-Version: 2.3
Name: test-organizer-pro
Version: 0.0.1
Summary: Test Organizer Pro - Find redundant pytest tests via coverage + mutation analysis
Author: Miha Zidar
Author-email: Miha Zidar <mzidar@fencer.dev>
Requires-Dist: click>=8.1
Requires-Dist: coverage>=7.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown

# TOP — Test Organizer Pro

Find redundant and useless pytest tests by combining coverage analysis with mutation testing.

TOP runs each test individually, measures which source lines it covers (minus import-time baseline), then mutates those lines to see which ones the test actually asserts on. The output is a JSON map you can use to identify tests that cover code without validating it.

## Install

```bash
pip install top
# or with uv
uv add top
```

### Install from a local checkout

If you have TOP cloned locally, add it as an editable dev dependency to any project:

```bash
uv add --dev --editable /path/to/top
```

This adds TOP and its dependencies (`click`, `coverage`) to the project's virtual environment so `top analyze` runs with the same interpreter as the project's own tests.

## Usage

Run TOP from the root of your pytest project:

```bash
top analyze . -s src
```

- `.` — root of your pytest project (where pytest discovers tests)
- `-s` / `--source` — source directory to measure coverage for (repeatable for multiple dirs)
- `-o` / `--output` — output file path relative to project (default: `.top/analysis.json`)
- `--stdout` — print JSON to stdout instead of writing to a file
- `--include-tests` — include test files in coverage/assertion output (off by default)
- `--html` — generate an HTML report in `.top/report/`

By default, results are written to `.top/analysis.json` inside the analyzed project. The `.top/` directory is created automatically.

### Examples

```bash
# Analyze a standard src-layout project (writes to .top/analysis.json)
top analyze . -s src

# Analyze with multiple source directories
top analyze . -s src -s lib

# Save results to a custom path
top analyze . -s src -o results.json

# Include test file coverage in output
top analyze . -s src --include-tests

# Generate an HTML report
top analyze . -s src --html

# Print to stdout for piping
top analyze . -s src --stdout

# Pipe through jq to find useless tests (coverage but zero assertions)
top analyze . -s src --stdout | jq 'to_entries[] | select(.value | to_entries | all(.value.asserted == []))'
```

### Running with uvx (no install needed)

```bash
uvx top analyze . -s src
```

## Output file

TOP writes its analysis to `.top/analysis.json` inside the project by default. This file contains a JSON object mapping each test to the source lines it covers and asserts:

```json
{
  "tests/test_example.py::test_foo": {
    "src/module.py": {
      "covered": [10, 11, 15],
      "asserted": [10, 15]
    }
  }
}
```

- **covered** — lines the test executes beyond the import-time baseline
- **asserted** — subset of covered lines where a mutation causes the test to fail

A test with empty `asserted` lists is useless — it runs code but validates nothing.
A test whose asserted lines are a strict subset of another test's is a candidate for removal.

By default, the output only includes source files — lines covered in the test file itself are excluded. Use `--include-tests` to include them.

The `.top/` directory is a good candidate for `.gitignore` since analysis results are generated and machine-specific.

## HTML report

Use `--html` to generate a browsable HTML report in `.top/report/`:

```bash
top analyze . -s src --html
open .top/report/index.html
```

The report provides a Codecov-style view of your codebase:

- **Index page** — lists all source files with a stacked coverage bar (green = asserted, yellow = covered only, gray = baseline)
- **File pages** — line-by-line source view where each line is color-coded:
  - **Green** — asserted: at least one test fails when this line is mutated
  - **Yellow** — covered only: tests execute this line but no mutation is caught
  - **Gray** — baseline: executed at import time (before any test runs)
  - **White** — not reached by any test

Hover over the test count column to see which tests cover and assert each line.

## Optimize — find and skip redundant tests

After running `top analyze`, use `top optimize` to identify tests whose assertions are entirely covered by other tests:

```bash
top optimize .
```

This reads `.top/analysis.json` and reports which tests are redundant — meaning every line they assert is also asserted by at least one other keeper test.

### Auto-skip redundant tests

Use `--skip` to automatically add skip decorators to redundant test functions:

```bash
top optimize . --skip
```

TOP detects the test framework and applies the appropriate decorator:

- **pytest tests** (standalone functions or classes without `unittest`):
  ```python
  @pytest.mark.skip(reason="Redundant: covered by tests/test_foo.py::test_bar")
  def test_redundant():
      ...
  ```

- **unittest tests** (methods in classes, file imports `unittest`):
  ```python
  @unittest.skip("Redundant: covered by tests/test_foo.py::TestBar::test_baz")
  def test_redundant(self):
      ...
  ```

The `--skip` flag is non-destructive — it only adds decorators, never deletes tests. To undo, remove the `@...skip` decorators. Tests that already have a skip decorator are left unchanged.

### Options

- `-i` / `--input` — analysis JSON path relative to project (default: `.top/analysis.json`)
- `-o` / `--output` — write JSON report with keepers and redundant tests
- `--skip` — add skip decorators to redundant test functions

### Examples

```bash
# Find redundant tests
top optimize .

# Find and auto-skip redundant tests
top optimize . --skip

# Use a custom analysis file
top optimize . -i results.json

# Save the redundancy report as JSON
top optimize . -o .top/optimize.json
```

## How it works

1. **Discover** all tests via `pytest --collect-only`
2. **Baseline** — for each test file, run a noop test that replicates its imports to capture import-time coverage
3. **Coverage** — run each test individually with `coverage.py`, subtract the baseline
4. **Mutation** — for each net-new covered line, apply AST mutations (operator swaps, constant changes, return value changes) and re-run the test. If a mutation makes the test fail, the line is "asserted"

## Requirements

- Python >= 3.12
- Your project must use pytest
- `coverage` (installed as a dependency of TOP)
