Metadata-Version: 2.3
Name: test-organizer-pro
Version: 0.0.3
Summary: Test Organizer Pro - Find redundant pytest tests via coverage + mutation analysis
Author: Miha Zidar
Author-email: Miha Zidar <mzidar@fencer.dev>
Requires-Dist: click>=8.1
Requires-Dist: coverage>=7.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown

# TOP — Test Organizer Pro

> **Disclaimer:** This is a fully vibe-coded project and is not yet ready for any serious use.

Find redundant and useless pytest tests by combining coverage analysis with mutation testing.

TOP runs each test individually, measures which source lines it covers (minus import-time baseline), then mutates those lines to see which ones the test actually asserts on. The output is a JSON map you can use to identify tests that cover code without validating it.

## Install

```bash
pip install top
# or with uv
uv add top
```

## Usage

Run TOP from the root of your pytest project:

```bash
top analyze . -s src
```

- `.` — root of your pytest project (where pytest discovers tests)
- `-s` / `--source` — source directory to measure coverage for (repeatable for multiple dirs)
- `-o` / `--output` — output file path relative to project (default: `.top/analysis.json`)
- `--stdout` — print JSON to stdout instead of writing to a file
- `--include-tests` — include test files in coverage/assertion output (off by default)
- `--html` — generate an HTML report in `.top/report/`

By default, results are written to `.top/analysis.json` inside the analyzed project. The `.top/` directory is created automatically.

### Examples

```bash
# Analyze a standard src-layout project (writes to .top/analysis.json)
top analyze . -s src

# Analyze with multiple source directories
top analyze . -s src -s lib

# Save results to a custom path
top analyze . -s src -o results.json

# Include test file coverage in output
top analyze . -s src --include-tests

# Generate an HTML report
top analyze . -s src --html

# Print to stdout for piping
top analyze . -s src --stdout

# Pipe through jq to find useless tests (coverage but zero assertions)
top analyze . -s src --stdout | jq 'to_entries[] | select(.value | to_entries | all(.value.asserted == []))'
```

### Running with uvx (no install needed)

```bash
uvx top analyze . -s src
```

## Output file

TOP writes its analysis to `.top/analysis.json` inside the project by default. This file contains a JSON object mapping each test to the source lines it covers and asserts:

```json
{
  "tests/test_example.py::test_foo": {
    "src/module.py": {
      "covered": [10, 11, 15],
      "asserted": [10, 15]
    }
  }
}
```

- **covered** — lines the test executes beyond the import-time baseline
- **asserted** — subset of covered lines where a mutation causes the test to fail

A test with empty `asserted` lists is useless — it runs code but validates nothing.
A test whose asserted lines are a strict subset of another test's is a candidate for removal.

By default, the output only includes source files — lines covered in the test file itself are excluded. Use `--include-tests` to include them.

The `.top/` directory is a good candidate for `.gitignore` since analysis results are generated and machine-specific.

## HTML report

Use `--html` to generate a browsable HTML report in `.top/report/`:

```bash
top analyze . -s src --html
open .top/report/index.html
```

The report provides a Codecov-style view of your codebase:

- **Index page** — lists all source files with a stacked coverage bar (green = asserted, yellow = covered only, gray = baseline)
- **File pages** — line-by-line source view where each line is color-coded:
  - **Green** — asserted: at least one test fails when this line is mutated
  - **Yellow** — covered only: tests execute this line but no mutation is caught
  - **Gray** — baseline: executed at import time (before any test runs)
  - **White** — not reached by any test

Hover over the test count column to see which tests cover and assert each line.

## Optimize — find and skip redundant tests

After running `top analyze`, use `top optimize` to identify tests whose assertions are entirely covered by other tests:

```bash
top optimize .
```

This reads `.top/analysis.json` and reports which tests are redundant — meaning every line they assert is also asserted by at least one other keeper test.

### Auto-skip redundant tests

Use `--skip` to automatically add skip decorators to redundant test functions:

```bash
top optimize . --skip
```

TOP detects the test framework and applies the appropriate decorator:

- **pytest tests** (standalone functions or classes without `unittest`):
  ```python
  @pytest.mark.skip(reason="Redundant: covered by tests/test_foo.py::test_bar")
  def test_redundant():
      ...
  ```

- **unittest tests** (methods in classes, file imports `unittest`):
  ```python
  @unittest.skip("Redundant: covered by tests/test_foo.py::TestBar::test_baz")
  def test_redundant(self):
      ...
  ```

The `--skip` flag is non-destructive — it only adds decorators, never deletes tests. To undo, remove the `@...skip` decorators. Tests that already have a skip decorator are left unchanged.

### Options

- `-i` / `--input` — analysis JSON path relative to project (default: `.top/analysis.json`)
- `-o` / `--output` — write JSON report with keepers and redundant tests
- `--skip` — add skip decorators to redundant test functions

### Examples

```bash
# Find redundant tests
top optimize .

# Find and auto-skip redundant tests
top optimize . --skip

# Use a custom analysis file
top optimize . -i results.json

# Save the redundancy report as JSON
top optimize . -o .top/optimize.json
```

## How it works

1. **Discover** all tests via `pytest --collect-only`
2. **Baseline** — for each test file, run a noop test that replicates its imports to capture import-time coverage
3. **Coverage** — run each test individually with `coverage.py`, subtract the baseline
4. **Mutation** — for each net-new covered line, apply AST mutations (operator swaps, constant changes, return value changes) and re-run the test. If a mutation makes the test fail, the line is "asserted"

## Requirements

- Python >= 3.12
- Your project must use pytest
- `coverage` (installed as a dependency of TOP)
