Metadata-Version: 2.5
Name: pytest-leela
Version: 0.9.0
Summary: Type-aware mutation testing for Python — fast, opinionated, pytest-native
Project-URL: Homepage, https://github.com/markng/pytest-leela
Project-URL: Issues, https://github.com/markng/pytest-leela/issues
Author-email: Mark Ng <mark@roaming-panda.com>
License-Expression: MIT
License-File: LICENSE
Keywords: mutation-testing,pytest,quality,testing,type-aware
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.12
Requires-Dist: pytest>=7.0
Provides-Extra: dev
Requires-Dist: django-stubs; extra == 'dev'
Requires-Dist: factory-boy; extra == 'dev'
Requires-Dist: faker; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pytest-describe>=2.0; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=5.2; extra == 'django'
Requires-Dist: pytest-django>=4.10; extra == 'django'
Description-Content-Type: text/markdown

# pytest-leela

**Type-aware mutation testing for Python.**

[![PyPI version](https://img.shields.io/pypi/v/pytest-leela)](https://pypi.org/project/pytest-leela/)
[![Python versions](https://img.shields.io/pypi/pyversions/pytest-leela)](https://pypi.org/project/pytest-leela/)
[![License](https://img.shields.io/pypi/l/pytest-leela)](https://github.com/markng/pytest-leela/blob/main/LICENSE)
[![CI](https://img.shields.io/github/actions/workflow/status/markng/pytest-leela/python-package.yml)](https://github.com/markng/pytest-leela/actions)

---

## What it does

pytest-leela runs mutation testing inside your existing pytest session. It injects AST mutations
via import hooks (no temp files), maps each mutation to only the tests that cover that line, and
uses type annotations to skip mutations that can't possibly fail your tests.

It's opinionated: we target latest Python, favour speed over configurability, and integrate
with pytest without separate config files or runners. If that fits your workflow, great.
MIT licensed — fork it if it doesn't.

---

## Install

```
pip install pytest-leela
```

---

## Quick Start

**Run mutation testing on your whole test suite:**

```bash
pytest --leela
```

When no `--target` is given, pytest-leela auto-discovers source files:

1. Looks for a `target/` or `src/` directory (in that order) and mutates all `.py` files inside it
2. If neither exists, falls back to the project root (skipping `.venv`, `build`, `dist`, `__pycache__`, `node_modules`, `.git`, and similar non-source directories)

Test files (`test_*.py`, `*_test.py`, `conftest.py`, `tests.py`) are always excluded from mutation.

Exclude patterns from `[tool.pytest-leela]` (see [Configuration](#configuration)) are applied
after target discovery, so you can auto-discover `src/` while still excluding specific paths.

**Target specific modules (pass `--target` multiple times):**

```bash
pytest --leela --target myapp/models.py --target myapp/views.py
```

**Only mutate lines changed vs a branch:**

```bash
pytest --leela --diff main
```

**Limit CPU cores:**

```bash
pytest --leela --max-cores 4
```

**Cap memory usage:**

```bash
pytest --leela --max-memory 4096
```

**Combine flags:**

```bash
pytest --leela --diff main --max-cores 4 --max-memory 4096
```

**Generate an interactive HTML report:**

```bash
pytest --leela --leela-html report.html
```

**Benchmark optimization layers:**

```bash
pytest --leela-benchmark
```

---

## Features

- **Type-aware mutation pruning** — uses type annotations to skip mutations that can't possibly
  trip your tests (e.g. won't swap `+` to `-` on a `str` operand)
- **Per-test coverage mapping** — each mutant runs only the tests that exercise its lines,
  not the whole suite
- **In-process execution via import hooks** — mutations applied via `sys.meta_path`, zero
  filesystem writes, fast loop
- **Git diff mode** — `--diff <ref>` limits mutations to lines changed since that ref
- **Framework-aware** — clears Django URL caches between mutants so view reloads work correctly
- **Resource limits** — `--max-cores N` caps parallelism; `--max-memory MB` guards memory
- **HTML report** — `--leela-html` generates an interactive single-file report with source viewer, survivor navigation, and test source overlay
- **CI exit codes** — exits non-zero when mutants survive or error, so CI pipelines fail on incomplete kill rates
- **Benchmark mode** — `--leela-benchmark` measures the speedup from each optimization layer

---

## Configuration

Configure pytest-leela in your `pyproject.toml` under `[tool.pytest-leela]`:

```toml
[tool.pytest-leela]
exclude = [
    "*/migrations/*",
    "manage.py",
    "*/conftest.py",
]
operators = [
    "arithmetic",
    "comparison",
    "boolean",
    "return",
]
fail_on_error = true
```

### `exclude`

A list of glob patterns. Files whose path (relative to the project root) matches any pattern
are excluded from mutation. Uses `fnmatch` matching — `*` matches within a single directory,
`*/migrations/*` matches any `migrations` directory at any depth.

Default: `[]` (nothing excluded).

### `operators`

A list of operator category names controlling which mutation types are applied.
Use `"all"` as a shorthand to enable every category.

Default: `["arithmetic", "comparison", "boolean", "unary", "return"]`

| Category | What it mutates | Default |
|---|---|---|
| `arithmetic` | `+` `-` `*` `/` `//` `%` `**` | on |
| `comparison` | `==` `!=` `<` `<=` `>` `>=` `is` `in` and negations | on |
| `boolean` | `and` / `or` | on |
| `unary` | unary `-` `+` `not` | on |
| `return` | return values (`True`/`False`, `None`, literals, expressions) | on |
| `bitwise` | `&` `\|` `^` `<<` `>>` | off |
| `augmented_assign` | `+=` `-=` `*=` etc. | off |
| `ternary` | `x if cond else y` branch swaps | off |
| `control_flow` | `break` / `continue` swaps | off |
| `exception` | `except` handler broadening, body-to-raise | off |

### `fail_on_error`

Whether errored mutants fail the session (non-zero exit). Default: `true`.

See [Mutant statuses](#mutant-statuses). Set it to `false` only when you accept that some
mutants go untested, for example while fixing a known collection problem.

---

## Mutant statuses

Each mutant ends in exactly one of three states:

| Status | Meaning | Counts toward the score |
|---|---|---|
| **killed** | At least one test failed or errored in setup/teardown, a test module failed to collect, the run timed out (an infinite loop introduced by the mutant), or a conftest failed to import because the mutated module raised while being imported. A setup error or a collection failure caused by the environment or by nondeterminism rather than the mutant (a flaky fixture, a module that sometimes fails to import) also counts as a kill ([#13](https://github.com/markng/pytest-leela/issues/13)) | yes |
| **survived** | Tests ran and all of them passed | yes |
| **error** | The inner run never tested the mutant: pytest crashed (for example a conftest that raised `Skipped` at import, or called `pytest.exit()` from `pytest_addoption`), a conftest called `pytest.exit()` at import or from a collection hook, a conftest's own check failed, the run exited abnormally with no test failure or collection failure (usage error, nothing collected, interrupted), or ran zero tests (everything skipped) | no |

The mutation score is `killed / (killed + survived)`. Errors are listed separately, with their
reason, in the terminal report, the HTML report and the JSON output, and they fail the session
unless `fail_on_error = false`. An error means leela could not tell whether your tests catch the
mutant, so it is reported as neither a kill nor a survival.

A test module that fails to collect under the mutant is a kill: the clean baseline has already
shown it collects unmutated, so the mutant broke it. That covers the *target module itself*
raising at import time (module-level code) and a test module's own module-level check, such as
`assert calc.RATIO == 5`. The killing tests are the files that failed to collect.

A conftest that fails to import counts as a kill only when the mutated module itself raised
during that import. A conftest's own check does not count, nor does a conftest that catches the
error and calls `pytest.exit()` or `pytest.skip()`: those are errors. So the same failing check
is deliberately treated differently by location: `assert calc.RATIO == 5` at module level in a
test module is a kill, but in a conftest it is an error, because a conftest failure is not
attributed to the mutant.

If a test caught the import error and the suite completed green, the mutant **survived**: no
test noticed it. If the mutant made the tests skip themselves (a module-level `pytest.skip`, a
`skipif` marker, or a conftest or test module that skips or calls `pytest.exit()` at import),
the mutant is an **error**: no test ran.

Known limits of these rules:

- **A module that can only be imported once per process.** If a test module imports something
  that refuses to load twice (for example it raises when a process-global registry is already
  installed), every re-import fails to collect. Through `pytest --leela` the outer session has
  already imported it, so the clean baseline fails and the run aborts. When `Engine.run` is called
  directly with `use_coverage=False` and a test directory, the baseline is the first import and
  passes, and every mutant is then scored KILLED with `tests_run: 0`
  ([#13](https://github.com/markng/pytest-leela/issues/13)).
- **A deliberate exit hides every kill not yet reported.** When a conftest or a test module
  catches the target's import error and calls `pytest.exit()`, the inner run ends there and the
  mutant is an **error**. Tests run only after collection, so no test failure is ever reported,
  including a test that imports the target inside its body; nor is a sibling module's collection
  failure that pytest would have reached after the exit. Only a module that already failed to
  collect before the exit stops the run first (`-x`) and still kills.

### Clean baseline before any mutant

Before the first mutant, leela runs the selected tests once with no mutation applied, through
the same in-process path every mutant uses. If anything fails or errors there, the run aborts
with `leela: aborted: tests do not pass with no mutation applied (...)` and a non-zero exit,
and no mutants are scored. Without this check, a test that fails for an unrelated reason (a
fixture that cannot run in-process, a missing service) would score every mutant as killed.

### pytest-django

pytest-django blocks database access for the whole session and only restores it at
unconfigure, after leela has run. Leela lifts that outer block for the duration of its run
(through pytest-django's own `django_db_blocker.unblock()`), so database tests in the inner
sessions can open connections as they do in a normal run.

Django model and admin modules are never reloaded between mutants (unless one is itself a
`--target`), and neither is any project module they reference. Re-executing them registers
models and admin classes a second time, which Django rejects, and re-importing a module they
reference would split class identity (a kept model subclassing the old copy of a reloaded
mixin). The trade-off: if a model or admin module imports a function from your target at
module level, calls through that reference run the unmutated code, which shows up as a
survivor, never as a fake kill.

### Installed packages are never reloaded

Between mutants leela evicts the project's own modules so that mutated code is re-imported.
Installed packages are left loaded, even when the virtualenv sits inside the project directory
(uv's default `.venv`): anything under `sys.prefix`, `sys.base_prefix`, `site.getsitepackages()`,
`site.getusersitepackages()`, or a `site-packages` / `dist-packages` directory. Re-importing them
would break C extensions such as numpy, which CPython can load only once per process.

---

## Pragma Skip

Add `# leela: skip` to suppress mutations on individual lines:

```python
result = x + y  # leela: skip
```

To skip an entire file, place the pragma on line 1:

```python
# leela: skip
"""This module is excluded from mutation testing."""

def legacy_code():
    ...
```

The pragma is detected via Python's `tokenize` module, so it only matches actual comments —
not string literals or docstrings that happen to contain the text.

---

## HTML Report

`--leela-html report.html` generates a single self-contained HTML file with no external dependencies.

**What it shows:**
- Overall mutation score badge
- Per-file breakdown with kill/survive/error counts
- Source code viewer with syntax highlighting

**Interactive features:**
- Click any line to see mutant details (original → mutated code, status, relevant tests)
- Survivor navigation overlay — keyboard shortcuts: `n` next survivor, `p` previous, `l` list all, `Esc` close
- Test source overlay — click any test name to see its source code

Uses the Catppuccin Mocha dark theme.

---

## Requirements

- Python >= 3.12
- pytest >= 7.0

---

## License

MIT
