Metadata-Version: 2.4
Name: assert-pytest-test-can-fail
Version: 20260908004717
Summary: CLI tool to assert that every pytest test is able to fail
Author-email: 10U Labs <dev@10ulabs.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/10U-Labs/assert-pytest-test-can-fail
Project-URL: Repository, https://github.com/10U-Labs/assert-pytest-test-can-fail
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Dynamic: license-file

# assert-pytest-test-can-fail

Assert that every pytest test is able to fail.

## Why Must a Test Be Able to Fail?

A test earns its place by being able to fail. A test whose assertion is
satisfied on every path it can reach runs, costs what it costs, and reports
green for the whole life of the thing it claims to watch. Coverage counts it.
The suite counts it. The only way anybody finds out is by reading it.

One shape produces this reliably. A test sets a flag to `False`, asks a remote
service about a resource, sets the flag to `True` when the call succeeds, and
sets the same flag to `True` again in the handler for the error that means the
resource is absent. Every other error is re-raised. Then it asserts the flag.
Every path that reaches the assertion has set it to `True`, so the assertion is
decoration, and a genuinely missing resource is reported as present.

Mutation testing answers this question in general and answers it better, by
mutating the code under test and seeing whether a test dies. It cannot answer
it for a test with no code under test — one that asserts about a deployed
resource through a client library — and it costs a suite run per mutant. This
tool asks a narrower question that a parse can settle.

## Installation

```bash
pip install assert-pytest-test-can-fail
```

## Usage

### Command Line

```bash
# Scan specific files
assert-pytest-test-can-fail test_example.py

# Scan directories recursively
assert-pytest-test-can-fail test/

# Use glob patterns
assert-pytest-test-can-fail "test/**/test_*.py"

# Exclude patterns
assert-pytest-test-can-fail test/ --exclude "**/conftest.py"

# Verbose output
assert-pytest-test-can-fail test/ --verbose

# Fail fast (exit on first finding)
assert-pytest-test-can-fail test/ --fail-fast

# Warn only (always exit 0)
assert-pytest-test-can-fail test/ --warn-only
```

### GitHub Actions

```yaml
- uses: 10U-Labs/assert-pytest-test-can-fail@latest
  with:
    files: "test/"
    verbose: "true"
```

### As a Python Module

```bash
python -m assert_pytest_test_can_fail test/
```

## Output Format

Default output shows one finding per line:

```text
path/to/test_file.py:23:test_bucket_exists: every path to the assert sets the flag, so the test cannot fail
```

Format: `file_path:line_number:function_name`, followed by the reason. The line
number is the `assert` statement.

## Exit Codes

- `0`: No findings (or `--warn-only` specified)
- `1`: Findings detected
- `2`: Error (missing files, syntax errors, etc.)

## What Is Reported

A function whose name begins with `test` is reported when all of these are
true:

- it contains a `try` statement with at least one `except` handler
- the statement that ends the `try` body — or the `else` clause, when there is
  one — assigns `True` to a bare name
- every handler on that `try` is closed: it ends in `raise`, or it ends
  assigning `True` to that same name, or it ends in an `if`/`else` whose two
  branches each do one of those
- an `assert` on that bare name follows the whole `try` statement

The last one matters. An assertion sitting inside the `try`, or above it, is
asking a different question, and is left alone.

Each reported function is reported once, at the first assertion that qualifies.

## What Is Never Reported

The shape alone is not the defect, and a rule that says otherwise is a rule
nobody keeps. A permission check written this way is correct: it asks whether
the credentials may inspect a resource, a 404 answers yes, and any error code
the handler does not name leaves the flag `False`, so the assertion fails.

```python
def test_can_describe_bucket():
    has_permission = False
    try:
        client.head_bucket(Bucket=name)
        has_permission = True
    except ClientError as error:
        code = error.response["Error"]["Code"]
        if code == "403":
            pytest.fail("No permission to inspect the bucket")
        if code == "404":
            has_permission = True
    assert has_permission
```

That handler ends in an `if` with no `else`, so it can complete with the flag
still `False`. It is not reported. Neither is a handler that ends in `pass`,
nor a `pytest.fail` that leaves other codes falling through — only `raise` and
an assignment on every branch close a handler.

Also never reported:

- a `try` with no `except` handler at all, such as `try`/`finally`
- a success path that computes the flag rather than asserting it, such as
  `exists = client.bucket_exists(name)`
- a chained assignment, `exists = asked = True`, or an attribute target,
  `record.exists = True`
- an assertion on anything but a bare name, such as `assert count == 1`
- a `try` inside a nested function, a lambda or a class the test defines: the
  tool does not descend into a scope the test opens
- a function whose name does not begin with `test`

Only files pytest would collect are scanned: `test_*.py` and `*_test.py`.

## Options

| Option                | Description                                |
| --------------------- | ------------------------------------------ |
| `--exclude PATTERNS`  | Glob patterns to exclude (comma-separated) |
| `--quiet`             | Suppress all output (exit code only)       |
| `--count`             | Output only the number of findings         |
| `--verbose`           | Show detailed processing information       |
| `--fail-fast`         | Exit after first finding                   |
| `--warn-only`         | Always exit 0, even with findings          |

## License

Apache 2.0 - See [LICENSE.txt](LICENSE.txt)
