Metadata-Version: 2.4
Name: pytest-flakefighters
Version: 0.3.1
Summary: Pytest plugin implementing flaky test failure detection and classification.
Author: TestFLARE Team
Project-URL: Documentation, https://pytest-flakefighters.readthedocs.io
Project-URL: Homepage, https://test-flare.github.io/
Project-URL: Issues, https://github.com/test-flare/pytest-flakefighters/issues
Project-URL: Repository, https://github.com/test-flare/pytest-flakefighters
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: GitPython>=3.1.45
Requires-Dist: coverage>=7.10.6
Requires-Dist: dotenv>=0.9.9
Requires-Dist: nltk>=3.9
Requires-Dist: pandas>=2.3
Requires-Dist: pytest>=6.2.0
Requires-Dist: pyyaml>=6
Requires-Dist: scikit-learn>=1.7
Requires-Dist: scipy<=1.15
Requires-Dist: sqlalchemy>=2.0.43
Requires-Dist: unidiff>=0.7.5
Provides-Extra: dev
Requires-Dist: black; extra == "dev"
Requires-Dist: myst_parser; extra == "dev"
Requires-Dist: nbsphinx; extra == "dev"
Requires-Dist: pre_commit; extra == "dev"
Requires-Dist: pylint; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-html; extra == "dev"
Requires-Dist: pytest-json-report>=1.5; extra == "dev"
Requires-Dist: sphinx-autoapi; extra == "dev"
Requires-Dist: sphinx_rtd_theme; extra == "dev"
Requires-Dist: tox>=4.31.0; extra == "dev"
Provides-Extra: pg
Requires-Dist: psycopg2>2.9; extra == "pg"
Dynamic: license-file

# Pytest FlakeFighters

[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)
[![PyPI version](https://img.shields.io/pypi/v/pytest-flakefighters.svg)](https://pypi.org/project/pytest-flakefighters)
[![Python versions](https://img.shields.io/badge/python-3.10_--_3.13-blue)](https://pypi.org/project/pytest-flakefighters)
![Test status](https://github.com/test-flare/pytest-flakefighters/actions/workflows/ci-tests.yaml/badge.svg)
[![codecov](https://codecov.io/gh/test-flare/pytest-flakefighters/branch/main/graph/badge.svg?token=04ijFVrb4a)](https://codecov.io/gh/test-flare/pytest-flakefighters)
[![Documentation Status](https://readthedocs.org/projects/pytest-flakefighters/badge/?version=latest)](https://pytest-flakefighters.readthedocs.io/en/latest/?badge=latest)
![GitHub License](https://img.shields.io/github/license/test-flare/pytest-flakefighters)

### Pytest plugin implementing flaky test failure detection and classification.
Read more about flaky tests [here](https://docs.pytest.org/en/stable/explanation/flaky.html).

## Features

- Implements the [DeFlaker algorithm](http://www.deflaker.org/get-rid-of-your-flakes/) for pytest
- Implements two traceback-matching classifiers from [Alshammari et al. (2024)](https://doi.org/10.1109/ICST60714.2024.00031).
- Implements a novel coverage-independence classifier that classifies tests as flaky if they fail independently of passing test cases that exercise overlapping code.
- Optionally rerun flaky failures
- Output results to JSON, HTML, or JUnitXML
- Save test outcome history to a remote or local database

## Installation

You can install \"pytest-flakefighters\" by cloning this repo and running `pip install .` from the root directory.
If you intend to develop the plugin, run `pip install -e .[dev]` instead.

We eventually intend to distribute our tool on PyPI.

## Usage

FlakeFighter is intended to run on git repositories that have test suites runnable with `pytest`.
Once you have installed FlakeFighter, you can run it from the root directory of your repo simply by running `pytest` in your usual way.
FlakeFighter has the following arguments.

```
  --target-commit=TARGET_COMMIT
                        The target (newer) commit hash. Defaults to HEAD (the most recent commit).
  --source-commit=SOURCE_COMMIT
                        The source (older) commit hash. Defaults to HEAD^ (the previous commit to target).
  --repo=REPO_ROOT      The commit hash to compare against.
  --suppress-flaky-failures-exit-code
                        Return OK exit code if the only failures are flaky failures.
  --no-save             Do not save this run to the database of previous flakefighters runs.
  -M LOAD_MAX_RUNS, --load-max-runs=LOAD_MAX_RUNS
                        The maximum number of previous runs to consider.
  -D DATABASE_URL, --database-url=DATABASE_URL
                        The database URL. Defaults to 'flakefighter.db' in current working directory.
  --store-max-runs=STORE_MAX_RUNS
                        The maximum number of previous flakefighters runs to store. Default is to store all.
  --time-immemorial=TIME_IMMEMORIAL
                        How long to store flakefighters runs for, specified as `days:hours:minutes`. E.g. to store
                        tests for one week, use 7:0:0.
```

## Contributing

Contributions are very welcome.
Tests can be run with [pytest](https://pytest.readthedocs.io/en/latest/), please ensure the coverage at least stays the same before you submit a pull request.

## Flake Fighters
Our plugin is made up of a collection of heuristics that come together to help inform whether a test failure is genuine or flaky.
These come in two "flavours": those which run live after each test, and those which run at the end of the entire test suite.
Both extend the base class `FlakeFighter` and implement the `flaky_failure` method, which returns `True` if the test is deemed to be flaky.

## Issues

If you encounter any problems, please [file an issue](https://github.com/test-flare/pytest-flakefighters/issues) along with a detailed description.

------------------------------------------------------------------------

This [pytest](https://github.com/pytest-dev/pytest) plugin was generated with [Cookiecutter](https://github.com/audreyr/cookiecutter) along with [\@hackebrot](https://github.com/hackebrot)\'s [cookiecutter-pytest-plugin](https://github.com/pytest-dev/cookiecutter-pytest-plugin) template.
