Metadata-Version: 2.5
Name: testcost
Version: 0.1.0
Summary: Find out where your test suite's time actually goes: collection, imports, fixtures, or the tests themselves.
Project-URL: Homepage, https://github.com/aviseth/testcost
Project-URL: Repository, https://github.com/aviseth/testcost
Project-URL: Changelog, https://github.com/aviseth/testcost/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/aviseth/testcost/issues
Author-email: Avi Seth <avi@crispa.ai>
License-Expression: MIT
License-File: LICENSE
Keywords: ci,fixtures,performance,profiling,pytest,slow-tests,test-speed
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: pytest>=8
Requires-Dist: tomli>=2.0; python_version < '3.11'
Description-Content-Type: text/markdown

# testcost

Find out where your test suite's time actually goes.

[![PyPI](https://img.shields.io/pypi/v/testcost.svg)](https://pypi.org/project/testcost/)
[![Python](https://img.shields.io/pypi/pyversions/testcost.svg)](https://pypi.org/project/testcost/)
[![CI](https://github.com/aviseth/testcost/actions/workflows/ci.yml/badge.svg)](https://github.com/aviseth/testcost/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

`pytest --durations` ranks tests by how long each one took. That answers the wrong question most
of the time: a four minute suite rarely contains a four minute test. It contains a session fixture
that builds a database, or three hundred modules imported during collection, or a function-scoped
fixture that costs 20ms and runs two thousand times.

testcost attributes the time to collection, imports, fixture setup and teardown, and the test
bodies themselves, so the largest line in the report is the thing worth fixing.

## Installation

```shell
pip install testcost
```

Requires Python 3.10+ and pytest 8+.

## Quick start

```shell
testcost run -- tests
```

```text
4m12.3s for 1,284 tests
18.4s of that is importing, pytest included

seconds  share  where
-------  -----  ------------
 2m41.0s   64%  fixtures
   58.2s   23%  tests
   19.8s    8%  collection
   13.3s    5%  unattributed

fixtures
 total  runs   each  scope     fixture
------  ----  -----  --------  --------------------------------
1m52.0s     1  112s  session   postgres_container  tests.conftest
  38.4s  1920   20ms  function  db_transaction      tests.conftest
  10.6s    64  166ms  module    seeded_catalogue    tests.fixtures.data

worth a wider scope
  db_transaction costs 20ms and runs 1920 times (38.4s total). If it does not need to be rebuilt
  per test, a module or session scope removes most of that.
```

## What the numbers mean

**fixtures** is setup plus teardown, totalled per fixture across every time it ran. The `each`
column is that total divided by the number of setups, so it includes teardown work too. Teardown is
attributed by wrapping finalizers as they are registered, since pytest reports setup time for you
but hands teardown to finalizers attached to the fixture definition.

**tests** is the call phase only. pytest charges fixture setup to the test's setup phase, so
adding the setup phase and the fixture totals together would count the same seconds twice.

**collection** is the time between pytest starting collection and finishing it, which is where
your test modules get imported.

**unattributed** is whatever the run spent that none of the buckets explain: interpreter startup,
pytest's own import, reporting, plugin overhead. It is reported rather than hidden, because a
large value there is itself worth knowing.

**imports** is measured separately, in its own `--collect-only` run under `-X importtime`, with a
bare interpreter's startup subtracted. It covers pytest and its plugins as well as your modules,
so it overlaps the breakdown rather than being another slice of it. That is why it sits on its own
line. Skip it with `--no-imports` if you only want the session numbers.

Fixtures with the same name in different files are tracked separately, so a `client` fixture
defined in three conftests does not appear as one confusing total.

## Budgets in CI

```toml
[tool.testcost]
pytest_args = ["tests"]
max_total_seconds = 300
max_collect_seconds = 10
max_import_seconds = 5
```

```shell
testcost check
```

```text
fail 6m02.1s total, 19.8s collecting, 24.1s importing, 1284 tests
     total took 6m02.1s, over the 5m00.0s budget by 1m02.1s
     imports took 24.1s, over the 5.00s budget by 19.1s

     heaviest fixtures: postgres_container (1m52.0s), db_transaction (38.4s), seeded_catalogue (10.6s)
```

Exits non-zero when a budget is blown, and names the heaviest fixtures so the next step is obvious.

## Configuration

All keys live under `[tool.testcost]` in `pyproject.toml`. All are optional.

| Key | Type | Default | Meaning |
| --- | --- | --- | --- |
| `pytest_args` | list of strings | `[]` | Arguments passed to pytest when none are given on the command line |
| `max_total_seconds` | number | none | Fail `check` if the whole run takes longer |
| `max_collect_seconds` | number | none | Fail `check` if collection takes longer |
| `max_import_seconds` | number | none | Fail `check` if imports take longer |

## Command reference

| Command | What it does |
| --- | --- |
| `testcost run -- <pytest args>` | Profile a run and print the breakdown |
| `testcost run --json` | The same, as JSON |
| `testcost run --limit N` | Rows per table, default 15 |
| `testcost run --no-imports` | Skip the separate collect-only import pass |
| `testcost check` | Profile and exit non-zero if a budget is exceeded |

Both commands exit non-zero if pytest itself did, so a CI step cannot go green after the suite
went red. The profile is still printed, because it is still valid.

## How it compares

| Tool | Ranks tests | Fixture attribution | Import cost | CI budget | Maintained |
| --- | --- | --- | --- | --- | --- |
| `pytest --durations` | yes | no | no | no | yes |
| `pytest-durations` | yes | no | no | no | yes |
| `pytest-profiling` | yes | no | no | no | last release 2024 |
| `pytest-monitor` | yes | no | no | no | last release 2023 |
| testcost | yes | yes | yes | yes | yes |

## Notes

Times come from one run, so a suite with genuinely variable timing needs more than one look. There
is no averaging across runs yet.

The plugin is installed as a pytest entry point, which means it is imported by every pytest
process on the machine. Without `--testcost-report` or `TESTCOST_REPORT` it registers nothing and
every hook returns immediately.

Fixture teardown attribution wraps finalizers on the fixture definition. If another plugin
replaces the finalizer list wholesale after setup, that fixture's teardown time will be missing
rather than wrong.

Fixtures are identified by the file that defines them rather than the module name, because every
standalone `conftest.py` imports as `conftest` and three of them defining `client` would otherwise
collapse into one row belonging to none of them.

Running under `-n` with `pytest-xdist` is not supported yet: timings from several worker processes
are not merged.

## Contributing

Bug reports and pull requests are welcome. `uv sync` then `uv run pytest` to get started.

## License

MIT.
