Metadata-Version: 2.3
Name: universal-test-runner
Version: 0.8.0
Summary: Universal, language-aware unit test runner.
Keywords: testing,test-runner,pytest
Author: David Brownman
Author-email: David Brownman <oss@xavd.id>
License: MIT License
         
         Copyright (c) 2023 David Brownman
         
         Permission is hereby granted, free of charge, to any person obtaining a copy
         of this software and associated documentation files (the "Software"), to deal
         in the Software without restriction, including without limitation the rights
         to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
         copies of the Software, and to permit persons to whom the Software is
         furnished to do so, subject to the following conditions:
         
         The above copyright notice and this permission notice shall be included in all
         copies or substantial portions of the Software.
         
         THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
         IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
         FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
         AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
         LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
         OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
         SOFTWARE.
Classifier: Programming Language :: Python :: 3
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Testing
Requires-Dist: click==8.1.3
Requires-Dist: colorama==0.4.6
Requires-Python: >=3.11
Project-URL: Homepage, https://github.com/xavdid/universal-test-runner
Project-URL: Bug Tracker, https://github.com/xavdid/universal-test-runner/issues
Project-URL: Author, https://xavd.id
Project-URL: Changelog, https://github.com/xavdid/universal-test-runner/blob/main/CHANGELOG.md
Description-Content-Type: text/markdown

# universal-test-runner

The Universal Test Runner is a zero-configuration, language-aware way to run unit tests in any project. It installs a command, `t`, which will determine how to run your test suite (and then run it).

<p align="center">
   <a href="https://github.com/xavdid/test-runner-demo/blob/main/_demo/demo-min.gif">
      <img src="https://raw.githubusercontent.com/xavdid/test-runner-demo/main/_demo/demo-min.gif"/>
   </a>
</p>

If you're working on a JS project, it runs `[your package manager here] test`. You've run `pytest` in this folder before? `pytest` it is. Rust project? `cargo test` coming right up. Is also clever about running all your `go` module tests (regardless of how they're organized). No matter the command, all args are passed directly into the test runner.

Currently [supports 7 languages](#supported-languages) (and their respective test frameworks). Please open an issue if I'm missing your favorite!

## Installation

The easiest way to install is by using [uv](https://docs.astral.sh/uv/):

```bash
uv tool install universal-test-runner
```

Or with [pipx](https://pypa.github.io/pipx/):

```bash
pipx install universal-test-runner
```

You can also use brew (which will build from source and take a little longer):

```bash
brew install xavdid/projects/universal-test-runner
```

## Usage

> You can also clone the [demo repo](https://github.com/xavdid/test-runner-demo) to play around with the test runner - it's got toy examples to show how tests are run in many languages!

Once installed, the command `t` will be available. Run it in a project folder's root and it'll do its best to run your unit tests:

```
% t
-> pytest
=============================== test session starts ================================
platform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/username/projects/test-runner
collected 78 items

tests/test_cli.py ...                                                        [  3%]
tests/test_context.py .....................                                  [ 30%]
tests/test_matchers.py ..................................................    [ 94%]
tests/test_runner.py ....                                                    [100%]

================================ 78 passed in 0.08s ================================
```

It passes all arguments and environment modifications down to the chosen test runner:

```
% t -k test_builder --verbose
-> pytest -k test_builder --verbose
=============================== test session starts ================================
platform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0
cachedir: .pytest_cache
rootdir: /Users/username/projects/test-runner
collected 78 items / 77 deselected / 1 selected

tests/test_context.py::test_builder PASSED                                   [100%]

========================= 1 passed, 77 deselected in 0.03s =========================
```

It prints the command it's running as part of the output. To disable that behavior, set `UTR_DISABLE_ECHO` environment variable to anything besides `0`.

If it can't guess the testing method, it will tell you so. Feel free to open an issue to request wider language support!

### Debugging

The package also ships a command to surface info about itself: `universal-test-runner`. It has a few key pieces of functionality:

- the `universal-test-runner --version` flag, which prints info about your installed package version
- the `universal-test-runner debug` command, which prints info about which command would run (and why):

```
% universal-test-runner debug
[universal-test-runner]: checking each handler for first match
[universal-test-runner]:   Checking command 01/11: pytest
[universal-test-runner]:     looking for: ".pytest_cache"
[universal-test-runner]:     no match, continuing
[universal-test-runner]:   Checking command 02/11: py
[universal-test-runner]:     looking for: "tests.py"
[universal-test-runner]:     no match, continuing
[universal-test-runner]:   Checking command 03/11: go_multi
[universal-test-runner]:     looking for: "go.mod" and no arguments
[universal-test-runner]:     no match, continuing
[universal-test-runner]:   Checking command 04/11: go_single
[universal-test-runner]:     looking for: "go.mod" or a file named "..._test.go"
[universal-test-runner]:     no match, continuing

...

[universal-test-runner]: no matching test handler. To add a new one, please file an issue: https://github.com/xavdid/universal-test-runner/issues
```

### Clearing the Terminal

To clear the terminal and scrollback buffer before running the test command, set the `UTR_CLEAR_PRE_RUN` environment variable to anything besides `0`.

This functionality has been tested on iTerm2, `Terminal.app`, and Kitty. Please open an issue if it doesn't work on your terminal.

## Supported Languages

This list describes how each language behaves (but not the order in which languages are matched; use the [debugger](#debugging) for that).

- Python
  - checks for `manage.py` (Django)
  - else tries to determine if you use `pytest` in rough order of simplicity. It checks:
    - if you've got a `.pytest-cache` or `pytest.ini`
    - if there's a `[pytest]` line in `tox.ini`
    - if there's a `setup.cfg` and a `[tool:pytest]` line
    - otherwise, it tries to read `pyproject.toml`
      - if you're on Python 3.11+, it parses the file and checks for dependency [locations](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-optional-dependencies) [for](https://docs.astral.sh/uv/concepts/dependencies/#development-dependencies) [popular](https://python-poetry.org/docs/managing-dependencies/#dependency-groups) [tools](https://pdm-project.org/latest/usage/dependency/#add-development-only-dependencies)
      - otherwise, it does a best-effort regex against the file contents, looking for `[tool.pytest.ini_options]` or [dependency specifiers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers) like `pytest >= 2`
  - if you're using a popular package manager (`uv`, `pdm`, `poetry`) it'll run `<package manager> run pytest`
  - otherwise, it runs `pytest` directly under the assumption it's available on the `$PATH`
  - lastly, if there are _any_ python-related files, it runs `python -m unittest`, which does its own discovery
- Rust
  - `cargo nextest run` if nextest is installed
  - `cargo test` otherwise
- Go
  - if there's a `X_test.go`, then runs a plain `go test`
  - if you pass any args at all, runs `go test your-args-here`
  - otherwise, runs `go test ./...`
- Elixir
  - `mix test`
- Clojure
  - `lein test`
- Javascript/Typescript
  - if there's a `package.json` and it has a `test` script, runs `[package manager] test`, where `[package manager]` is:
    - `npm` if there's a `package-lock.json`
    - `yarn` if there's a `yarn.lock`
    - `pnpm` if there's a `pnpm-lock.yaml`
  - `bun test` if there's a `bun.lockb`
- [Just](https://github.com/casey/just)
  - if there are any common justfile names, it uses the JSON api to find a `test` command
  - if `just` isn't installed, it does its best to parse the file as a string
- Makefile
  - looks for a line that starts with `test:`

### Exercism

[Exercism](https://exercism.org/) is a platform for learning new programming languages. It has more than 65 tracks available. The Universal Test Runner supports nearly all of them out of the box using the [Exercism CLI](https://exercism.org/docs/using/solving-exercises/working-locally)'s `exercism test` command. Just like this tool, it knows how to run each track's tests and invokes the correct one automatically.

Rather than re-implement all of the test commands `exercism` can handle, the runner will invoke the Exercism CLI when run from an exercise directory. This requires version `3.2.0` of the Exercism CLI installed.

> fun fact: I [added the test command](https://github.com/exercism/cli/pull/1092) after it was suggested in the [forum thread](https://forum.exercism.org/t/introducing-the-universal-test-runner/6228) where I announced the Universal Test Runner

## Motivation

I work in a few languages at a time, so I've actually had a [version of this in my dotfiles](https://github.com/xavdid/dotfiles/blob/6bd5f56b1f9ad2dcef9f8b72413d30779b378aef/node/aliases.zsh#L45-L73) for a while. Also, as I've been doing [Exercism's #12in23 program](https://exercism.org/challenges/12in23), I'm _really_ switching languages. It's nice not to have to re-learn any muscle memory. Plus, increasingly complex `bash` was holding me back.

### Design Philosophy

1. The runner itself should need no configuration - it Just Works
2. It should pass all arguments through to the underlying test command
3. It should have wide language and test runner support; please open an issue if your use case isn't supported!

## FAQ

### `just` errors when passing CLI args

If you run with args (like `t -k whatever`) and see an error from `just` like:

```
error: Justfile does not contain recipes `-k` or `whatever`.
```

That means your `test` recipe doesn't accept any options. Make sure it has an `*options` arg that you pass through to your test command:

```justfile
test *options:
    pytest {{options}}
```

## Development

This section is people making changes to this package.

To get set up, run `uv sync`.

This installs the package in `--edit` mode and makes its dependencies available. You can now run `uv run -- t` to run tests and `uv run -- universal-test-runner` to access help, version, and debugging info.

### Running Tests

Use the development version this package to run its own tests: `just dev`. Or, run `just _test` directly.

### Releasing New Versions

> these notes are mostly for myself (or other contributors)

1. run `just bump <major|minor|patch>` and add `CHANGELOG` entry
2. commit & push
3. Run `just release`
4. paste the stored API key (If you're getting invalid password, verify that `~/.pypirc` is empty)
