Metadata-Version: 2.5
Name: pytest-twinport
Version: 0.2.0
Summary: Pytest plugin that leases free TCP/UDP ports without race conditions between parallel test runs
Project-URL: Repository, https://github.com/messa/pytest-twinport
Project-URL: Issues, https://github.com/messa/pytest-twinport/issues
Author: Petr Messner
License-Expression: MIT
License-File: LICENSE
Keywords: e2e,free port,port,pytest,unused port,xdist
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: pytest>=7
Description-Content-Type: text/markdown

# pytest-twinport

Pytest plugin that gives your tests a free TCP/UDP port, without collisions
between tests running in parallel.

The usual recipe, `bind(0)`, read the port number, close the socket, has a race
condition: between closing the socket and starting your server, anyone else can
get the same port. With many E2E tests running at once, this shows up as random
`Address already in use` failures.

`pytest-twinport` keeps the port reserved for the whole test, across processes
and across separate pytest runs. It needs no daemon and no lock files.

## Installation

```bash
pip install pytest-twinport
```

## Usage

Use the `twinport` fixture to get a port reserved until the end of the test:

```python
def test_server(twinport):
    server = start_server(port=twinport)
    ...
```

Use `twinport_factory` when a test needs more ports:

```python
def test_cluster(twinport_factory):
    ports = [twinport_factory() for _ in range(3)]
    ...
```

Use `lease_port()` for any other fixture scope, or outside of fixtures. Stop the
server inside the `with` block, so that it stops before the port is released:

```python
from pytest import fixture
from pytest_twinport import lease_port


@fixture(scope="session")
def server():
    with lease_port() as port:
        process = start_server(port=port)
        yield process
        process.stop()
```

`lease_port()` can also be used without `with`. Keep the returned object for as
long as you need the port, because a garbage collected lease is released:

```python
lease = lease_port()
start_server(port=lease.port)
...
lease.release()
```

A lease garbage collected before `release()` emits a `ResourceWarning`, as a
sign that the port was given up earlier than intended. Using a released lease in
`with` raises `RuntimeError`.

If your server can bind to port 0 and report the port it got, you don't need
this plugin. It is meant for servers that must be told their port up front, such
as external processes, Docker containers and services configured by a file.

## How it works

Ports are split into fixed pairs: an even _guard_ port and the odd port after it.

1. Pick a random pair and bind the guard port. `bind()` is atomic, so only one
   process can succeed.
2. Check that the odd port is free.
3. Give the odd port to the caller and keep the guard bound.

Other processes can't take the pair, because they fail on the guard. When the
process ends, even with `SIGKILL`, the kernel closes the guard. Nothing stale is
left behind.

The lock lives in the network namespace of the process that holds it, so it only
protects ports used in that same namespace. See the note on containers below.

Ports are only chosen outside the ephemeral range. This matters: outgoing
connections take their source port from that range at any moment, and they know
nothing about guards. Linux, macOS and FreeBSD report the range, Windows and
other systems use a fixed assumption (see [Platform support](#platform-support)).
On Linux, ports listed in `ip_local_reserved_ports` are skipped too.

## Guarantees

| Scenario | Protected? |
| --- | --- |
| Tests in one `pytest-xdist` run | Yes |
| Separate pytest runs in one network namespace | Yes |
| Pytest runs in separate network namespaces publishing ports to one host | No, see below |
| Outgoing connections from anything on the machine | Yes, leased ports are outside the ephemeral range |
| Server restarting during a test | Yes, nobody else gets the port |
| Crashed or killed pytest process | Yes, the kernel releases the port |
| Software using a fixed port in the same range | No, the port is only checked when it is leased |
| Port allocation by other tools | No, every allocator has to use guards |

The port is free on TCP and UDP, on IPv4 and IPv6, and the first bind works
without `SO_REUSEADDR`. A server that restarts needs `SO_REUSEADDR` as usual,
because its old connections can keep the port in `TIME_WAIT`. Every lease uses two
port numbers.

A process created with `fork()` without `exec()` inherits the guard, and the
port stays blocked until that process ends too. This can last for a whole test
session, but can't cause a collision.

Containers with their own network stack have their own ports, so tests running
inside them don't collide. That changes once they publish ports to a shared host,
for example when pytest runs in a container and starts other containers with
`docker run -p PORT:80` through a shared Docker socket. The guard is then in a
different namespace than the published port, so two such runs can get the same
port. To avoid this, run pytest with `--network host`, give every run its own
`TWINPORT_RANGE`, or let Docker choose the port with `-p 80` and read it with
`docker port`.

## Configuration

Ports are chosen from 15000-65535, minus 30000-32767 and the ephemeral range.
The range 30000-32767 holds the Kubernetes NodePorts. Their traffic is
redirected by packet filter rules without any socket bound to the port, so a
free port there may still not reach your server. To change the range:

```bash
export TWINPORT_RANGE=20000-30000
```

`lease_port(port_range=(20000, 30000))` does the same for a single lease. Both
bounds are inclusive, and the range must contain at least one even port and the
odd port after it, otherwise `ValueError` is raised. Ephemeral and reserved
ports are left out in both cases, 30000-32767 only from the default range.

If no pair can be leased, `NoFreePortError` is raised with the range in its
message. When every pair of the range is excluded, the message also lists the
excluded ports that overlap the range. This happens when the ephemeral range of
the system covers nearly all ports, and then the only fix is to make it smaller.

## Measurements

`scripts/stress.py` runs parallel processes that repeatedly get a port, wait up
to 2 ms like a starting server, then bind the port without `SO_REUSEADDR`. It
counts failed binds. These are the results of three runs. Each run used 16
processes, 8000 allocations in total and a range of 200 ports (Linux, 6 cores):

| Method | Collisions |
| --- | --- |
| `lease_port()` | 0, 0, 0 |
| Random port checked by bind and close | 15, 16, 15 |
| `bind(0)` and close | 1, 0, 0 |

`bind(0)` is not limited to the small range, so it collides less often in this
test. An additional run of `lease_port()` with 32 processes and 64000
allocations had 0 collisions too.

## Platform support

The tests pass on Linux, macOS and Windows. BSD systems have not been tried out.

There are two limits on Windows:

- A bind to a wildcard address does not conflict with a bind to a specific
  address. Ports are checked on wildcard and loopback addresses, so a service
  listening only on another address of the machine is not detected.
- The ephemeral range is not read from the system. Ports from 32768 up are left
  out, which covers the default range. If the range was changed with `netsh` to
  start lower, set `TWINPORT_RANGE` to stay out of it.

## Development

```bash
uv sync
uv run pytest
uv run ruff check
uv run ruff format
uv run mypy
uv run scripts/stress.py
```

## Releasing

Set the new version in `pyproject.toml`, move the changes from Unreleased in the
[changelog](#changelog) to a new section, and commit it. Publishing a GitHub
release then runs the tests, builds the package and uploads it to PyPI. The
release fails if its tag does not match the version:

```bash
gh release create v0.1.0 --generate-notes
```

## Related projects

These get a free port for a moment and don't reserve it, so they have the race
condition described above:

- [pytest-asyncio](https://pypi.org/project/pytest-asyncio/) has the
  `unused_tcp_port` and `unused_udp_port` fixtures and their factories, based on
  `bind(0)` and close.
- [anyio](https://pypi.org/project/anyio/) has the `free_tcp_port` and
  `free_udp_port` fixtures and their factories. They work the same way, and the
  TCP and UDP ports are checked on both IPv4 and IPv6.
- [pytest-unused-port](https://pypi.org/project/pytest-unused-port/) has an
  `unused_port` fixture based on `bind(0)` and close.
- [pytest-server-fixtures](https://pypi.org/project/pytest-server-fixtures/)
  picks a random port below the usual ephemeral range, on a random loopback
  address.

These coordinate between processes in other ways:

- [portpicker](https://pypi.org/project/portpicker/) by Google, which inspired
  this project, solves the race with an optional `portserver` daemon.
  `pytest-twinport` is not an integration of that library.
- [pytest-services](https://pypi.org/project/pytest-services/) has a
  `port_getter` fixture that keeps the list of used ports in a shared lock file.
- [pytest-postgresql](https://pypi.org/project/pytest-postgresql/) marks its
  port with a file in the temporary directory of the test run, which coordinates
  `pytest-xdist` workers.
- [portkeeper](https://pypi.org/project/portkeeper/) keeps reservations in a JSON
  registry protected by a file lock.

These solve neighboring problems:

- [port-for](https://pypi.org/project/port-for/) manages persistent port
  assignments. It avoids the ephemeral range as well.
- [ephemeral-port-reserve](https://pypi.org/project/ephemeral-port-reserve/)
  forces an ephemeral port into the `TIME_WAIT` state before releasing it.
- [portlock](https://pypi.org/project/portlock/) uses a bound port to make sure
  that only one instance of a script is running.
- [pytest-localserver](https://pypi.org/project/pytest-localserver/) provides
  test servers that bind to port 0 themselves, so they don't need a free port up
  front.

## Changelog

### 0.2.0 (2026-11-08)

- Using a released lease in `with` raises `RuntimeError`. It used to return the
  port with no protection.
- The default range leaves out 30000-32767, the Kubernetes NodePorts.
- A range without a whole pair raises `ValueError` instead of `NoFreePortError`,
  also when it comes from `TWINPORT_RANGE`.
- A `port_range` whose items are not integers raises `TypeError`, one with other
  than two items raises `ValueError`.
- Errors name the range and where it came from. When every pair is excluded,
  they also list the excluded ports that overlap the range.
- Pairs are tried from a random start with a random stride instead of a linear
  scan, so leases don't pile up in a crowded range.
- Systems other than Linux, macOS and FreeBSD don't try `/proc` or `sysctl` and
  use the fallback ephemeral range right away. A malformed file in `/proc` is
  treated like a missing one.
- An empty `port_range` raises `ValueError` instead of using the default range.
- System errors such as running out of file descriptors are raised as they are,
  instead of ending with a misleading `NoFreePortError`.
- On Windows, the guard uses `SO_EXCLUSIVEADDRUSE`, so that a socket with
  `SO_REUSEADDR` can't take it over.

### 0.1.0 (2026-11-07)

- Initial release.

## License

MIT
