Metadata-Version: 2.4
Name: aiohttp-request-id-logging
Version: 1.0.0
Summary: Setup proper request id logging for your Aiohttp app
Maintainer-email: Petr Messner <petr.messner@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/messa/aiohttp-request-id-logging
Project-URL: Bug Reports, https://github.com/messa/aiohttp-request-id-logging/issues
Project-URL: Source, https://github.com/messa/aiohttp-request-id-logging
Keywords: aiohttp,logging
Classifier: Development Status :: 5 - Production/Stable
Classifier: Framework :: AsyncIO
Classifier: Framework :: aiohttp
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp
Dynamic: license-file

aiohttp-request-id-logging
==========================

Motivation
----------

When logging from an async application (e.g. aiohttp web application),
log messages from different async tasks, http requests etc. are intertwined and you cannot surely tell which line was generated by what request.
For example:

```
2020-01-15 15:35:37,501  INFO: Processing transfer id 1234
2020-01-15 15:35:37,976  INFO: Processing transfer id 5678
2020-01-15 15:35:38,201 ERROR: Oh no, something bad has happened! Cannot finish the transfer.
2020-01-15 15:35:38,504  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 200 165 "-" "curl/7.68.0"
2020-01-15 15:35:38,982  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 500 165 "-" "curl/7.68.0"
```

So, which transfer has failed? The one with id 1234, or the one with id 5678?

Of course, ideally the error message should contain also the transfer id.
But it is not always this case, or you may want to inspect also other log messages that were logged by that specific task/request.

When you start to use this library in your aiohttp web application, this is how your log messages will look like:

```
2020-01-15 15:58:47,238  INFO: [req:kbyLG8d] Processing GET / (__main__:hello)
2020-01-15 15:58:47,950  INFO: [req:xtMacpA] Processing GET / (__main__:hello)
2020-01-15 15:58:48,240  INFO: [req:kbyLG8d] Processing transfer id 1234
2020-01-15 15:58:48,953  INFO: [req:xtMacpA] Processing transfer id 5678
2020-01-15 15:58:49,182 ERROR: [req:xtMacpA] Oh no, something bad has happened! Cannot finish the transfer.
2020-01-15 15:58:49,242  INFO: [req:kbyLG8d] 127.0.0.1 "GET / HTTP/1.1" 200 165 "-" "curl/7.68.0"
2020-01-15 15:58:49,959  INFO: [req:xtMacpA] 127.0.0.1 "GET / HTTP/1.1" 500 165 "-" "curl/7.68.0"
```


Installation
------------

```shell
$ uv add https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
# or
$ python3 -m pip install https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
```

Or add this line to `requirements.txt`:

```
aiohttp-request-id-logging @ https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
```


Usage
-----

```python
from aiohttp.web import Application, Response, RouteTableDef, run_app
from logging import DEBUG, basicConfig
from aiohttp_request_id_logging import (
    setup_logging_request_id_prefix,
    RequestIdMiddleware,
    RequestIdAccessLogger,
)

routes = RouteTableDef()

@routes.get('/')
async def hello(request):
    return Response(text="Hello, world!\n")

basicConfig(
    level=DEBUG,
    format='%(asctime)s [%(threadName)s] %(name)-37s %(levelname)5s: %(requestIdPrefix)s%(message)s')

setup_logging_request_id_prefix()

app = Application(middlewares=[RequestIdMiddleware()])
app.router.add_routes(routes)

run_app(app, access_log_class=RequestIdAccessLogger)
```

For more complete example see [examples/demo.py](examples/demo.py).

Customization of the middleware behavior is demonstrated in
[`examples/demo_customization_injection.py`](examples/demo_customization_injection.py)
(constructor parameters) and
[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py)
(subclassing).


How it works
------------

This library helps you to add request (correlation) id to the log messages in a few steps:

1. **`RequestIdMiddleware`:** Generate random `request_id` for each aiohttp request and

   - store it in a ContextVar `aiohttp_request_id_logging.request_id`
   - store it also in the request storage: `request[aiohttp_request_id_logging.REQUEST_ID_KEY]`
     (and, by default, also under the plain string key `request['request_id']` for backward compatibility)
   - return it to the client in the `X-Request-Id` response header

2. **`setup_logging_request_id_prefix()`:** Modify logging record factory so that the `request_id` is attached to every logging record created

   - so you should modify your log format, for example `logging.basicConfig(format=... %(levelname)5s: %(requestIdPrefix)s%(message)s')`

3. Because the aiohttp access logging happens out of the middleware scope, the request id ContextVar would be already resetted. So **`RequestIdAccessLogger`** is provided that adds the request_id to the access log message.

4. Unhandled exceptions ("500 errors") are logged and converted to a 500 response directly by **`RequestIdMiddleware`** – if they were left to the standard aiohttp error handling, the traceback would be logged outside of the middleware scope, without the request id (see the [Reference](#requestidmiddleware) for details).

5. If you use **[Sentry](https://docs.sentry.io/platforms/python/aiohttp/)**, a `request_id` [tag](https://docs.sentry.io/enriching-error-data/context/?platform=python#tagging-events) is added when the request is processed.

Sentry integration will be active only if you have `sentry_sdk` installed.


Reference
---------

All names are importable directly from the `aiohttp_request_id_logging` package.

### `RequestIdMiddleware`

The aiohttp middleware. Generates a request id for every request, stores it
in the `request_id` ContextVar and in the request, adds an `X-Request-Id`
response header, and (if Sentry is installed) creates a Sentry scope with
a `request_id` tag.

Exceptions raised from the handler are processed by the middleware too:

- an `HTTPException` (the aiohttp way of returning error responses from
  anywhere in the handler) is re-raised for aiohttp to process, with the
  request id response header added to it first
- any other unhandled exception is logged together with its traceback and
  converted to a plain text `500 Internal Server Error` response (also
  carrying the request id header) right in the middleware

The latter is deliberate: if the exception were left to the standard aiohttp
error handling, it would be logged outside of the middleware scope, where the
`request_id` ContextVar is already reset (and the Sentry scope with the
`request_id` tag closed) – so the traceback, usually the log message you need
to pair with a request the most, would be the one line without the request id.
Override `get_response_for_exception(request, exc)` to customize the error
response (a nicer error page, JSON for API paths...).

Constructor parameters (all keyword-only):

- `request_id_factory` – a zero-argument callable returning the request id string,
  used when `get_request_id` returns `None` (which the default implementation
  always does); default: `random_request_id_factory`
- `get_request_id` – a callable `(request)` returning the request id for the
  given request, or `None` to have one generated with `request_id_factory`;
  if you adopt an incoming header value here, validate it – it is controlled
  by the client
- `log_request_start` – a callable `(request, handler)` that logs the request
  start message, replacing the default `Processing GET / (...)` one;
  pass `noop` to disable the message
- `log_function_name` – include the handler name in the default request start
  message; default: `True`
- `add_response_request_id_header` – a callable `(response, req_id)` that adds
  the request id header to the response, replacing the default behavior
  (which keeps a header already set by the handler); pass `noop` to disable
  the header
- `request_id_header_name` – name of the response header with the request id;
  default: `X-Request-Id`
- `no_fallback_request_id_key` – if `True`, the request id is stored in the request
  only under `REQUEST_ID_KEY`, not also under the backward compatibility plain
  string key `request['request_id']`; default: `False`

The behavior can also be customized by subclassing – overriding the class
attributes (`request_id_header_name`, `log_function_name`) or the methods:
`get_request_id`, `before_request`, `after_request`,
`get_response_for_exception`, `log_request_start`, `set_request_keys`,
`setup_sentry_scope`, `add_response_request_id_header`, `get_function_name`.

Both approaches are demonstrated in
[`examples/demo_customization_injection.py`](examples/demo_customization_injection.py) and
[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py).

Functions stored in class attributes are tricky (Python would bind them
as methods), that is why callables like `request_id_factory` are passed
via the constructor parameters instead.

The `get_request_id`, `log_request_start` and `add_response_request_id_header`
parameters take precedence over the methods of the same name – when the
parameter is passed, the method (even one overridden in a subclass) is not
called.

The middleware does not adopt a request id sent by the client – how (and
whether) to trust such a value depends on the deployment. If you want that,
pass (or override) `get_request_id(request)` and validate the incoming value
there; see
[`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py).

`request_id_middleware()` is a backward compatibility wrapper of this class;
unlike the constructor, its `log_request_start` parameter is a bool –
`log_request_start=False` translates to `log_request_start=noop`.

### `setup_logging_request_id_prefix()`

Wraps the logging record factory so that every log record gets two extra
attributes usable in the log format:

- `%(requestIdPrefix)s` – `"[req:kbyLG8d] "`, or an empty string outside of a request
- `%(request_id)s` – the raw request id, or `None`

The prefix can be customized with the `prefix_format` parameter
(default: `"[req:{request_id}] "`).

Safe to call multiple times (does the setup only once).

### `RequestIdAccessLogger`

Subclass of `aiohttp.web_log.AccessLogger` that sets the `request_id`
ContextVar while writing the access log line. Needed because aiohttp writes
the access log outside of the middleware scope. Pass it to
`run_app(app, access_log_class=RequestIdAccessLogger)`.

### `request_id`

`ContextVar` holding the request id of the currently processed request.
Read it with `request_id.get()` – it returns `None` outside of a request.

### `REQUEST_ID_KEY`

Key under which the request id is stored in the request:
`request[REQUEST_ID_KEY]`. It is a `web.RequestKey` instance on aiohttp
versions that support it, otherwise the plain string `'request_id'`.
(The plain string key `request['request_id']` is set as well by default,
for backward compatibility; pass `RequestIdMiddleware(no_fallback_request_id_key=True)`
to store the id only under `REQUEST_ID_KEY`.)

`FALLBACK_REQUEST_ID_KEY` is that backward compatibility plain string key –
`'request_id'`, or `None` on aiohttp versions without `web.RequestKey`
(there `REQUEST_ID_KEY` itself is the plain string, so there is no fallback).

### Request id factories

- `random_request_id_factory(length=7)` – the default; returns a random URL-safe
  string of the given length, avoiding visually ambiguous characters
  (`1`/`l`/`I`, `2`/`Z`, `O`/`0`)
  (`generate_request_id` is its backward compatibility alias)
- `sequential_request_id_factory` / `SequentialRequestIdFactory` – alternative
  factory producing ids like `Wxyz0000`, `Wxyz0001`… – a random per-process
  prefix followed by a sequential number

Usage: `RequestIdMiddleware(request_id_factory=sequential_request_id_factory)`

Caveat: if the request ids are ever exposed to clients (response header,
error page…), sequential ids reveal how many requests the server processes
and how many server processes there are. Note that the middleware sends the
request id in the `X-Request-Id` response header by default – pass
`add_response_request_id_header=noop` to disable that. If the exposure is
a concern, stick with the default random factory.

### `noop`

A do-nothing function. Pass it as the `log_request_start` or
`add_response_request_id_header` constructor parameter to disable
the corresponding default behavior.

### `RequestIdKeyAlreadySetError`

Raised by the middleware when the request already contains a request id –
most likely the middleware is applied twice, or something else sets it too.
The existing id is available as the `existing_request_id` attribute.


Development
-----------

The project uses [uv](https://docs.astral.sh/uv/) for dependency and environment
management, [Ruff](https://docs.astral.sh/ruff/) for linting and formatting
and [ty](https://docs.astral.sh/ty/) for type checking.

```shell
$ make check      # lint + ty check + tests (pytest)
$ make lint       # ruff check + ruff format --check
$ make lint-fix   # apply ruff fixes and formatting
$ make typecheck  # ty check
```

The Makefile targets use `uv run`, which automatically creates the `.venv`
virtualenv and installs the dependencies (including the `dev` dependency group)
on the first run.


Version changelog
-----------------

### 1.0.0 (2026-07-16)

- Added `aiohttp_request_id_logging.__version__`
- `RequestIdContextAccessLogger` was renamed to `RequestIdAccessLogger`;
  the old name still works as a backward compatibility alias
- The middleware was refactored into a class `RequestIdMiddleware` so that it can be
  subclassed and individual parts of the behavior customized by overriding class
  attributes and methods (`before_request`, `after_request`,
  `log_request_start`, `set_request_keys`, `setup_sentry_scope`,
  `add_response_request_id_header`...)
  - `request_id_middleware()` is kept as a backward compatibility wrapper function
  - the deprecated-in-aiohttp-4 `@web.middleware` decorator is no longer used
  - see the new examples
    [`examples/demo_customization_injection.py`](examples/demo_customization_injection.py) and
    [`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py)
- The middleware now returns the request id to the client in the `X-Request-Id`
  response header (a header already set by the handler is not overwritten);
  the header name can be changed with the `request_id_header_name`
  parameter, or the header disabled completely with `add_response_request_id_header=noop`
- New hook `get_request_id(request)` (a constructor parameter or an overridable
  method) – returns the request id for the given request, e.g. adopted from
  an incoming header, or `None` to have a new id generated with
  `request_id_factory` (if you adopt an incoming value, validate it – see
  [`examples/demo_customization_subclassing.py`](examples/demo_customization_subclassing.py))
- The response request id header is silently skipped for responses that
  were already prepared (streaming/WebSocket handlers)
- The request start message ("Processing ...") can be disabled
  (`request_id_middleware(log_request_start=False)`,
  `RequestIdMiddleware(log_request_start=noop)`) or replaced with a custom callable
  (`RequestIdMiddleware(log_request_start=my_log_function)`)
- The middleware now raises `RequestIdKeyAlreadySetError` when the request
  already contains a request id, for example when the middleware is applied twice
  or something else also sets the request id
- New parameter `RequestIdMiddleware(no_fallback_request_id_key=True)` stores the
  request id only under `REQUEST_ID_KEY`, skipping the backward compatibility
  plain string key `request['request_id']` (the plain string key is now also
  exported as `FALLBACK_REQUEST_ID_KEY`)
- Breaking: all `request_id_middleware()` parameters are now keyword-only; positional calls
  like `request_id_middleware(my_factory)` raise `TypeError` – use
  `request_id_middleware(request_id_factory=my_factory)` instead
- `HTTPException` raised from a handler is re-raised as before, but the request id
  response header is now added to it first (the exception is also the response
  aiohttp sends to the client)
- The 500 response for an unhandled exception from the handler can be customized
  by overriding the `get_response_for_exception(request, exc)` method
- New parameter `setup_logging_request_id_prefix(prefix_format=...)` customizes the log
  record prefix (default: `"[req:{request_id}] "`)
- The `request_id` ContextVar now has a default of `None`, so it can be read
  with just `request_id.get()` – no more `LookupError` (or the `request_id.get(None)`
  workaround) outside of a request
- Breaking: `random_request_id_factory()` has a new `length` parameter, replacing the
  removed module-level variables `request_id_default_length` and
  `default_request_id_factory`; the module-level function `get_function_name`
  was also removed (it is now a `RequestIdMiddleware` static method)
- Generated request ids now avoid visually ambiguous characters
  (`1`/`l`/`I`, `2`/`Z`, `O`/`0`)
- The package was split into more modules (`middleware.py`, `errors.py`, `context.py`,
  `logging_setup.py`, `request_id_factories.py`); everything is
  still importable directly from `aiohttp_request_id_logging`, which now also defines `__all__`
- Added type hints, together with the `py.typed` marker so that the types
  are visible to type checkers (mypy, pyright) in projects using this library
- Added example [examples/demo_legacy.py](examples/demo_legacy.py) demonstrating
  the backward compatible `request_id_middleware()` usage
- Tests: every example in [examples/](examples/) is now run and checked
  (the request id in the response header, the log line prefixes, clean stderr);
  added tests for the HTTPException and 500 error handling
- Development: the project is now managed with [uv](https://docs.astral.sh/uv/)
  (`uv.lock`, dev dependencies in the `[dependency-groups]` table, which replaced
  the `test` extra); linting was migrated from flake8 to
  [Ruff](https://docs.astral.sh/ruff/) and the codebase was reformatted
  with `ruff format` (see the new `make lint` and `make lint-fix` targets)
- Development: added type checking with [ty](https://docs.astral.sh/ty/)
  (`make typecheck`, also run in CI); `sentry-sdk` was added to the dev
  dependencies so that the Sentry integration is type checked too
- Packaging: modernized the pyproject metadata – SPDX license expression
  (`license = "MIT"`), added the Development Status, Framework and
  Typing :: Typed classifiers
- Github Actions: lint runs in a separate job; uv is used to install Python
  and the dependencies; the test matrix also includes the latest aiohttp
  release (unpinned) to catch regressions with new aiohttp versions early

### 0.0.8 (2026-07-14)

- Store the request id in the request under a `web.RequestKey` instance,
  exported as `aiohttp_request_id_logging.REQUEST_ID_KEY` – fixes `NotAppKeyWarning`
  emitted by aiohttp 3.13/3.14
  - the plain string key `request['request_id']` is still set for backward compatibility
- Moved `demo.py` to [examples/](examples/)
- Fixed pyproject metadata
- Tests: run the demo in Python dev mode and assert clean stderr
  (aiohttp-internal warnings are whitelisted)
- Github Actions: added Python 3.14, updated action versions

### 0.0.7 (2025-05-12)

- Support new scope handling (`isolation_scope`) in sentry_sdk 2.x
  and removed an incorrect upgrade warning
  ([#6](https://github.com/messa/aiohttp-request-id-logging/pull/6), thanks @martinhanzik)
- Github Actions: added Python 3.13

### 0.0.6 (2024-06-02)

- Added `sequential_request_id_factory` as an alternative request id generator
- Improved `demo.py` and added a test running it

### 0.0.5 (2024-05-02)

- `RequestIdContextAccessLogger` no longer fails when the request id is not set,
  for example when an error occurs in a middleware
- Github Actions: added Python 3.12

### 0.0.4 (2023-05-11)

- Fixed `request_id_default_length` handling: the generated request id is now
  exactly that many characters long (and the default was changed from 5 to 7)

### 0.0.3 (2023-05-11)

- Added `req:` to the log record prefix: `[x12y3]` → `[req:x12y3]`

### 0.0.2 (2023-05-11)

- Migrated packaging from `setup.py` to `pyproject.toml`
- Added flake8 lint configuration
- Github Actions: added Python 3.10 and 3.11, added aiohttp version matrix

### 0.0.1 (2021-05-18)

- Initial release: `request_id_middleware()`, `setup_logging_request_id_prefix()`,
  `RequestIdContextAccessLogger` and the Sentry `request_id` tag integration


Alternatives
------------

- [belvaio-request-id](https://pypi.org/project/belvaio-request-id/) – another
  request id middleware + access logger for aiohttp; it also adopts the id
  from the incoming `X-Request-Id` header and attaches it to log records via
  a logging filter (instead of the log record factory used here); last
  released in 2021 and its source repository is no longer available
- [asgi-correlation-id](https://github.com/snok/asgi-correlation-id) – the
  same idea for the ASGI world (FastAPI, Starlette, Django...; aiohttp is not
  ASGI, so it cannot be used here) – correlation id middleware, a logging
  filter and a Sentry integration
- [django-guid](https://github.com/snok/django-guid) – correlation id
  middleware and logging integration for Django
- [structlog contextvars](https://www.structlog.org/en/stable/contextvars.html) –
  if you log via structlog, you can bind a request id to the context-local
  context yourself (`structlog.contextvars.bind_contextvars(request_id=...)`)
  in a small custom middleware
- [aiotask-context](https://github.com/Skyscanner/aiotask-context) – stores
  contextual data (e.g. a request id) directly in the asyncio task and
  includes an aiohttp request id logging example; predates `contextvars`,
  its last release was in 2020 and the repository is now archived
- [OpenTelemetry Python](https://opentelemetry.io/docs/languages/python/) –
  the heavyweight solution: full distributed tracing with an aiohttp server
  instrumentation, and the logging instrumentation can optionally inject
  `otelTraceID`/`otelSpanID` into log records (using the same log record
  factory technique as this library)


Links
-----

In general, you should be familiar with Aiohttp documentation related to production deployment:

- https://docs.aiohttp.org/en/stable/web_advanced.html#deploying-behind-a-proxy
- https://docs.aiohttp.org/en/stable/deployment.html

This is where this started: https://stackoverflow.com/a/58801740/196206
