aiohttp-request-id-logging 1.0.0

Setup proper request id logging for your Aiohttp app

aiohttp-request-id-logging

Motivation

When logging from an async application (e.g. aiohttp web application), log messages from different async tasks, http requests etc. are intertwined and you cannot surely tell which line was generated by what request. For example:

2020-01-15 15:35:37,501  INFO: Processing transfer id 1234
2020-01-15 15:35:37,976  INFO: Processing transfer id 5678
2020-01-15 15:35:38,201 ERROR: Oh no, something bad has happened! Cannot finish the transfer.
2020-01-15 15:35:38,504  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 200 165 "-" "curl/7.68.0"
2020-01-15 15:35:38,982  INFO: 127.0.0.1 [15/Jan/2020:14:35:36 +0000] "POST / HTTP/1.1" 500 165 "-" "curl/7.68.0"

So, which transfer has failed? The one with id 1234, or the one with id 5678?

Of course, ideally the error message should contain also the transfer id. But it is not always this case, or you may want to inspect also other log messages that were logged by that specific task/request.

When you start to use this library in your aiohttp web application, this is how your log messages will look like:

2020-01-15 15:58:47,238  INFO: [req:kbyLG8d] Processing GET / (__main__:hello)
2020-01-15 15:58:47,950  INFO: [req:xtMacpA] Processing GET / (__main__:hello)
2020-01-15 15:58:48,240  INFO: [req:kbyLG8d] Processing transfer id 1234
2020-01-15 15:58:48,953  INFO: [req:xtMacpA] Processing transfer id 5678
2020-01-15 15:58:49,182 ERROR: [req:xtMacpA] Oh no, something bad has happened! Cannot finish the transfer.
2020-01-15 15:58:49,242  INFO: [req:kbyLG8d] 127.0.0.1 "GET / HTTP/1.1" 200 165 "-" "curl/7.68.0"
2020-01-15 15:58:49,959  INFO: [req:xtMacpA] 127.0.0.1 "GET / HTTP/1.1" 500 165 "-" "curl/7.68.0"

Installation

$ uv add https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip
# or
$ python3 -m pip install https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip

Or add this line to requirements.txt:

aiohttp-request-id-logging @ https://github.com/messa/aiohttp-request-id-logging/archive/v1.0.0.zip

Usage

from aiohttp.web import Application, Response, RouteTableDef, run_app
from logging import DEBUG, basicConfig
from aiohttp_request_id_logging import (
    setup_logging_request_id_prefix,
    RequestIdMiddleware,
    RequestIdAccessLogger,
)

routes = RouteTableDef()

@routes.get('/')
async def hello(request):
    return Response(text="Hello, world!\n")

basicConfig(
    level=DEBUG,
    format='%(asctime)s [%(threadName)s] %(name)-37s %(levelname)5s: %(requestIdPrefix)s%(message)s')

setup_logging_request_id_prefix()

app = Application(middlewares=[RequestIdMiddleware()])
app.router.add_routes(routes)

run_app(app, access_log_class=RequestIdAccessLogger)

For more complete example see examples/demo.py.

Customization of the middleware behavior is demonstrated in examples/demo_customization_injection.py (constructor parameters) and examples/demo_customization_subclassing.py (subclassing).

How it works

This library helps you to add request (correlation) id to the log messages in a few steps:

1. RequestIdMiddleware: Generate random request_id for each aiohttp request and

   • store it in a ContextVar aiohttp_request_id_logging.request_id
   • store it also in the request storage: request[aiohttp_request_id_logging.REQUEST_ID_KEY] (and, by default, also under the plain string key request['request_id'] for backward compatibility)
   • return it to the client in the X-Request-Id response header

2. setup_logging_request_id_prefix(): Modify logging record factory so that the request_id is attached to every logging record created

   • so you should modify your log format, for example logging.basicConfig(format=... %(levelname)5s: %(requestIdPrefix)s%(message)s')

3. Because the aiohttp access logging happens out of the middleware scope, the request id ContextVar would be already resetted. So RequestIdAccessLogger is provided that adds the request_id to the access log message.

4. Unhandled exceptions ("500 errors") are logged and converted to a 500 response directly by RequestIdMiddleware – if they were left to the standard aiohttp error handling, the traceback would be logged outside of the middleware scope, without the request id (see the Reference for details).

5. If you use Sentry, a request_id tag is added when the request is processed.

Sentry integration will be active only if you have sentry_sdk installed.

Reference

All names are importable directly from the aiohttp_request_id_logging package.

RequestIdMiddleware

The aiohttp middleware. Generates a request id for every request, stores it in the request_id ContextVar and in the request, adds an X-Request-Id response header, and (if Sentry is installed) creates a Sentry scope with a request_id tag.

Exceptions raised from the handler are processed by the middleware too:

• an HTTPException (the aiohttp way of returning error responses from anywhere in the handler) is re-raised for aiohttp to process, with the request id response header added to it first
• any other unhandled exception is logged together with its traceback and converted to a plain text 500 Internal Server Error response (also carrying the request id header) right in the middleware

The latter is deliberate: if the exception were left to the standard aiohttp error handling, it would be logged outside of the middleware scope, where the request_id ContextVar is already reset (and the Sentry scope with the request_id tag closed) – so the traceback, usually the log message you need to pair with a request the most, would be the one line without the request id. Override get_response_for_exception(request, exc) to customize the error response (a nicer error page, JSON for API paths...).

Constructor parameters (all keyword-only):

• request_id_factory – a zero-argument callable returning the request id string, used when get_request_id returns None (which the default implementation always does); default: random_request_id_factory
• get_request_id – a callable (request) returning the request id for the given request, or None to have one generated with request_id_factory; if you adopt an incoming header value here, validate it – it is controlled by the client
• log_request_start – a callable (request, handler) that logs the request start message, replacing the default Processing GET / (...) one; pass noop to disable the message
• log_function_name – include the handler name in the default request start message; default: True
• add_response_request_id_header – a callable (response, req_id) that adds the request id header to the response, replacing the default behavior (which keeps a header already set by the handler); pass noop to disable the header
• request_id_header_name – name of the response header with the request id; default: X-Request-Id
• no_fallback_request_id_key – if True, the request id is stored in the request only under REQUEST_ID_KEY, not also under the backward compatibility plain string key request['request_id']; default: False

The behavior can also be customized by subclassing – overriding the class attributes (request_id_header_name, log_function_name) or the methods: get_request_id, before_request, after_request, get_response_for_exception, log_request_start, set_request_keys, setup_sentry_scope, add_response_request_id_header, get_function_name.

Both approaches are demonstrated in examples/demo_customization_injection.py and examples/demo_customization_subclassing.py.

Functions stored in class attributes are tricky (Python would bind them as methods), that is why callables like request_id_factory are passed via the constructor parameters instead.

The get_request_id, log_request_start and add_response_request_id_header parameters take precedence over the methods of the same name – when the parameter is passed, the method (even one overridden in a subclass) is not called.

The middleware does not adopt a request id sent by the client – how (and whether) to trust such a value depends on the deployment. If you want that, pass (or override) get_request_id(request) and validate the incoming value there; see examples/demo_customization_subclassing.py.

request_id_middleware() is a backward compatibility wrapper of this class; unlike the constructor, its log_request_start parameter is a bool – log_request_start=False translates to log_request_start=noop.

setup_logging_request_id_prefix()

Wraps the logging record factory so that every log record gets two extra attributes usable in the log format:

• %(requestIdPrefix)s – "[req:kbyLG8d] ", or an empty string outside of a request
• %(request_id)s – the raw request id, or None

The prefix can be customized with the prefix_format parameter (default: "[req:{request_id}] ").

Safe to call multiple times (does the setup only once).

RequestIdAccessLogger

Subclass of aiohttp.web_log.AccessLogger that sets the request_id ContextVar while writing the access log line. Needed because aiohttp writes the access log outside of the middleware scope. Pass it to run_app(app, access_log_class=RequestIdAccessLogger).

request_id

ContextVar holding the request id of the currently processed request. Read it with request_id.get() – it returns None outside of a request.

REQUEST_ID_KEY

Key under which the request id is stored in the request: request[REQUEST_ID_KEY]. It is a web.RequestKey instance on aiohttp versions that support it, otherwise the plain string 'request_id'. (The plain string key request['request_id'] is set as well by default, for backward compatibility; pass RequestIdMiddleware(no_fallback_request_id_key=True) to store the id only under REQUEST_ID_KEY.)

FALLBACK_REQUEST_ID_KEY is that backward compatibility plain string key – 'request_id', or None on aiohttp versions without web.RequestKey (there REQUEST_ID_KEY itself is the plain string, so there is no fallback).

Request id factories

• random_request_id_factory(length=7) – the default; returns a random URL-safe string of the given length, avoiding visually ambiguous characters (1/l/I, 2/Z, O/0) (generate_request_id is its backward compatibility alias)
• sequential_request_id_factory / SequentialRequestIdFactory – alternative factory producing ids like Wxyz0000, Wxyz0001… – a random per-process prefix followed by a sequential number

Usage: RequestIdMiddleware(request_id_factory=sequential_request_id_factory)

Caveat: if the request ids are ever exposed to clients (response header, error page…), sequential ids reveal how many requests the server processes and how many server processes there are. Note that the middleware sends the request id in the X-Request-Id response header by default – pass add_response_request_id_header=noop to disable that. If the exposure is a concern, stick with the default random factory.

noop

A do-nothing function. Pass it as the log_request_start or add_response_request_id_header constructor parameter to disable the corresponding default behavior.

RequestIdKeyAlreadySetError

Raised by the middleware when the request already contains a request id – most likely the middleware is applied twice, or something else sets it too. The existing id is available as the existing_request_id attribute.

Development

The project uses uv for dependency and environment management, Ruff for linting and formatting and ty for type checking.

$ make check      # lint + ty check + tests (pytest)
$ make lint       # ruff check + ruff format --check
$ make lint-fix   # apply ruff fixes and formatting
$ make typecheck  # ty check

The Makefile targets use uv run, which automatically creates the .venv virtualenv and installs the dependencies (including the dev dependency group) on the first run.

Version changelog

1.0.0 (2026-07-16)

• Added aiohttp_request_id_logging.__version__
• RequestIdContextAccessLogger was renamed to RequestIdAccessLogger; the old name still works as a backward compatibility alias
• The middleware was refactored into a class RequestIdMiddleware so that it can be subclassed and individual parts of the behavior customized by overriding class attributes and methods (before_request, after_request, log_request_start, set_request_keys, setup_sentry_scope, add_response_request_id_header...)
  • request_id_middleware() is kept as a backward compatibility wrapper function
  • the deprecated-in-aiohttp-4 @web.middleware decorator is no longer used
  • see the new examples examples/demo_customization_injection.py and examples/demo_customization_subclassing.py
• The middleware now returns the request id to the client in the X-Request-Id response header (a header already set by the handler is not overwritten); the header name can be changed with the request_id_header_name parameter, or the header disabled completely with add_response_request_id_header=noop
• New hook get_request_id(request) (a constructor parameter or an overridable method) – returns the request id for the given request, e.g. adopted from an incoming header, or None to have a new id generated with request_id_factory (if you adopt an incoming value, validate it – see examples/demo_customization_subclassing.py)
• The response request id header is silently skipped for responses that were already prepared (streaming/WebSocket handlers)
• The request start message ("Processing ...") can be disabled (request_id_middleware(log_request_start=False), RequestIdMiddleware(log_request_start=noop)) or replaced with a custom callable (RequestIdMiddleware(log_request_start=my_log_function))
• The middleware now raises RequestIdKeyAlreadySetError when the request already contains a request id, for example when the middleware is applied twice or something else also sets the request id
• New parameter RequestIdMiddleware(no_fallback_request_id_key=True) stores the request id only under REQUEST_ID_KEY, skipping the backward compatibility plain string key request['request_id'] (the plain string key is now also exported as FALLBACK_REQUEST_ID_KEY)
• Breaking: all request_id_middleware() parameters are now keyword-only; positional calls like request_id_middleware(my_factory) raise TypeError – use request_id_middleware(request_id_factory=my_factory) instead
• HTTPException raised from a handler is re-raised as before, but the request id response header is now added to it first (the exception is also the response aiohttp sends to the client)
• The 500 response for an unhandled exception from the handler can be customized by overriding the get_response_for_exception(request, exc) method
• New parameter setup_logging_request_id_prefix(prefix_format=...) customizes the log record prefix (default: "[req:{request_id}] ")
• The request_id ContextVar now has a default of None, so it can be read with just request_id.get() – no more LookupError (or the request_id.get(None) workaround) outside of a request
• Breaking: random_request_id_factory() has a new length parameter, replacing the removed module-level variables request_id_default_length and default_request_id_factory; the module-level function get_function_name was also removed (it is now a RequestIdMiddleware static method)
• Generated request ids now avoid visually ambiguous characters (1/l/I, 2/Z, O/0)
• The package was split into more modules (middleware.py, errors.py, context.py, logging_setup.py, request_id_factories.py); everything is still importable directly from aiohttp_request_id_logging, which now also defines __all__
• Added type hints, together with the py.typed marker so that the types are visible to type checkers (mypy, pyright) in projects using this library
• Added example examples/demo_legacy.py demonstrating the backward compatible request_id_middleware() usage
• Tests: every example in examples/ is now run and checked (the request id in the response header, the log line prefixes, clean stderr); added tests for the HTTPException and 500 error handling
• Development: the project is now managed with uv (uv.lock, dev dependencies in the [dependency-groups] table, which replaced the test extra); linting was migrated from flake8 to Ruff and the codebase was reformatted with ruff format (see the new make lint and make lint-fix targets)
• Development: added type checking with ty (make typecheck, also run in CI); sentry-sdk was added to the dev dependencies so that the Sentry integration is type checked too
• Packaging: modernized the pyproject metadata – SPDX license expression (license = "MIT"), added the Development Status, Framework and Typing :: Typed classifiers
• Github Actions: lint runs in a separate job; uv is used to install Python and the dependencies; the test matrix also includes the latest aiohttp release (unpinned) to catch regressions with new aiohttp versions early

0.0.8 (2026-07-14)

• Store the request id in the request under a web.RequestKey instance, exported as aiohttp_request_id_logging.REQUEST_ID_KEY – fixes NotAppKeyWarning emitted by aiohttp 3.13/3.14
  • the plain string key request['request_id'] is still set for backward compatibility
• Moved demo.py to examples/
• Fixed pyproject metadata
• Tests: run the demo in Python dev mode and assert clean stderr (aiohttp-internal warnings are whitelisted)
• Github Actions: added Python 3.14, updated action versions

0.0.7 (2025-05-12)

• Support new scope handling (isolation_scope) in sentry_sdk 2.x and removed an incorrect upgrade warning (#6, thanks @martinhanzik)
• Github Actions: added Python 3.13

0.0.6 (2024-06-02)

• Added sequential_request_id_factory as an alternative request id generator
• Improved demo.py and added a test running it

0.0.5 (2024-05-02)

• RequestIdContextAccessLogger no longer fails when the request id is not set, for example when an error occurs in a middleware
• Github Actions: added Python 3.12

0.0.4 (2023-05-11)

• Fixed request_id_default_length handling: the generated request id is now exactly that many characters long (and the default was changed from 5 to 7)

0.0.3 (2023-05-11)

• Added req: to the log record prefix: [x12y3] → [req:x12y3]

0.0.2 (2023-05-11)

• Migrated packaging from setup.py to pyproject.toml
• Added flake8 lint configuration
• Github Actions: added Python 3.10 and 3.11, added aiohttp version matrix

0.0.1 (2021-05-18)

• Initial release: request_id_middleware(), setup_logging_request_id_prefix(), RequestIdContextAccessLogger and the Sentry request_id tag integration

Alternatives

• belvaio-request-id – another request id middleware + access logger for aiohttp; it also adopts the id from the incoming X-Request-Id header and attaches it to log records via a logging filter (instead of the log record factory used here); last released in 2021 and its source repository is no longer available
• asgi-correlation-id – the same idea for the ASGI world (FastAPI, Starlette, Django...; aiohttp is not ASGI, so it cannot be used here) – correlation id middleware, a logging filter and a Sentry integration
• django-guid – correlation id middleware and logging integration for Django
• structlog contextvars – if you log via structlog, you can bind a request id to the context-local context yourself (structlog.contextvars.bind_contextvars(request_id=...)) in a small custom middleware
• aiotask-context – stores contextual data (e.g. a request id) directly in the asyncio task and includes an aiohttp request id logging example; predates contextvars, its last release was in 2020 and the repository is now archived
• OpenTelemetry Python – the heavyweight solution: full distributed tracing with an aiohttp server instrumentation, and the logging instrumentation can optionally inject otelTraceID/otelSpanID into log records (using the same log record factory technique as this library)

Links

In general, you should be familiar with Aiohttp documentation related to production deployment:

• https://docs.aiohttp.org/en/stable/web_advanced.html#deploying-behind-a-proxy
• https://docs.aiohttp.org/en/stable/deployment.html

This is where this started: https://stackoverflow.com/a/58801740/196206