auditry

A clean, framework-agnostic observability middleware for FastAPI and Quart that provides comprehensive request/response logging, correlation ID tracking, business event extraction, and sensitive data redaction.

Contents

• Installation
• Quick Start
• Configuration
• Request / Correlation IDs
• Correlation Propagation Beyond HTTP
• Tracing (OpenTelemetry)
  • Setup
  • TracingConfig
  • correlation_id vs trace_id
  • SQS and SNS
  • What Gets Scrubbed
  • Operator Guidance
  • Limitations
• Metrics (CloudWatch EMF)
• User Tracking
• Business Event Tagging (For Analytics)
• Log Output
• Sensitive Data Handling
• Best Practices
• Example Output: Success vs Failure
• Migration Guide: 0.2.x to 0.3.0
• Development
  • Local setup
  • Running tests
  • Linting and formatting
  • Continuous integration
• License
• Contributing
• Support

Installation

Requires Python 3.11 or later.

For FastAPI

pip install 'auditry[fastapi]'

For Quart

pip install 'auditry[quart]'

For tracing (OpenTelemetry)

pip install 'auditry[otel]'

See Tracing (OpenTelemetry).

For everything

pip install 'auditry[all]'

Quick Start

FastAPI

from fastapi import FastAPI
from auditry import configure_logging, ObservabilityConfig, get_logger
from auditry.fastapi import create_middleware

# Configure structured logging at startup.
# service/version/environment are stamped on every line — pass them here or set
# SERVICE_NAME / SERVICE_VERSION / ENVIRONMENT.
configure_logging(
    level="INFO",
    service="my-service",
    version="1.0.0",
    environment="prod",
)

app = FastAPI()

# Add observability middleware (single line!)
app = create_middleware(
    app,
    config=ObservabilityConfig(
        service_name="my-service",
        # Set these explicitly. False is the safe choice: request/response
        # bodies are user-supplied content that field-name redaction cannot
        # reliably scrub. Opt in deliberately, per service, only when you
        # know the bodies are safe to persist.
        log_request_body=False,
        log_response_body=False,
    ),
)

logger = get_logger(__name__)

@app.get("/")
async def root():
    logger.info("Hello World")
    return {"message": "Hello World"}

Quart

from quart import Quart
from auditry import configure_logging, ObservabilityConfig, get_logger
from auditry.quart import create_middleware

# Configure structured logging at startup
configure_logging(
    level="INFO",
    service="my-service",
    version="1.0.0",
    environment="prod",
)

app = Quart(__name__)

# Add observability middleware (single line!)
app = create_middleware(
    app,
    config=ObservabilityConfig(
        service_name="my-service",
        log_request_body=False,   # set explicitly; False is the safe choice
        log_response_body=False,
    ),
)

logger = get_logger(__name__)

@app.route("/")
async def root():
    logger.info("Hello World")
    return {"message": "Hello World"}

Configuration

Required Configuration

config = ObservabilityConfig(
    service_name="your-service-name",  # REQUIRED
)

Note: The service_name is required and must be provided. This ensures all services have meaningful names in logs rather than generic defaults.

Full Configuration Options

config = ObservabilityConfig(
    # REQUIRED: Service name for log filtering (no default)
    service_name="my-service-name",

    # Request ID header name (default: X-Request-ID)
    # Override if your org uses a different header
    correlation_id_header="X-Request-ID",

    # Maximum request/response body size to log (default: 10KB)
    payload_size_limit=10_240,

    # Additional sensitive field patterns to redact
    additional_redaction_patterns=["internal_id", "employee_ssn"],

    # Whether to log request headers (default: True)
    log_request_headers=True,

    # Whether to log response headers (default: False)
    log_response_headers=False,

    # Whether to log query parameters (default: True)
    log_query_params=True,

    # Whether to log request bodies. Bodies are user-supplied content that
    # field-name redaction cannot reliably scrub; set False unless you know
    # the bodies on every endpoint are safe to persist.
    log_request_body=False,

    # Whether to log response bodies (same caveat as request bodies)
    log_response_body=False,

    # OpenTelemetry tracing (requires auditry[otel]; from auditry import
    # TracingConfig). See Tracing (OpenTelemetry).
    tracing=TracingConfig(
        instrument_server=False,     # start a server span per request
        query_param_allowlist=[],    # query values allowed on exported spans
        db_statement="sanitize",     # "sanitize" | "drop" | "keep"
        keep_client_address=False,   # client IP/port on server spans
    ),
)

# For FastAPI:
from auditry.fastapi import create_middleware
app = create_middleware(app, config)

# For Quart:
from auditry.quart import create_middleware
app = create_middleware(app, config)

Request / Correlation IDs

Request IDs are automatically handled:

• Incoming requests: Extracts from X-Request-ID header (or your custom header)
• Generated if missing: Creates a new UUID if no request ID provided
• Added to response: Returns the request ID in the response header
• Included in logs: Automatically included in all structured logs

One nuance: correlation_id is an optional field of the root schema. It is present whenever a context is bound — every line inside a request, and every line inside a worker task that binds one — and absent on lines logged outside any context (import time, startup hooks). Consumers should treat the field as optional rather than assuming it on every line; auditry deliberately does not invent a per-line fallback ID, because each line would get a different ID, which falsely implies correlation.

trace_id and span_id are optional in the same way: present only on lines written inside a valid OpenTelemetry span context (see Tracing (OpenTelemetry)), absent otherwise.

Using Correlation IDs in Your Code

from auditry import get_logger, get_correlation_id

logger = get_logger(__name__)

@app.get("/users/{user_id}")
async def get_user(user_id: str):
    # Correlation ID is automatically available
    correlation_id = get_correlation_id()

    # All logs automatically include the correlation ID
    logger.info(f"Fetching user {user_id}")

    return {"user_id": user_id, "correlation_id": correlation_id}

Propagating to Downstream Services

outbound_headers() builds the headers for you, generating an ID if none is bound yet so an outbound call is never made without one:

import httpx
from auditry import outbound_headers

@app.get("/proxy")
async def proxy_request():
    async with httpx.AsyncClient() as client:
        response = await client.get(
            "https://downstream-service.com/api/data",
            headers=outbound_headers(),          # {"X-Request-ID": "<id>"}, plus traceparent in a span
        )

    return response.json()

The header name defaults to your configured correlation_id_header (seeded when create_middleware runs), so a service that customizes the inbound header sends the same name outbound. Pass extra= to merge with headers you were already sending, and header_name= to override the name for one call. Attach these to third-party API calls too, wherever the SDK accepts custom headers — it makes the vendor's audit trail line up with yours. Inside a span they also carry traceparent and tracestate, so the vendor receives your trace and span IDs (random, no request content); outbound_headers(trace_context=False) sends only the correlation ID.

Correlation Propagation Beyond HTTP

The middleware binds the correlation ID for HTTP requests. Everything outside that request cycle — queue workers, schedulers, cron jobs, scripts — has to bind one itself, before the first log line, or those logs correlate with nothing.

Workers and Background Jobs

from auditry import configure_logging, get_logger, with_correlation

# Workers too, not just the API. version falls back to SERVICE_VERSION if unset.
configure_logging(service="my-worker", version="1.4.2", environment="prod")
logger = get_logger(__name__)

@with_correlation
async def process_job(ctx, job_spec, correlation_id=None):
    # The ID is bound before this body runs, so every line below carries it.
    logger.info("job started", job_type=job_spec["type"])

The decorator takes the ID from a correlation_id keyword argument when the producer passes one, and generates a fresh UUID4 otherwise. It leaves the kwarg in place if your function accepts it and strips it if not, so you can decorate functions that never declared it.

If a decorated handler runs while an ID is already bound in the context (the consumer loop wrapped the message in bound_message_context), it continues that trace rather than starting a new one — an ID in flight is propagated, never replaced.

When the unit of work isn't a function, bound_correlation_id() does the same thing as a context manager — binds for the block, restores the previous context after, so a long-lived worker never logs a finished job's ID against the next:

from auditry import bound_correlation_id, extract_correlation_id

for message in receive():
    with bound_correlation_id(extract_correlation_id(message)):
        logger.info("processing")        # carries this message's ID
# nothing after the block does

bind_correlation_id(value) is the sticky form — it binds and leaves it bound for the rest of the context's life. Use it at a true edge (a script, a one-shot process, a startup hook), not per job in a long-lived worker. ensure_correlation_id() returns the current ID, binding a fresh one if unset.

Queue Hops (SQS / SNS)

from auditry import bind_from_sqs_message, bound_message_context, sqs_message_attributes

# Producer
sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=json.dumps(job),
    MessageAttributes=sqs_message_attributes(),
)

# Consumer in a long-lived loop: binds this message's ID for the block,
# then restores the previous context
for message in response["Messages"]:
    with bound_message_context(message):
        logger.info("processing message")

# A true edge (a one-shot process): call this BEFORE the first log line.
# The binding is sticky.
bind_from_sqs_message(message)

Both fall back to a fresh ID when the attribute is absent, so a consumer never logs without one. Whenever the OpenTelemetry API is importable (with the otel extra, or through Langfuse), the same helpers also carry the W3C trace context; see SQS and SNS.

IDs that auditry generates are always random UUID4s — opaque, never derived from user data. Inbound IDs are propagated verbatim (that is the point of propagation), so the randomness guarantee holds end-to-end only when the edge service generated the ID. Don't accept correlation IDs from untrusted callers into systems that assume opacity.

Tracing (OpenTelemetry)

With the otel extra (pip install 'auditry[otel]'), auditry ties OpenTelemetry traces to its JSON logs, and its exporters keep sensitive data out of the spans they export:

• Log fields. Every log line written inside a valid span context carries trace_id and span_id (lowercase hex), so a log line opens its trace.
• Propagation. outbound_headers() adds traceparent (and tracestate) inside a span; sqs_message_attributes() adds them as message attributes. Consumers read them back with bound_message_context(). auditry uses W3C Trace Context only, whatever OTEL_PROPAGATORS says, and never sends baggage.
• correlation_id on server spans. The middleware records the request's correlation ID as the span attribute correlation_id, on spans from auditry's own server instrumentation or from yours.
• Scrubbing. ScrubbingSpanExporter wraps your exporter and, before export, removes credentials, PII, exception messages, stack traces and the customer content its rules recognize, whichever instrumentation created the span. Only spans exported through it are scrubbed: installing the extra does not wrap an exporter for you. See What Gets Scrubbed and Limitations.
• Opt-in server spans for FastAPI and Quart, named by route and honoring excluded_paths.

Trace IDs on log lines and trace context propagation turn on whenever the OpenTelemetry API (opentelemetry-api) is importable, not only with the extra: Langfuse, for one, depends on it, so a service using Langfuse gets them inside every Langfuse span. Without the API, all of this is a no-op and the rest of auditry works as before.

Setup

auditry does not create a TracerProvider; set one up and wrap its exporter:

from fastapi import FastAPI
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

from auditry import ObservabilityConfig, TracingConfig
from auditry.fastapi import create_middleware
from auditry.otel import ScrubbingSpanExporter

provider = TracerProvider(resource=Resource.create({"service.name": "my-service"}))
provider.add_span_processor(BatchSpanProcessor(ScrubbingSpanExporter(OTLPSpanExporter())))
trace.set_tracer_provider(provider)

app = FastAPI()
app = create_middleware(
    app,
    ObservabilityConfig(
        service_name="my-service",
        log_request_body=False,
        log_response_body=False,
        tracing=TracingConfig(instrument_server=True),
    ),
)

ScrubbingSpanExporter(inner, policy=None) takes any SpanExporter. Without a policy, it reads the one create_middleware derives from ObservabilityConfig at export time, so the order of the two calls does not matter.

Zero-code setup. The package registers two exporters with the SDK, so opentelemetry-instrument can select them by name:

pip install opentelemetry-distro
OTEL_TRACES_EXPORTER=auditry_otlp \
OTEL_EXPORTER_OTLP_ENDPOINT=https://collector.example.com:4318 \
OTEL_LOGS_EXPORTER=none \
OTEL_METRICS_EXPORTER=none \
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=fastapi \
opentelemetry-instrument uvicorn app:app

OTEL_METRICS_EXPORTER=none is required: opentelemetry-distro defaults the metrics exporter to OTLP over gRPC, which the otel extra does not install, and without it auto-initialization fails, so the service runs with no spans.

auditry_otlp is OTLP over HTTP, configured by the usual OTEL_EXPORTER_OTLP_* variables. auditry_console prints scrubbed spans to stdout for local development.

For a FastAPI service, OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=fastapi turns off the instrumentor zero-code setup would apply, so set instrument_server=True and auditry creates the server spans instead, skipping excluded_paths exactly as request logging does. auditry's spans ignore OTEL_PYTHON_FASTAPI_EXCLUDED_URLS and OTEL_PYTHON_EXCLUDED_URLS, and the instrumentor's should not rely on them: they search for each pattern anywhere in a URL built from the client's Host header, so a request sent with Host: health matches /health and gets no span, whatever its path. If zero-code setup does instrument FastAPI, auditry detects it and does not add a second server span. Scrubbing and the correlation_id attribute apply either way.

Server spans. tracing.instrument_server=True starts one server span per request, without per-receive/send child spans, so a streaming response is one span:

• FastAPI: FastAPIInstrumentor, named GET /files/{name} by route template.
• Quart: there is no official Quart instrumentation, so auditry wraps the app in OpenTelemetry's ASGI middleware (outermost) and renames each span, websockets included, to its route rule, GET /files/<name>.

Paths in excluded_paths get no span, and still get the request-ID header. Spans and request logging skip exactly the same requests, method-specific patterns included: both match the request's path inside the app (without any ASGI root_path) and its method, never a URL built from the client's Host header.

TracingConfig

Field	Default	Effect
instrument_server	False	Start a server span per request (see above). Leave off when the service instruments itself.
query_param_allowlist	[]	Query parameters whose values may be exported. Every other value is [REDACTED], and so is an allowlisted name that matches a redaction pattern.
db_statement	"sanitize"	db.statement / db.query.text: "sanitize" replaces string (including PostgreSQL dollar-quoted) and numeric literals with ?, "drop" removes the attribute, "keep" exports it unchanged.
keep_client_address	False	Keep the client IP address and port on server spans.

additional_redaction_patterns applies to spans too: to query parameter names and captured header names.

correlation_id vs trace_id

	correlation_id	trace_id
Identifies	One request chain, as a single flat ID	One trace: a tree of timed spans. span_id on a log line says which operation wrote it.
Created by	asgi-correlation-id, when the inbound header is missing or not a valid UUID4	The OTel SDK, when there is no valid inbound traceparent
Randomness	uuid4(), from os.urandom	random.getrandbits(128), not cryptographic. Never use it as a secret.
Format	32 hex characters	32 hex characters. They look identical, so never put one in the other's field.
Header	X-Request-ID (configurable)	traceparent: 00-<trace_id>-<parent span_id>-<flags>, plus tracestate
Returned to the client	Yes, in the response header	No
Inbound trust	Any caller's valid UUID4 is accepted	Any caller's traceparent is accepted, and under the default sampler its sampled flag forces or suppresses the span; see Operator Guidance
Present when	A middleware or propagation helper bound it	An SDK and instrumentation are installed. Unsampled requests still log a trace_id, but the backend has no trace for it.
Used for	Searching logs, support tickets	Opening the latency tree in the trace backend

SQS and SNS

from auditry import bound_message_context, sqs_message_attributes

# Producer, inside a span: correlation_id plus traceparent/tracestate
sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=json.dumps(job),
    MessageAttributes=sqs_message_attributes(),
)

# Consumer: binds the message's correlation ID and attaches its trace context
# for the block, then restores both
for message in response.get("Messages", []):
    with bound_message_context(message):
        logger.info("processing")   # carries the producer's trace_id

Both boto3 records (MessageAttributes / StringValue) and Lambda event records (messageAttributes / stringValue) are read. extract_trace_context(message) returns the context without attaching it. bind_from_sqs_message() attaches it too, and like its correlation binding, that is sticky until the next call: a message without trace context gets the consumer's own context, not the previous message's.

SQS allows 10 message attributes. When the trace attributes would push a message past that, they are skipped with one warning per process (the correlation ID is still added); in strict mode MessageAttributeLimitError is raised. Attributes you pass in existing are never overwritten.

What Gets Scrubbed

Traces are a general-access destination, so the rules aim to let only system-generated operational values (timings, status codes, route templates, random IDs, error types) survive. They work from attribute names and value shapes, so content they cannot recognize, such as a file name in a client span's URL path, is exported (see Limitations). The exporter applies these rules to span, event and link attributes; resource attributes are not touched.

Rule	What happens
Exception events	exception.message and exception.stacktrace are removed; exception.type and exception.escaped stay.
Status description	Replaced by the first exception event's exception.type as scrubbed (email addresses redacted), or removed. The status code stays.
User identity	Every enduser.* and user.* attribute is removed (user_agent.* stays).
Database statements	Per db_statement (literals become ? by default). A statement whose literals cannot be paired reliably, such as one with a backslash-escaped quote, is removed. Query parameters (db.query.parameter.*, and db.statement.parameters from capture_parameters=True) are always removed.
GenAI content	Prompts, completions, input/output messages, system instructions, tool call arguments and results, tool definitions, retrieval query text and documents, and evaluation explanations are removed, as are gen_ai.* events. Model, usage, finish-reason and operation attributes stay.
Query strings	In url.full, http.url, url.query and http.target, every value not in query_param_allowlist becomes [REDACTED]; names stay. A query that does not parse (for example ?q3-deck.pdf) is redacted whole. Full URLs also lose user:password@ and any #fragment, and url.fragment is removed.
Server paths	On server spans with a route, the path in url.path, http.target, url.full and http.url becomes the route template.
Client address	On server spans, client.address, net.peer.ip and the other client address and port attributes are removed, unless keep_client_address is set.
Langfuse	The Langfuse SDK attaches to the global tracer provider, so its spans reach auditry's exporter. Every langfuse.* attribute is removed except observation type and level, model name, usage and cost details, prompt name and version, completion start time, environment, release and version. Langfuse itself still receives the full spans.
Captured headers	http.request.header.* / http.response.header.* values whose name matches a redaction pattern become ("[REDACTED]",).
Span names	Server spans are named METHOD route, or just METHOD without a route. Any name is cut at ?.
Email addresses	Any email address (including %40-encoded) left in a value or a span name becomes [REDACTED].

Scrubbing fails closed: a span whose scrubbing raises is dropped, never exported unscrubbed, and the rest of the batch is exported. One warning per exception type per exporter goes to the auditry.otel logger, without attribute values. In strict mode the error is raised instead.

Exception detail still reaches the handler registered with set_trace_handler. With trace_id on the error log line, a span can be matched to its traceback there.

Operator Guidance

• Use BatchSpanProcessor, so scrubbing runs on the export thread rather than the request thread.
• Set OTEL_LOGS_EXPORTER=none, and never set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true: GenAI prompt and response content goes out through the OpenTelemetry Logs API, which the span exporter never sees.
• Don't enable the process resource detector; it records command-line arguments.
• Header capture (OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_*) is opt-in; sensitive header values are still redacted when it is on.
• On public endpoints, any caller's traceparent is trusted. Under the default sampler (parentbased_always_on) its sampled flag decides whether the request gets a span at all: set, it forces one; cleared, it suppresses it. Its trace ID becomes the trace_id on the request's log lines either way. Strip traceparent and tracestate at the edge (load balancer or gateway) for outside callers. OTEL_TRACES_SAMPLER=always_on alone keeps every span but still logs the caller's trace ID; a ratio sampler decides from that same caller-chosen ID.
• With Langfuse, set up the tracer provider before constructing Langfuse(). When Langfuse runs first, it registers its own global provider, a later trace.set_tracer_provider() is ignored with only a warning, and auditry's exporter never sees a span. Langfuse(tracer_provider=...) keeps Langfuse on a provider of its own instead.

Limitations

• Unrouted requests (404s, raw ASGI apps) have no route template, so their spans keep the raw path, with email addresses redacted.
• Client span paths. Route templating applies only to server spans. The URL path of a client or other non-server span is exported unchanged apart from email addresses, so an outbound path that carries a file name or customer identifier (an S3 object key, a vendor API resource ID) reaches the trace backend. Query values are still redacted.
• Exception detail is available only through set_trace_handler, never on spans.
• Scrubbing covers only spans exported through auditry's exporters. A second span processor with a plain exporter, or anything sent through the OpenTelemetry Logs API, is not scrubbed. Vendor SDKs add such processors to the global provider: Langfuse's (which by default sends only LLM spans, to Langfuse) and, if you turn it on, Sentry's OTLP integration, which sends every span to Sentry.

Metrics (CloudWatch EMF)

MetricsLogger emits CloudWatch Embedded Metric Format: a metric is a structured JSON log line that CloudWatch extracts at ingestion. That means no AWS SDK dependency, no network call on the request path, and no credentials to manage — metrics ride the log driver you already have.

from auditry import MetricsLogger

metrics = MetricsLogger(namespace="MyOrg/MyService", service="my-service")

metrics.count("JobsSubmitted")                       # +1
metrics.count("JobsSubmitted", 5)                    # +5
metrics.timing("RenderLatency", 42.7)                # milliseconds
metrics.zero("RateLimited")                          # see below

Metrics Ride the Log Pipeline

Once configure_logging() has run, a metric record is emitted through the same pipeline as every other line — so alongside the EMF payload it carries the root schema (timestamp, service, version, environment) and the correlation_id of the request that produced it. CloudWatch ignores the extra keys when extracting the metric; you get to tie a latency spike back to the exact request in Logs Insights. Inside an OpenTelemetry span, the record also carries trace_id and span_id. The metrics logger's level is pinned to INFO, so a WARNING root level can't silently discard metrics. If logging was never configured (a script, a bare test), records fall back to a raw single-line write on stdout — EMF extraction never depends on logging setup.

Because metric and dimension values share one flat JSON object with the log fields, a metric or dimension may not be named after one: _aws, event, exc_info, message, level, timestamp, service, version, environment, correlation_id, trace_id or span_id. Such a record is dropped with a warning (MetricRecordError in strict mode).

Timing Dependency Calls

dependency_call() times a call and records success or failure, with the exception class as an ErrorType dimension:

with metrics.dependency_call("dynamodb", resource="jobs-table"):
    table.get_item(Key={"id": job_id})

That emits Latency (ms), Success, and Error under a Dependency (plus optional Resource) dimension. A success emits Error: 0 and a failure emits Success: 0, so neither series ever goes silent. The exception propagates unchanged — this measures, it doesn't swallow.

Why zero() Exists

Monitoring systems forget metrics that stop reporting. If you only emit RateLimited when rate limiting happens, then "no data" and "no problem" look identical, and an alarm on that metric can never fire reliably. Emitting an explicit zero on the healthy path keeps the series alive so "no data" alarms work.

Dimensions Are Guarded

Dimension names are validated against a forbidden-pattern list (userid, email, filename, prompt, message, and similar). A match means the record is dropped, not emitted — and one warning is logged per offending name, so a hot path with a bad dimension can't turn into a log storm:

metrics.count("Uploads", dimensions={"user_email": email})   # dropped + warned
metrics.count("Uploads", dimensions={"FileType": "pdf"})     # fine

Metrics stores are unencrypted, broadly readable, and not selectively erasable — you cannot delete one user's data out of a metric after the fact. That is why the guard exists. But instrumentation must never break the code it measures: a retry counter that 500s a request is a liability, not observation. So on the emit path a violation is a dropped metric and a warning, never an exception in your handler.

Two exceptions to that, both deliberate:

• default_dimensions are validated at construction and always raise (ForbiddenDimensionError). That runs at startup, not on the hot path, and failing fast there is cheap.
• Strict mode makes emit-path violations raise too — but only where auditry knows it is not production, and you don't set it per call site. See Strict Mode: Loud in Non-Production.

Dimension values: numbers and booleans are ordinary values and are coerced with str() — {"Attempt": 3} is a retry counter, not a privacy violation. Anything else (a dict, None) is a TypeError, kept distinct from ForbiddenDimensionError so "you passed the wrong type" never reads as "you leaked PII". Values must be short, single-line identifiers (≤128 chars); a free-text value is dropped as user content.

There is also a hard cap of 8 dimensions per record, since every distinct dimension set is a separately billable metric. Over the cap is dropped like any other violation (ValueError in strict mode).

Strict Mode: Loud in Non-Production

The production rule is absolute: instrumentation never fails the unit of work it measures. But a PII-named dimension that only ever produces a warning in a production log is a mistake you find late. So auditry has one process-wide strict mode, in which the same failures raise — a dropped metric becomes a ForbiddenDimensionError / TypeError / ValueError at the call site, and a broken trace handler re-raises instead of being swallowed.

You don't set it per call site. configure_logging() resolves it once, from the environment you already pass:

configure_logging(service="my-service", environment=env_stage)
#  "local", "dev", "sandbox", "plat-sandbox", "test", "ci", "staging" -> strict
#  "prod", "customer-prod", anything unrecognized, or unset          -> production-safe

Only a known non-production name turns strict on: local, local-*, dev, dev-*, development, sandbox, plat-sandbox, test, testing, ci, qa, staging, stage. Everything else — including a dedicated customer account whose stage name carries no hint, and a process that never set an environment — is treated as production. Deriving the other way round ("strict unless it says prod") is exactly how a customer account ends up with raising instrumentation.

Overrides, highest first: configure_logging(strict=True) (or False), then the AUDITRY_STRICT=1 (or 0) environment variable, then the derived value. MetricsLogger(strict=...) still overrides per instance, and auditry.is_strict() tells you what was resolved. A test suite that calls configure_logging(environment="test") gets strict mode for free — which is where you want a bad dimension to fail.

Rollup Dimension Sets

An error metric dimensioned by error type is unalarmable on its own: an alarm on the coarse series finds no data, and you cannot enumerate every exception class a dependency might raise. So dependency_call() records each error under both [Service, Dependency, ErrorType] and [Service, Dependency] — one record, no double counting within a set. Alarm on the coarse series, then use the fine one to see which error type drove it.

When resource= is passed, the fine set gains a Resource dimension but the rollup stays [Service, Dependency], and successes roll up there too — so the per-dependency alarm series always has data (Error: 0 between failures), no matter how the calls are scoped.

emit() exposes the same mechanism directly:

metrics.emit(
    {"Error": 1},
    dimensions={"Dependency": "s3", "ErrorType": "ClientError"},
    rollup_dimension_sets=[["Service", "Dependency"]],   # also record under [Service, Dependency]
)

Rollup sets are recorded exactly as given: default dimensions like Service join a rollup only when it names them. Rollup names must already be present on the record; otherwise the record is dropped with a warning (ValueError in strict mode).

Explicit Dimension Sets

Every record is recorded under its full dimension set, plus any rollups. For two independent breakdowns of the same event that is one set too many: a record carrying both Route and Tenant also creates [Service, Route, Tenant], one billable metric per route and tenant pair, which nobody asked for. dimension_sets= lists every set instead, and only those are recorded:

metrics.emit(
    {"Latency": 12.5, "Success": 1, "Error": 0},
    units={"Latency": "Milliseconds"},
    dimensions={"Route": "checkout", "Tenant": "t-1"},
    dimension_sets=[["Service", "Route"], ["Service", "Tenant"]],
)

That is still one log line, with each value on it once. CloudWatch creates one metric per listed set and counts the value once in each, so summing within a set is exact. A query that spans sets counts the record once per set: pin the set with SCHEMA("MyOrg/MyService", Service, Route) in Metrics Insights, or an exact dimension match in a dashboard.

Sets are recorded as given, and default dimensions join only the sets that name them. A set naming a dimension the record lacks, the same set listed twice (in any order), a set of more than 30 keys (EMF's per-set limit), an empty list, or dimension_sets combined with rollup_dimension_sets drops the record with a warning (ValueError in strict mode). An empty set, [], is valid and records the metrics without dimensions.

User Tracking

The middleware automatically extracts user IDs from your authentication system and includes them in logs.

FastAPI User Tracking

from fastapi import Request, Depends

async def get_current_user(request: Request):
    # Your authentication logic here
    user_id = "user-123"

    # Set user_id in request state for auditry to capture
    request.state.user_id = user_id
    # OR if you have a user object:
    # request.state.user = user_object  # Must have .id or .user_id attribute

    return user_id

@app.get("/protected")
async def protected_route(user_id: str = Depends(get_current_user)):
    return {"message": "Protected content"}

Quart User Tracking

from quart import request

@app.before_request
async def authenticate():
    # Your authentication logic here

    # Set user on request for auditry to capture
    request.current_user = AuthenticatedUser(id="user-123")
    # OR use g.user or g.user_id
    # from quart import g
    # g.user_id = "user-123"

The middleware automatically finds the user ID from these locations:

• FastAPI: request.state.user_id or request.state.user.id
• Quart: request.current_user.id, g.user.id, or g.user_id

Business Event Tagging (For Analytics)

Tag specific endpoints as "business events" to make analytics queries easier for your sales/product teams.

Configuration

Tag endpoints in your middleware config - zero code changes needed in your actual endpoints:

from auditry import ObservabilityConfig, BusinessEventConfig

config = ObservabilityConfig(
    service_name="my-service-name",

    # Define which endpoints to tag for analytics
    business_events={
        "POST /workflows": BusinessEventConfig(
            event_type="workflow.created",
            extract_from_request=["file_id"],  # Pull file_id from request body
            extract_from_response=["id"],       # Pull workflow id from response
        ),
        "DELETE /workflows/{workflow_id}": BusinessEventConfig(
            event_type="workflow.deleted",
            extract_from_path=["workflow_id"],    # Pull workflow_id from URL path
        ),
    },
)

# Apply to your framework
from auditry.fastapi import create_middleware  # or auditry.quart
app = create_middleware(app, config)

Log Output with Event Tags

Regular log (no tagging):

{
  "service": "my-service-name",
  "message": "Request completed: POST /workflows - Status: 201",
  "request": {...},
  "response": {...}
}

Tagged business event log:

{
  "service": "my-service-name",
  "message": "Request completed: POST /workflows - Status: 201",
  "event_type": "workflow.created",          // ← Filterable in log platform
  "business_context": {
    "file_id": "file_123",                   // ← From request body
    "id": "workflow_789"                    // ← From response body
  },
  "request": {...},
  "response": {...}
}

Supported Extract Locations

• extract_from_request: Fields from request JSON body
• extract_from_response: Fields from response JSON body
• extract_from_path: Parameters from URL path (e.g., /workflows/{workflow_id})

Log Output

All logs are structured JSON, ready for log aggregators:

{
  "timestamp": "2025-10-28T12:34:56.789012+00:00",
  "level": "INFO",
  "service": "my-service-name",
  "correlation_id": "550e8400-e29b-41d4-a716-446655440000",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "message": "Request completed: POST /workflows - Status: 201 - Duration: 45.23ms",
  "request": {
    "method": "POST",
    "path": "/workflows",
    "query_params": {},
    "headers": {"user-agent": "curl/7.64.1", "authorization": "[REDACTED]"},
    "body": {"name": "My Workflow", "password": "[REDACTED]"},
    "user_id": "user_12345"
  },
  "response": {
    "status_code": 201,
    "duration_ms": 45.23,
    "body": {"id": "workflow_789", "name": "My Workflow"}
  }
}

correlation_id, trace_id and span_id are optional fields: correlation_id appears whenever a context is bound, and trace_id / span_id only inside a valid OpenTelemetry span context.

Sensitive Data Handling

Automatic Redaction

Automatically redacts these sensitive field patterns in all logged request bodies, response bodies, headers, and query parameters:

• password / passwd
• token / jwt / bearer
• api_key / apikey / x-api-key
• access_key / private_key
• secret / credential
• authorization / proxy-authorization / x-auth
• cookie / set-cookie
• otp / one_time_password
• signature
• csrf / xsrf
• ssn / social_security_number
• credit_card / creditcard
• x-amz-security-token

Add custom patterns via configuration:

config = ObservabilityConfig(
    service_name="my-service-name",
    additional_redaction_patterns=["internal_token", "employee_id"],
)

Disabling Body Logging (Application-Wide)

For applications handling sensitive data, you can disable logging of request and/or response bodies across the entire application:

config = ObservabilityConfig(
    service_name="my-service-name",

    # Disable request body logging for all endpoints
    log_request_body=False,

    # Disable response body logging for all endpoints
    log_response_body=False,
)

When body logging is disabled, logs will show [BODY_LOGGING_DISABLED] instead of the actual content, while still logging metadata like headers, status codes, and timing information.

Set both flags explicitly. Bodies are user-supplied content, and field-name redaction cannot reliably scrub free text: a prompt, an uploaded document, or a generated completion has no field names to match on. False is the safe choice; an explicit True is a deliberate, per-service decision. Leaving either flag implicit raises a DeprecationWarning.

Exception Details

Error logs are the single most likely place for customer content to leak. Exception messages routinely interpolate exactly the input that caused the failure — a parse error quotes the document, a validation error quotes the field value, an SDK error quotes the payload. Redaction cannot help, because a traceback has no field names to match.

A failed request logs the exception class and the correlation ID, and nothing else:

{
  "level": "ERROR",
  "error_type": "ValueError",
  "correlation_id": "abc-123",
  "message": "Request failed: POST /workflows"
}

You debug by taking the correlation ID and searching your logs, rather than by reading the exception text off the error line.

Three escape hatches, in increasing order of exposure:

# 1. Route full exception details to a destination you control the access to.
import logging
import traceback

from auditry import set_trace_handler

# An error tracker gets the live exception object:
set_trace_handler(
    lambda error_type, exc_info, event_dict: sentry_sdk.capture_exception(exc_info[1])
)

# Or a dedicated logger with its OWN handler, shipping to an encrypted,
# access-controlled destination. propagate=False keeps it off the root
# handler — never write traces back to stdout; that defeats the point.
secure_logger = logging.getLogger("app.secure-traces")
secure_logger.propagate = False
secure_logger.addHandler(logging.FileHandler("/var/log/secure/traces.log"))

def route_to_secure_log(error_type, exc_info, event_dict):
    secure_logger.error(
        "%s correlation_id=%s\n%s",
        error_type,
        event_dict.get("correlation_id"),
        "".join(traceback.format_exception(*exc_info)),
    )

set_trace_handler(route_to_secure_log)

set_trace_handler is the seam that lets traces go somewhere access-controlled without auditry needing to know anything about that destination — a restricted-access log group, an error tracker, whatever you use. Pass None to remove the handler.

The handler receives (error_type, exc_info, event_dict): the exception class name, the live (type, value, traceback) tuple, and a snapshot of the line's fields under the root schema (message, correlation_id, service, …). Render the tuple to text yourself if you need text. If the handler raises, auditry swallows it and marks the line trace_handler_error: true rather than letting your logging path break the request — in production. In strict mode it re-raises, so a broken handler fails your tests instead of silently dropping traces.

This discipline applies to auditry's own records — the middleware's request/response lines and anything logged through get_logger(). Plain stdlib loggers (logging.getLogger(...) in your code or a vendor SDK) share the root schema but keep their tracebacks, in the JSON-escaped exception field: auditry does not delete the stack trace of every except block in your process.

# 2. Put exception messages back on the standard stream, per service.
config = ObservabilityConfig(
    service_name="my-service",
    log_exception_messages=True,   # default False
)

# 3. Local development only: inline the full traceback in the `exception`
#    field (JSON-escaped, so the stream line stays single-line and the value
#    stays a real, tool-parseable traceback). Read once at configure_logging().
#    AUDITRY_FULL_TRACEBACKS=true

Excluding Paths

Skip logging for specific endpoints like health checks or streaming:

# Basic exclusion
config = ObservabilityConfig(
    service_name="my-service-name",
    excluded_paths=[
        '/health',           # Exact match
        '/metrics',          # Exact match
        '/stream*',          # Wildcard - matches /stream, /streaming, /stream/events
        '/api/*/internal',   # Wildcard - matches /api/v1/internal, /api/v2/internal
        '/admin/',           # Prefix - matches /admin/* (trailing slash indicates prefix)
    ],
)

# Method-specific
config = ObservabilityConfig(
    service_name="my-service",
    excluded_paths={
        'GET': ['/health'],
        'POST': ['/webhook/*'],
        '*': ['/admin/*']  # all methods
    }
)

Useful for streaming endpoints that don't play nice with middleware:

from quart import Quart, Response, stream_with_context
from auditry import ObservabilityConfig
from auditry.quart import create_middleware

app = Quart(__name__)
config = ObservabilityConfig(
    service_name="stream-svc",
    excluded_paths=['/stream/*']  # skip these
)
app = create_middleware(app, config)

@app.route('/stream/data')
async def stream_data():
    # this won't be logged
    @stream_with_context
    async def generate():
        for i in range(100):
            yield f"data: {i}\n\n"
    return Response(generate(), mimetype='text/event-stream')

Note: Excluded paths still get correlation IDs but no logging.

Health Probes Are Excluded by Default

These paths are merged into excluded_paths automatically:

/health  /healthz  /livez  /live  /ready  /readyz  /api/health

A load balancer probing every task every 15–30 seconds dominates log volume in most deployed services, and those lines carry no information. Probes still receive the correlation-ID header — they just stop producing request/response log lines. Opt out with:

config = ObservabilityConfig(
    service_name="my-service",
    include_default_excluded_paths=False,
)

Log-derived request counts exclude probe traffic; size any low-log-volume alarm on real requests, not on probes.

Best Practices

1. Configure Logging Early

Call configure_logging() at application startup, before any other code:

from auditry import configure_logging

# First thing in your app
configure_logging(level="INFO")

app = FastAPI()
# ... rest of your app

2. Use Structured Logging

Prefer get_logger(__name__) over standard Python logging:

from auditry import get_logger

logger = get_logger(__name__)

# Good - structured key/value fields, queryable individually
logger.info("Processing payment", amount=100.50, currency="USD")

# Works, but flat - the line still carries the JSON root schema
# (timestamp, service, correlation ID), but the data is baked into
# the message string instead of being queryable fields
import logging
logging.info("Processing payment amount=%s", 100.50)

Plain-stdlib records — including those from libraries you don't control (uvicorn, boto3) — are rendered through the same processor chain, so every line on stdout is schema-carrying JSON either way. get_logger is about making your fields structured and queryable.

3. Propagate Correlation IDs

When calling downstream services, always pass the correlation ID — outbound_headers() builds the headers and guarantees an ID is present (binding a fresh one if none exists yet), so don't assemble them by hand:

from auditry import outbound_headers

response = await client.get(url, headers=outbound_headers())

# If your org uses a different header name, or you have headers already:
response = await client.get(
    url,
    headers=outbound_headers(header_name="X-Trace-ID", extra={"Accept": "application/json"}),
)

4. Customize for Your Organization

Match your org's conventions:

config = ObservabilityConfig(
    service_name="my-service-name",
    correlation_id_header="X-Trace-ID",  # If your org uses a different header
    additional_redaction_patterns=["ssn", "tax_id"],  # Your sensitive fields
)

Example Output: Success vs Failure

Successful Request

{
  "level": "INFO",
  "service": "my-service-name",
  "correlation_id": "abc-123",
  "message": "Request completed: POST /workflows - Status: 201 - Duration: 45ms",
  "request": {...},
  "response": {...}
}

Failed Request

{
  "level": "ERROR",
  "service": "my-service-name",
  "version": "1.4.2",
  "environment": "prod",
  "correlation_id": "abc-123",
  "message": "Request failed: POST /workflows",
  "request": {...},
  "error_type": "ValueError",
  "execution_duration_ms": 12.34
}

Note what is not there: no traceback, and no exception_message. See Exception Details for why, and for how to get them back when you need them.

Migration Guide: 0.2.x to 0.3.0

Breaking Changes

1. Default header changed from X-Correlation-ID to X-Request-ID

   All services should upgrade to 0.3.0 together so they consistently use X-Request-ID. If you need to do a phased rollout, you can temporarily pin the old header on already-upgraded services:

   config = ObservabilityConfig(
       service_name="my-service",
       correlation_id_header="X-Correlation-ID",  # Temporary: remove once all services are on 0.3.0
   )
   

New Features

• Streaming-friendly middleware — Safer handling of streaming responses so large bodies are not buffered for logging when not appropriate.
• excluded_paths — Skip request/response logging for configured paths (for example health checks or long-lived streams) while still attaching the request ID header.

Development

Three scripts cover the whole local workflow, and CI runs the same two gates you do. Each takes --help.

Local setup

./scripts/bootstrap.sh

This provisions everything on macOS: the Xcode command line tools, the CPython build dependencies, pyenv, the interpreter pinned in .python-version, uv, a .venv built from that interpreter, and every dependency resolved from uv.lock. Every step checks before it installs, so re-running only does the work that is actually missing.

Homebrew is the one prerequisite it will not install for you -- its installer is interactive and wants sudo. If brew is missing, the script stops and prints the command to install it.

./scripts/bootstrap.sh --no-shell-init   # do not touch ~/.zshrc
./scripts/bootstrap.sh --run-tests       # also run the suite at the end
AUDITRY_PYTHON_VERSION=3.11.9 ./scripts/bootstrap.sh

To use the environment directly afterwards, source .venv/bin/activate.

Running tests

./scripts/runtests.sh

Any arguments are passed through to pytest:

./scripts/runtests.sh -k redaction
./scripts/runtests.sh -x -vv tests/test_redaction.py

The suite imports auditry as an installed package, so the script verifies the interpreter and the install before pytest starts rather than letting a missing install surface as a collection error. It exits 3 if the environment is not usable, and otherwise passes pytest's own exit code through.

Linting and formatting

One script covers Python, shell and Markdown. Within each, lint runs before the format check, so a syntax-level problem is never reported as a formatting one. Markdown is the exception: pymarkdown fix applies the fixes it has, then pymarkdown scan reports whatever is left.

Language	Lint	Format
Python	ruff check	ruff format
Shell	shellcheck	shfmt
Markdown	pymarkdown scan	pymarkdown fix (rules with a fix only)

./scripts/lintme.sh             # apply autofixes, then format in place
./scripts/lintme.sh --check     # report only, writes nothing -- what CI runs
./scripts/lintme.sh --unsafe-fixes

Every linter comes out of .venv, installed from uv.lock like any other dependency — shellcheck and shfmt are binaries, but the shellcheck-py and shfmt-py wheels carry them, so there is one install path and one pinned version per tool rather than Homebrew locally and apt in CI. A missing linter exits 3 rather than being skipped: a check that did not run must never read as clean.

Ruff's configuration lives in pyproject.toml under [tool.ruff]. Markdown is excluded from its formatter, which rewrites Python code blocks inside .md files — the examples in this README are spaced and annotated by hand. Shell formatting is shfmt -i 4 -ci -bn, and shellcheck covers every *.sh in the repository, not only scripts/.

PyMarkdown's rules are modeled on markdownlint's (same MD0xx ids) and configured under [tool.pymarkdown]. Prose wraps at 80 columns; tables and code blocks are exempt. It lints every *.md outside .gitignore except CLAUDE.md.

Continuous integration

Two workflows run on every pull request, drafts included, and again when a draft is marked ready for review:

Check	Runs	Covers
Lint	./scripts/lintme.sh --check	ruff, shellcheck, shfmt, pymarkdown
Tests	./scripts/runtests.sh	pytest

Both install with uv sync --all-extras --locked, which fails if uv.lock has drifted from pyproject.toml — so re-lock with uv lock whenever you change a dependency. Tests runs once per supported Python (3.11 and 3.12), as pytest (3.11) and pytest (3.12).

License

MIT License - see LICENSE file for details.

Contributing

Contributions welcome! Please submit a Pull Request. See Development for setup, tests and linting; a pull request is expected to pass ./scripts/lintme.sh --check and ./scripts/runtests.sh.

Support

For issues and questions: GitHub Issues