baltimore-patapsco (Python)

Reusable Django runtime primitives for Baltimore civic applications.

INSTALLED_APPS += ["baltimore.patapsco"]

REST_FRAMEWORK = {
    "EXCEPTION_HANDLER": "baltimore.patapsco.api.exception_handler",
}

urlpatterns = [path("", include("baltimore.patapsco.urls"))]

Place RequestContextMiddleware early in MIDDLEWARE — after SecurityMiddleware (and WhiteNoise when present), before session, auth, and anything that can short-circuit a response — so every later middleware, view, and log line sees the bound request ID. The reference app shows the canonical order:

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "whitenoise.middleware.WhiteNoiseMiddleware",
    "baltimore.patapsco.observability.RequestContextMiddleware",
    # sessions, common, csrf, auth, messages, clickjacking...
]

An error that carries data, such as a 412 rebase, declares it, and the handler sends it in the canonical envelope with its headers:

from baltimore.patapsco.api import PublicAPIError

raise PublicAPIError(
    status=412,
    code="precondition_failed",
    message="This record changed since you opened it.",
    details={"current_etag": etag},
    headers={"ETag": etag},
)

The status must be a 4xx, and the code must match ERROR_CODE_PATTERN. The details must render as strict JSON, so NaN and Infinity are refused; they are kept as they will render, so a later change to the caller's own list or dict can't reach the response. The headers can't include one the platform sets, such as X-Request-ID or Set-Cookie. A mistake raises ValueError where the error is declared.

Idempotent uploads

baltimore.patapsco.idempotency is optional. Add it to INSTALLED_APPS (it brings one table) and decorate the DRF view methods a client may resend. A repeated Idempotency-Key from the same user then gets the first answer instead of a second record:

from baltimore.patapsco.idempotency import idempotent


class PhotoUploadView(APIView):
    @idempotent
    def post(self, request): ...

idempotency_settings(window=timedelta(hours=24)) sets the window, and manage.py patapsco_idempotency_cleanup deletes expired records. The SDK's upload queue sends the key. See the uploads guide.

Stable health and readiness views

Applications that mount or extend the operational views directly use the public facade:

from baltimore.patapsco.api import HealthView, ReadinessView

HealthView is process-only liveness. ReadinessView runs the established database check and returns its public-safe 200 or 503 status document. Both use no authentication classes and AllowAny, and both remain ordinary DRF APIView classes with as_view(). They declare no throttle classes, so app defaults cannot turn a dependency probe into a 429. Both stay outside ATOMIC_REQUESTS on every database, subclasses included, so an outage is answered by the view, not a 500. A database failure is reported once per worker as it begins, at error level with the exception; later failed probes log readiness.database_unavailable at warning level with only the exception type, and the first healthy probe logs readiness.database_recovered. A probe every few seconds during an outage therefore sends one Sentry event, not one per probe.

HealthView and DiagnosticsView render their errors in the Patapsco envelope their OpenAPI declares, whatever the app's REST_FRAMEWORK["EXCEPTION_HANDLER"] is. A subclass may override get_exception_handler() and then owns its schema.

An application may subclass either class, override get(), and call super().get(request) before composing app-owned public-safe checks. The parent response status, request ID, and database result remain the starting contract; the application owns an explicit serializer and OpenAPI annotation for any extended payload.

The previous baltimore.patapsco.api.views imports resolve to the same class objects for compatibility. DiagnosticsView remains available only from that module and through the packaged URL configuration; it is intentionally absent from baltimore.patapsco.api.__all__ because its staff/public policy is a separate operational surface. The settings-free package root exports neither view class.

api_error_response() and the DRF exception handler force every 5xx response to the public internal_error code, generic message, and no details. Application callers cannot opt a server failure out of that safety boundary. An unexpected exception also rolls back every active database transaction managed by Django's ATOMIC_REQUESTS, so a failed request cannot commit partial writes. Applications retain ownership of explicitly managed transactions.

Use build_logging_config() for the shared JSON/pretty log shape. Modules are incrementally adoptable; installing the package does not enable auth, email, or Sentry by itself. Structured log context is copied and recursively redacted, including credentials nested in mappings and sequences. Credential keys are matched across case and separator conventions, including clientAPIKey and X-API-Key; repeated safe context is preserved and cycles are cut safely. A Django or DRF request object anywhere in a record, such as the request extra Django adds to every django.request 4xx/5xx record, renders as {"method": ..., "path": ...}, never as its repr, so a request object never carries its query string into a log. Lines other loggers write are their own: runserver's django.server request line still prints the full URL. configure_sentry() renders request objects in Sentry extras and log breadcrumbs the same way.

Bind a record's identity once it is known and every later line of the request, including request.complete, carries it under context:

from baltimore.patapsco.observability import bind_log_context

bind_log_context(case_id=case.public_id)

Bind identifiers, never resident data: logs redact credentials, not civic contact or location fields. None values are ignored, an explicit record extra wins over a bound value, and outside a request the binding persists unless the work runs inside request_context(). Bound values do not tag Sentry events.

File downloads

stream_csv() streams a CSV export that a client can always tell is incomplete, and content_disposition() names a file safely for every client:

from baltimore.patapsco.api import stream_csv


class CaseExportView(APIView):
    def get(self, request):
        rows = cases_for(request).values_list("number", "opened", "status").iterator()
        return stream_csv(request, rows, header=["Case", "Opened", "Status"], filename="cases.csv")

A download holds one 64 Ki-character chunk in memory whatever its row count, under WSGI and ASGI. A failure before the first chunk is the view's ordinary error response. A failure after it is logged as download.stream_failed with the request ID and re-raised, so the server aborts the transfer instead of ending it. max_rows= answers 413 with the error envelope (export_too_large, details.max_rows) before anything is sent, and otherwise sends the file with a Content-Length. bom=True adds the byte order mark Excel needs for non-ASCII text. A request that carries X-Download-End-Marker, as downloadFile() in @city-of-baltimore/patapsco sends, gets its token echoed and appended to the body, so a cut is visible even where HTTP framing hides it (runserver, or a proxy that speaks HTTP/1.0 to gunicorn). Call it from a synchronous view: it reads the first rows before returning.

A workbook, or any export that isn't a CSV, answers an oversized request the same way: capped_rows(rows, max_rows=N) returns at most N rows for the app's XLSX library to write, or raises ExportTooLarge, the same 413 from a DRF view; a plain Django view returns export_too_large_response(N). Answer the written file with file_response(content, filename=..., content_type=XLSX_CONTENT_TYPE) (a PDF the same way): it carries stream_csv's download headers, the safe Content-Disposition, Cache-Control: no-store and X-Content-Type-Options: nosniff, from one place.

Every cell of the header and rows goes through spreadsheet_safe_cell(), with no opt-out, so no cell opens in a spreadsheet as a formula. Text that starts with TAB, CR or LF, or whose first character after whitespace is =, +, -, @ or a full-width form of one, gets a leading '. Typed numbers and text that is exactly a negative decimal (-12.5, not -0500) are left alone. Vertical tab and form feed become a space, and the other C0 controls except TAB, LF and CR are removed, so a pasted control never fails an export. Call it yourself for a CSV written anywhere else, such as an import report on disk:

from baltimore.patapsco.api import spreadsheet_safe_cell

writer.writerow([spreadsheet_safe_cell(value) for value in row])

For an XLSX text cell, spreadsheet_safe_text() gives the same text with the controls scrubbed and no '. The writer's string type is the protection (openpyxl data_type = "s", xlsxwriter write_string); a ' prefix never is, because XLSX stores it as visible text:

from baltimore.patapsco.api import spreadsheet_safe_text

cell = sheet.cell(row=row, column=column, value=spreadsheet_safe_text(value))
cell.data_type = "s"

The wire contract states the whole policy. The TypeScript SDK's spreadsheetSafeText, spreadsheetSafeCell and buildCsv apply it in the browser, tested against the same vectors.

spreadsheet_cell_vectors() returns those vectors from the installed package, so an app's own export tests run every one through its real writer instead of copying them (app consistency standard §8.2). It needs no Django settings. Each vector is a frozen SpreadsheetCellVector with case, input, the XLSX text and the CSV output:

from baltimore.patapsco.contracts import spreadsheet_cell_vectors


@pytest.mark.parametrize("vector", spreadsheet_cell_vectors(), ids=lambda v: v.case)
def test_the_export_is_spreadsheet_safe(vector):
    assert exported_csv_cell(vector.input) == vector.output

content_disposition(filename, disposition="attachment") adds what Django's header lacks: an ASCII filename="..." fallback beside filename* for non-ASCII names, and a name reduced to its final path component, without control or bidi-control characters, and cut to 255 UTF-8 bytes.

Session lifecycle

An application that wants one session policy applies it in settings, installs the middleware after AuthenticationMiddleware, and authenticates its API with Patapsco's SessionAuthentication:

from datetime import timedelta

from baltimore.patapsco.sessions import session_settings

globals().update(session_settings(idle_timeout=timedelta(hours=1)))
MIDDLEWARE += ["baltimore.patapsco.sessions.middleware.SessionActivityMiddleware"]
REST_FRAMEWORK["DEFAULT_AUTHENTICATION_CLASSES"] = [
    "baltimore.patapsco.sessions.authentication.SessionAuthentication",
]

Only requests a person starts renew the session. Health, readiness, the status read and views marked @session_passive never do. A signed-out API request gets 401 with a WWW-Authenticate challenge. baltimore.patapsco.urls then mounts GET /api/session/status, POST /api/session/renew and POST /api/session/end ("Sign in again now"). Session lifecycle is the adoption sequence, and the wire contract is the exact behaviour. Nothing changes until session_settings() is applied.

Send email

from baltimore.patapsco.mail import email_settings
from baltimore.patapsco.observability.runtime import runtime_environment

globals().update(
    email_settings(
        production=runtime_environment() == "production",
        from_email="Baltimore OPI <no-reply@baltimorecity.gov>",
    )
)

Apps send through Django's email API. Production sends through SendGrid's SMTP relay with SENDGRID_API_KEY from the environment; anywhere else mail is printed, or, with PATAPSCO_EMAIL_ALLOWLIST, sent only to the addresses and domains listed. A key alone never turns sending on, and the allowlist is refused in production. assert_sends_through_the_relay is the assertion each app's suite shares. Send email is the adoption sequence, including the sender authentication City IT sets up.

Passwords and reset

from baltimore.patapsco.auth import password_settings

globals().update(password_settings(proxies=1))

Django's four validators with a minimum of 15 characters, one-hour reset links and the reset throttle, which names a client through the proxies or edge-set client_header the app declares. Use proxies=1 behind nginx and client_header="HTTP_FLY_CLIENT_IP" only behind Fly's trusted edge. The reset window retains its first deadline with approximate counts in shared cache; its database-cache companion owns rollover and refuses stale writers. Drain old reset-serving workers during upgrade to acquire the corrected window. The public client_address(request, proxies=None) positional and keyword forms remain available, and the DRF throttles in baltimore.patapsco.api.throttling (AnonRateThrottle, UserRateThrottle, ScopedRateThrottle) name a client by it, proven by the shared ThrottleIdentityVectors. An app that offers self-service reset mounts baltimore.patapsco.auth.urls (or, on Wagtail, sets WAGTAILADMIN_USER_PASSWORD_RESET_FORM), and runs the shared PasswordResetVectors from the testing extra in its own suite. An imported account with no password gets its first one through an activation link (send_activation, manage.py send_activation_links), never through self-service reset. Passwords is the adoption sequence.

Optimistic concurrency

from baltimore.patapsco.api import ETagField, OptimisticConcurrencyMixin


class PlanViewSet(
    OptimisticConcurrencyMixin, viewsets.ModelViewSet
): ...  # the serializer declares etag = ETagField()

A read answers ETag (and each list row its etag); a write must send If-Match, is compared with the locked row and saved by compare-and-swap (claim_revision), and a stale one answers 412 precondition_failed with details.current_etag, a missing one 428. Service code uses check_if_match and claim_revision. Save and autosave is the adoption sequence.

Branded error pages

handler404 = "baltimore.patapsco.pages.page_not_found"
handler500 = "baltimore.patapsco.pages.server_error"

With page_settings(product_name=...) in settings, a server-rendered site's 404 and 500 are the City's page with the Reference ID, the value of X-Request-ID. too_many_requests(request, retry_after=...) answers a lockout the same way, with Retry-After. A request that prefers JSON keeps the canonical envelope, and the 500 renders with no database and no context processor. Error pages is the adoption sequence.

Audit events

from baltimore.patapsco.audit import acting_as, audit, SystemActor, target_of

audit("data.export", outcome="succeeded", target=target_of(report), rows=340)
with acting_as(SystemActor("importer")):
    audit("access.override", outcome="succeeded", reason="backfill")

One event shape for who did which sensitive action, in the structured log by default. Installing "baltimore.patapsco.audit" adds an append-only table (PATAPSCO_AUDIT_SINKS = ["log", "database"], with PATAPSCO_AUDIT_RETENTION_DAYS) and records sign-ins and sign-outs. Audit events is the adoption sequence.

Local sign-in

globals().update(sign_in_settings())  # from baltimore.patapsco.auth.sign_in
path("", include("baltimore.patapsco.auth.sign_in_urls"))

With the signin extra, POST /api/session/ signs in and DELETE signs out. Every refusal answers one 400 sign_in_failed. Five failures for one name from one address lock that pair for fifteen minutes, and thirty from one address across every name lock the address (django-axes plus Patapsco's address count; fixed windows, proxy-aware, #720). SignInVectors in the testing extra proves it against the app's routes. Local sign-in is the adoption sequence.

Sentry privacy defaults

Install the observability extra and call the shared initializer only when a DSN is configured:

from baltimore.patapsco.observability import configure_sentry

configure_sentry(
    environment="production",
    release="my-app@1.2.3",
    traces_sample_rate=0.1,
)

configure_sentry() is deny-by-default: it disables automatic PII, request-body capture, and stack-frame local variables. Its event pipeline also removes request cookies and query strings, strips query/fragment data from request URLs and the Referer request header, reduces breadcrumb, span, and trace data URLs (url, http.url, url.full) to origin plus path and removes http.query, http.fragment, url.query, and url.fragment from them, and recursively redacts the shared credential-key fragments, Sentry's credential/session denylist, and common civic contact and precise-location fields. This protection also applies to transaction events. Safe operational fields such as request IDs, routes, and HTTP methods remain available.

traces_sampler optionally accepts a callable with the SDK sampling-context dictionary and a return rate from zero to one. It decides whether a trace is recorded at creation, before event processing; it takes precedence over traces_sample_rate and the parent decision. Apps own route exclusions and whether to honor context["parent_sampled"]. Omitting the callback preserves SDK defaults. Callback exceptions retain SDK behavior and can propagate, so app samplers must handle their expected failure cases. Initialization failures still return False without exposing configuration in logs.

Pass dsn="" to explicitly disable reporting even when SENTRY_DSN exists; omitting the argument or passing None permits that environment fallback. For domain enrichment, register a public SDK global event processor once, after successful initialization. It runs before the shared scrubber and final privacy hooks for errors and transactions. Retain stronger app policies there; do not import private Patapsco filters or replace the client's privacy options.

Redaction is key-based. Applications must not attach sensitive values under misleading keys or place resident data in exception messages. Product-specific fields still require a privacy review before being added to Sentry context.

Display-zone settings

from baltimore.patapsco.time import time_settings

globals().update(time_settings(display_time_zone="America/New_York"))

The additive helper validates the IANA zone and returns TIME_ZONE plus PATAPSCO_TIME["DISPLAY_TIME_ZONE"]. Calling it leaves settings and existing timestamp serialization unchanged; the app applies the mapping explicitly and passes the same zone to Bromo. It is safe before Django initializes. See the time guide for adoption. It is available in the current released package.

Configuration surface

The observability, diagnostics and session settings summarized here are not the complete configuration surface. See the module guides for auth, mail, sessions, cache, audit, uploads and time, including each module's activation and settings.

The library's existing diagnostics fallback remains local for existing consumers. Newly generated civic apps require an explicit accepted APP_ENV before settings startup; their environment contract names the runtime and preparation modes. Existing apps migrate that boundary deliberately.

Numeric XLSX exports (1.0.0)

Use spreadsheet_numeric_value from baltimore.patapsco.api for each exact integer/Decimal quantity and computed total before releasing a workbook. Accepted inputs return unchanged; unsupported precision, range or types raise ExportNumericPrecisionUnsupported, a safe correlated 422. Finish the private buffer before file_response and the app's successful-export audit; never round or substitute text. Plain Django views can return export_numeric_precision_unsupported_response().

The numeric wire contract defines the supported set and nominal decimal reader proof. spreadsheet_numeric_vectors() from baltimore.patapsco.contracts supplies exact source strings for regression tests through the app's actual writer. This API ships in 1.0.0; published 0.23.0 does not export it.