Camden

Platform baseline: baltimore-patapsco==2.7.10

Baltimore's civic data SDK.

Named for Camden Yards, the Inner Harbor rail-and-ballpark hub where Baltimore's traffic gathers and arrives — just as this SDK gathers the city's data and delivers it with provenance.

Camden gives analysts and data staff in the Baltimore City Mayor's Office of Performance and Innovation (OPI) one consistent, safe, repeatable way to do the things we do every week: pull SharePoint lists into pandas, query city databases, fetch Open Baltimore layers, geocode addresses, send report emails, and post to Teams.

Part of the Baltimore civic foundations

Camden is one of three independently versioned foundations OPI maintains for Baltimore civic software:

• camden (this repo) — Python data SDK (baltimore-camden): Graph/SharePoint, Open Baltimore, city databases, geocoding, provenance, delivery.
• patapsco — platform / integration point (baltimore-patapsco + @city-of-baltimore/patapsco): request IDs, safe error envelopes, observability, health/readiness, typed frontend client, and app scaffold. Authentication remains app-local today.
• bromo / Baltimore Design System — TypeScript presentation layer (@city-of-baltimore/bromo): tokens, theme, shell, accessible components.

How the pieces fit together: patapsco/docs/use/foundations.md is the canonical architecture overview. The shared naming rule (baltimore-<name> on PyPI, @city-of-baltimore/<name> on npm) is ADR-0005.

Contents

• Part of the Baltimore civic foundations
• Philosophy
• What Camden supports
• Installation
• Configuration
• Quickstart
• Coming from r4OPI
• Development
• Roadmap
• Contributing
• License

Philosophy

Camden exists to lower the ramp: reusable helpers, copy-pasteable recipes, and shared standards, so each analyst doesn't hand-roll their own auth, connection strings, and API calls.

• Python is the preferred default for new shared, repeatable, production-facing, integrated, or automated workflows.
• R remains welcome for statistical analysis, exploratory work, and existing R/Shiny assets.

Design principles: plain-English function names, predictable return types (pandas in, pandas out), small composable functions, no magic, no hard-coded credentials, and everything configurable without editing code.

New to the toolkit? Start with the prose-first Camden overview, which explains the people it serves, how data moves through it, its safety model, and where product responsibility begins and ends. The product requirements and user stories define Camden's users, complete capability scope, acceptance boundaries, failure behavior, and verification evidence. The technical specification is the current system specification that maps those outcomes to Camden's architecture, boundaries, and verification model. The evidence-mapped service standards state what consuming teams can rely on, how each promise is proven, and what Camden deliberately does not promise. The documentation map explains which page owns each kind of guidance and when it must change.

What Camden supports

What Camden is not: a stats library, a dashboarding framework, or a place for one-off analysis code. If a helper is only useful to one project, it belongs in that project.

Installation

Camden supports Python 3.13 and 3.14. The repository also pins the family-wide Node.js 26 tooling baseline in .nvmrc (the version in patapsco/contracts/platform-bom.toml); Camden itself ships no JavaScript runtime or npm package.

Releases are published to PyPI as baltimore-camden from a matching vX.Y.Z GitHub release through OIDC trusted publishing. Consumer applications pin a released version rather than a moving Git branch.

The PyPI project and trusted-publisher binding are active; releases publish through the workflow without a stored registry token.

Install the published release from PyPI. In an application, lock the version you verified (uv add records it, and uv lock pins it exactly):

# with uv (preferred)
uv add "baltimore-camden[all]"

# with pip
pip install "baltimore-camden[all]"

This README is also the package's PyPI description, so it names no version. Which release is published, and which version a source checkout is preparing, is recorded in the changelog and on the documentation home. A source checkout is useful for development and review, but it is not registry-backed release evidence.

Extras keep the core light — install only what you need:

Configuration

Camden reads CAMDEN_* environment variables (or a local .env file — see .env.example). Safe, non-secret defaults are provided where a shared default makes sense. Features that call protected systems still require their documented credentials or connection profile. The legacy Graph variables opi_graph_id / opi_graph_secret remain supported.

Keep credentials out of code. Camden's environment-backed Settings models secret-bearing credential fields as secret values so their ordinary representation does not reveal the configured value. Source and delivery functions expose the settings= or client= injection documented in their signatures when a job needs an explicit override. Inspect resolved configuration with camden.get_settings(); when diagnostics are needed, import and log only safe_summary() from baltimore.camden.runtime.

Quickstart

from baltimore import camden
from baltimore.camden.graph import GraphAuth, GraphClient

# SharePoint lists <-> pandas
site = camden.get_site()  # the OPI site by default
updates = site.get_list("Book of Business - Weekly Updates")
on_track = updates.to_df(where="fields/Status eq 'On track'", labels="display")
updates.choices("Status")  # valid options before you write
updates.append(new_rows_df)  # person columns: use site.user_lookup_id(email)

# Sync a tracker safely: dry-run the plan, then apply. Deletes are opt-in.
# `columns=` protects comparisons and existing-row updates; project weekly_df
# to owned fields first because new rows include all its non-reserved columns.
print(updates.diff(weekly_df, key="Title").summary())  # add 3, update 2, delete 0, unchanged 41
updates.sync(weekly_df, key="Title")

# Paste a link, get data — and write it back the same way
df = camden.read_sharepoint("https://bmore.sharepoint.com/:x:/r/sites/opi/...xlsx")
camden.write_sharepoint(df, "weekly.xlsx", folder_link)  # DataFrame -> xlsx -> SharePoint

# Document libraries, by path
docs = site.drive()  # the default "Documents" library
docs.ls("General", recursive=True)  # name/path/kind/size/modified
docs.download_dir("General/Reports", "reports/")
docs.upload("weekly.xlsx", "General/Reports")

# City databases -> DataFrames (parameterize with ?, never f-strings)
permits = camden.read_sql(
    "SELECT * FROM dbo.Permits WHERE issued_date >= ?", "citistat", params=["2026-01-01"]
)
neighborhoods = camden.read_table("dbo.Neighborhoods", "citistat")
camden.write_sql(
    camden.check_frame(scores_df, columns={"score": "number"}, no_nulls=["agency"]),
    "dbo.Scores",
    "opi-sandbox",
    if_exists="append",
)
conn = camden.connect("citistat")  # raw connection when you need one

# Open Baltimore: search the catalog, then pull any dataset by name, page URL, or id
camden.search_datasets("parking fines")  # title, dataset_id, snippet, modified
hoods = camden.get_shape("neighborhoods")  # GeoDataFrame, EPSG:4326
vacants = camden.get_layer(  # paste a dataset page straight from the catalog
    "https://data.baltimorecity.gov/datasets/8ed6565f93a04455ae6bd5bcb7d272db_19/explore"
)
camden.describe_layer("real_property")  # provenance: rows, last edit, owner
camden.assert_fields("vbns", {"NoticeDate": "Date"})  # fail fast on schema drift
camden.layer_fields("real_property")  # inspect the schema before filtering
camden.layer_count("real_property")  # size up a big pull first (~238k)
fines = camden.get_table(  # JSON rows only — no geometry, much faster
    "d2a2330d6a374ad39a24a0d7f7b58f19_0",  # parking fines: ~11M rows
    where="ViolFine > 500",
    fields=["Citation", "Description", "ViolFine", "ViolDate"],
)
camden.download_layer(  # page-written Parquet; no full DataFrame
    "d2a2330d6a374ad39a24a0d7f7b58f19_0", "fines.parquet", where="ViolDate >= DATE '2026-01-01'"
)  # duplicate detection still retains every object ID; memory grows with rows
fines = camden.cached_table(  # or cache it: downloads at most once per max_age
    "d2a2330d6a374ad39a24a0d7f7b58f19_0",
    where=camden.where_since("ViolDate", "2026-01-01"),  # incremental pulls
    max_age="6h",
)  # cache_layer(...) returns the Parquet path instead; every download writes a
# manifest commit marker — camden.manifest_for(path) verifies the file hash and
# says exactly what query produced it — and iter_layer_rows() streams row-by-row

# Geocoding + spatial tagging
located = camden.geocode_frame(df["address"], fallback=True, rate_limit_per_second=5)
tagged = camden.spatial_join(located, hoods, keep=["Name"])  # neighborhood per address

# Styled Excel in one call (branded header, filters, number formats)
camden.write_excel(summary_df, "weekly.xlsx", currency=["TotalFines"], percent=["ShareResolved"])

# Email a report; post to Teams as a signed-in user
camden.send_mail(
    to="team@baltimorecity.gov",
    subject="Weekly VBN report",
    body="Attached.",
    attachments=["vbns.xlsx"],
)
with GraphClient(auth=GraphAuth(flow="device_code")) as delegated_graph:
    camden.send_channel_message(
        "OPI",
        "Book of Business",
        "Reporting period is <b>open</b>.",
        client=delegated_graph,
    )

# City utilities
camden.fiscal_year_label("2026-07-01")  # "FY27"
camden.week_bounds("2026-07-08")  # Fri-Thu reporting week: (Jul 3, Jul 9)
camden.dollars(121324520.45)  # "$121,324,520.45"

Teams note: Graph only allows channel messages from signed-in users (delegated auth), and send_channel_message() does not choose or validate that flow for you. Pass an explicitly delegated client as shown above. For unattended or scheduled jobs, use camden.teams.send_webhook() with a channel's Workflows URL. That URL is itself bearer-like authority; store and redact it as a credential.

Coming from r4OPI

New in Camden (no r4OPI equivalent): send_mail, send_channel_message, send_webhook, upload_file, read_sharepoint, list update/delete, SharePoint URL parsing, shared_with_me() listing.

Development

One command contract for humans, CI, and agents — Taskfile.yml (install go-task):

git clone https://github.com/city-of-baltimore/camden && cd camden
task setup      # locked Python + documentation tooling, and verified pre-push hook
task test       # fast offline unit tests + docstring examples
task typecheck  # mypy only
task build      # inspect and clean-install the wheel and sdist
task prepush    # complete pre-push gate: static checks, coverage, package install, docs
task check      # alias for prepush
task validate   # locked setup + prepush + foundation currency
task fmt        # auto-format and fix lint findings
task ci         # lean hosted gate: locked setup, static checks, lean-CI policy check
task docs-serve # live-preview the documentation site (mkdocs-material + mkdocstrings)
task docs:verify # verify already-built docs: local assets, navigation, themes, accessibility
task docs:setup # provision the pinned docs browser once before its first verification
task hooks:install # reinstall only the verified pre-push hook
task integration # no Graph/delivery credentials required; no remote provider mutation

Three tiers with the same meaning as every repo in the family — ci is static checks only, prepush adds tests/coverage/builds, validate is the release gate. See patapsco/docs/reference/app-consistency-standard.md section 4; Patapsco's published platform-check (run by task ci as task ci:policy) enforces the boundary mechanically. Because tests live pre-push, installing the pre-push hook is not optional: task setup installs it, or task hooks:install on its own.

Documentation development also needs Node 26 and npm 11. The private npm project bundles released Bromo tokens and self-hosted fonts for MkDocs; it is separate from the Python SDK. task docs builds the site, and the local gate verifies it before inspecting the Python archives. See the documentation maintenance guide for adapter ownership, upgrades, and browser evidence.

Unit tests run offline with no credentials. task integration needs no Graph or delivery credentials: it reads small public Open Baltimore and geocoder samples and verifies one provenance-marked local artifact. It makes no remote provider mutation or delivery, although it writes temporary files and its evidence bundle locally. Protected Graph and SharePoint reads use task integration:tenant; scratch writes and deliveries have separate named profiles whose commands carry the invocation-only --live-writes or --live-sends consent flags. Those named tasks are the supported interface; the explicit flags in the direct command line are the enforcement boundary. During a live run the harness rejects PYTEST_ADDOPTS, PYTEST_DEBUG, PYTEST_PLUGINS, any configured addopts beyond Camden's reviewed -ra baseline, anything other than file-descriptor capture, passed-test output, live or file logging, pytest debug/pastebin export, warning-policy overrides, explicit pytest plugin changes, debugger modes, and selectors that could reduce a profile. Those ambient diagnostics could expose signed URLs before report redaction. No live-harness or Pytest configuration variable can grant side-effect authority. Protected verification still requires a trusted runner, process environment, certificate store, network path, and proxy configuration.

Every active run writes redacted JSON and Markdown evidence below build/live-verification/. A required check that is missing, skipped, or duplicated is non-green. A write run is also non-green when cleanup fails or could have left residue. The prose-first live-verification guide explains the profiles, safe scratch-target setup, GitHub environment, evidence meanings, and operator response.

See AGENTS.md for architecture, conventions, and testing patterns, and CONTRIBUTING.md for the pull-request workflow.

Roadmap

Current state, completed release history, external permission follow-up, and the two-project data-foundation backlog live in ROADMAP.md. Execution is tracked with GitHub milestones and issues.

Contributing

Contributions are welcome — new helpers, recipes, docs, and bug reports. Branch, open a PR to main, and CI must pass. If you have an idea but aren't sure where it fits, open an issue and ask.

License

Distributed under the MIT License. See LICENSE for more information.

Maintainers

Mayor's Office of Performance and Innovation (OPI).