dansk-register-platform

A platform for epidemiological studies on the Danish national registers, and the study pipelines built on it.

The package contains no data. It is the code that reads a register delivery inside a secure research environment, builds cohorts and outcomes from it, and estimates and reports the results.

What is in it

The dependency direction is one way and enforced by a test: the platform never imports a study, and studies never import each other. Code that turns out to be general is lifted into dansk_register rather than shared sideways.

The register runner and the estimators are Rust, compiled into the wheel as one extension module. The methods crates it builds on, cohort matching, semicontinuous models, staggered difference-in-differences and the chronic disease phenotype, are published separately and consumed by version.

Install

pip install dansk-register-platform                 # everything, all three studies
pip install "dansk-register-platform[study2]"       # names a study; installs the same files
pip install "dansk-register-platform[all]"          # all three, explicitly

Before 0.7.0 the distribution was published as dansk-register. The import packages did not change; only the name on pip install did. The old name still resolves, to a distribution that stopped being updated at the rename.

The estimators are written in polars and validated against R, so the pipeline carries no numeric stack of its own. matplotlib is a base dependency because numpy arrived on every real run anyway; what keeps the boundary real is that its imports live under a figures package and a contract test fails the build if one escapes into an estimator. Without a renderer every result is still produced and the figure stage records that it could not draw.

repair is the only optional dependency: pyarrow, for register-repair-utf8 alone. Install it with pip install 'dansk-register-platform[repair]'. The same kind of contract test keeps pyarrow out of everything else.

Commands

edu-labour --list-profiles                          # what each profile runs
edu-labour-preflight --raw-root PATH                # what a delivery contains
edu-labour --output-dir RUN --profile production \
    --raw-root PATH --allow-expensive-stages --resume
edu-labour-figures --bundle RUN/outputs/export --out figures

societal-costs --capability PATH                     # what this study needs vs. what a delivery has
societal-costs --output-dir RUN --profile production \
    --raw-root PATH --allow-expensive-stages --diagnostics
societal-costs secondary_sensitivity --output-dir RUN   # the arm no profile runs

register-repair-utf8 PATH --report utf8_scan.csv    # find invalid UTF-8 in converted Parquet
register-repair-utf8 PATH --fix --report utf8_fix.csv

--allow-expensive-stages is required for production: matching scans the whole delivery, and without the flag the runner refuses rather than silently starting a multi-hour match. --profile production runs 19 stages but not secondary_sensitivity, the analyse-only sensitivity and subgroup arm, which reads a completed run and writes into a subdirectory of it.

register-repair-utf8 needs the repair extra and says so if it is missing.

Add --dry-run to any edu-labour invocation to resolve the plan and settings without touching data.

Design notes

Stages talk through files. Every stage writes named artifacts into a run directory and reads its inputs from there, so any stage can be re-run alone against a previous run's outputs, and --resume can skip what is already done.

Estimators are written against polars. Aalen-Johansen, Fine-Gray, Cox and the multi-state occupancy are implemented directly rather than taken from a modelling library, and each is validated against reference values from R with those values checked into the tests. That keeps the numbers checkable line by line and avoids a dependency that may not install in an offline environment.

Disclosure control is part of the pipeline. Results leave the secure environment as a bundle of CSV tables with small cells suppressed and the estimates resting on them blanked. Figures are rendered from that bundle rather than from the raw artifacts, so a figure cannot show what the table beside it was not cleared to show.

What this covers, and what it does not

Secure-environment research is usually described with the Five Safes, and it is worth saying plainly which of the five a piece of software can touch at all.

"Partly" is the operative word in both rows. The output checks are cell-by-cell and cannot see a small number recovered by differencing across tables; each bundle's MANIFEST.txt says so in its own text. Running this framework is not a substitute for your host's output review, and it does not make an unapproved analysis approved.

Status

Research code, developed alongside the studies it implements. The public API is not stable.

Licence

MIT. See LICENSE.