bolero-denmark

The Danish register documentation and literature corpus for Bolero, as one data-only PyPI distribution. Parsed text only: no PDFs, no upstream release archives, no raw CSV dumps — those stay at their canonical homes and are pointed to, not shipped. The wheel's one archive is the delivery vehicle: git archive HEAD of this repository, which bolero-harness denmark unpacks.

Use

pip install bolero-ai bolero-denmark
bolero-harness denmark            # unpack this repository into ./
bolero-harness denmark DIR --force

bolero-harness denmark (bolero-ai >= 0.1.8.post3) unpacks the repository archive this wheel carries, with the same hardened tar extraction the harness uses for its own source tree. Without bolero-ai:

python3 -c "import bolero_denmark, tarfile; tarfile.open(bolero_denmark.archive_path()).extractall('.', filter='data')"

The wheel is git archive HEAD, so it carries every committed file and not the DST_agentic_ext/ submodule. A checkout that needs that folder too (GitHub access and membership of the private upstream required):

git clone --recurse-submodules https://github.com/Northwestern-CSSI/Bolero-denmark.git
# or, in an existing clone:
git submodule update --init

What is inside

data-registry-bolero/    the curated dataset–group–table markdown tree over
                         Statistics Denmark's documentation systems:
                         fsv.md + fsv/ (531 registers in 13 native subject
                         groups, per-variable coverage segments, orderable
                         datasets, history, granted flags) and times.md +
                         times/ (121 registers in 16 native categories,
                         9,843 documented variables), plus the two
                         deterministic generators that built them
data-registry/           provenance of the raw source layer: SOURCES.md
                         (where every table comes from and how to re-fetch
                         it) and gefion-grant/ (the 143 registers granted
                         for the Gefion upload, with coverage audit)
papers/                  the literature corpus: danish-register-econ/ (the
                         Northwestern-CSSI systematic corpus of published
                         economics papers on Danish administrative data —
                         selection tables, extraction records, replication
                         packs metadata) and md/ (full-text markdown
                         conversions of the pilot and replication-target
                         papers; see papers/NOTE.md)
replication/             one registered replication deck per target paper
                         (manual, references.bib, transcribed seed with
                         authored proposals), runnable unchanged under
                         `campaign.mode=replica`; see replication/README.md
DST_agentic_ext/         git submodule pinned to abjer/DST_agentic_ext, the
                         Copenhagen–Northwestern collaboration repository
                         (metadata pulls, reproduction targets, harness
                         notes); private upstream, so its content is not in
                         the wheel — populate it with `git submodule update
                         --init` from a clone

What is deliberately not inside

• Paper PDFs, figure crops, and archives. Published-journal PDFs are copyrighted; only their text-extraction markdowns ship. The PDFs live beside their markdowns in the working tree this repository was assembled from. The danish-register-econ snapshot is likewise text only: its figure-crop PNGs, schema/mapping.zip extracts, and _trash/ workspace are pruned (papers/NOTE.md lists exactly what).
• The expanded raw registry layer (~876 MB). Every table is reconstructible from the upstream release archives named in data-registry/SOURCES.md (public GitHub: EZlzh/danish-registry-schema, vendored in abjer/DST_agentic_ext — the submodule above, when populated); the curated tree above is the layer meant for reading.
• The DST_agentic_ext content. The upstream repository is private and marked not for sharing outside the project team; this repository pins its commit and nothing more. git archive skips submodules, so no release carries its files.
• Paper PDFs of the replication targets. The decks under replication/ cite the markdown conversions in papers/md/; the PDFs stay in the assembly tree. The two generators in data-registry-bolero/ rebuild the curated tree from that expanded layer.

Licensing and provenance

The license expression is MIT AND LicenseRef-Corpus-Rights: the packaging code is MIT (LICENSE); the corpus content is not, and NOTICE.md — the LicenseRef-Corpus-Rights text — states whose it is. In short: the register documentation is public metadata from Statistics Denmark's documentation systems (TIMES, FSV, HKV — crawled and translated upstream, see data-registry/SOURCES.md); the extraction records belong to the danish-register-econ project (its own README.md carries its provenance); the full-text markdowns in papers/md/ are conversions of copyrighted articles, shipped for research use with copyright remaining with their authors and publishers; the registration decks under replication/ are this project's own text (MIT) quoting only numbers the originals printed. Identifiers, register codes, and DST URLs are public metadata; no row-level or personal data is included.

Release

A release is one tag push. .github/workflows/release.yml builds the distribution from the tagged tree, publishes it as the tag's GitHub Release, and publishes the same files to PyPI through the project's Trusted Publisher (no API token; PyPI environment pypi). The tag must equal v + the version in pyproject.toml.

# bump version in pyproject.toml, commit, then:
git tag v0.1.0.post2 && git push origin main v0.1.0.post2

packaging/build-release.sh is what the workflow runs and builds the same files locally (dist/) for inspection. The wheel embeds git archive HEAD of this repository, so a release is exactly the committed tree; dist/, build/, and _source/ are build artifacts and never committed.