mostlyright-data

mostlyright-data converts a question and selected sources into a reproducible Parquet dataset. An external agent may select sources and author a declarative recipe. A human approves the exact recipe digest. Deterministic workers acquire bytes, transform data, run checks, and seal an immutable Build.

For question-to-dataset work, the external host agent owns orchestration through delivery of a verified Build and its table notebook. Exact proposed-contract or Recipe-digest approval is the only user pause: typed refusals and failed checks drive bounded retry, supported substitution, or a replacement Recipe. Workers remain fail-closed, changed Recipe digests require new approval, and Studio alone owns release. If every registered lawful alternative is unsupported, the agent reports that typed impossibility instead of bypassing a refusal or fabricating a dataset.

Source discovery is a host-owned, multi-lane search. network_search_bundled: false is a truthful capability disclosure: the credential-isolated Harness does not contain or execute a browser. It is not a switch that disables the host's public-web tools. For question-to-dataset work, the host must search every available lane—registered Harness acquisition adapters, a configured sealed catalog, public web search, official provider catalogs or APIs, and user-supplied sources—with no preferred lane. It then merges and deduplicates source options, inspects them through registered boundaries, and compares them on the canonical harness-source-fitness.v1 dimensions. Every lane cites an immutable host capability receipt, every query cites a host-tool receipt, and every comparison cites source or receipt evidence. A host receipt is a strict JSON object whose SHA-256 is calculated over its canonical UTF-8 JSON bytes; preserve those bytes in research.ipynb or record the exact durable host-result reference beside the digest. Those receipts and the report remain untrusted host input: mr-data source-audit checks their structure and internal bindings, but does not claim a browser ran. It still performs no search, fetch, approval, or acquisition itself.

Selected and eligible sources also record the exact promoted inspection coordinate—locator kind, credential-free locator, certified Reader identity plus decode-options digest (or null for native CSV handling), format, and host platform—and its immutable inspection-receipt digest. Typed acquisition refusals are not fitness failures. Non-final retries remain rejected; recovery and lawful exhaustion are terminal, and an unsupported decision requires every typed-refusal route to be lawfully exhausted.

The repository includes a complete report example and the canonical host-receipt objects whose digests it cites. Audit the report with:

uv run --extra dev mr-data source-audit \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --discovery-report examples/city-temperature/source-discovery-report.json \
  --json

The package version is 0.9.0 and requires Python >=3.13,<3.14.

Two install profiles

mr-data is one binary with two install profiles, and the command names are the same in both.

Profile	Install	Where it runs	What it is
Hosted (the default)	pip install mostlyright-data	Any operating system with CPython 3.13 -- Linux, macOS and Windows alike	Pure Python. One py3-none-any wheel, no compiled extension in the install closure and, today, no third-party package at all. It signs in, submits work to the Mostly Right backend, streams a run's live progress, downloads the dataset it produced, and shows the receipts.
Local	pip install "mostlyright-data[local]"	Linux; installable on macOS, untested there	Everything above plus the engines, the certified Readers, the Clean room, and every build command below. Not a product surface: ADR 0021 makes every lane hosted, and this extra is operator and developer tooling for the machines that build the workers.

One wheel serves every platform, and that is checked rather than asserted: thin-client-smoke builds exactly one artifact and installs that same file on ubuntu, macOS and Windows before running the hosted command surface on each.

Windows gets the hosted profile and not the local one, and the reason is not that nobody has ported it: the local lane runs every acquisition and every Reader behind a kernel boundary built from POSIX process control -- fork and cgroups v2 -- and Windows has neither. Since ADR 0021 that is also true of macOS: Seatbelt was the second boundary implementation, it existed so a person's Mac could run a lane, and it left with the product surface that needed it. Note that pip install "mostlyright-data[local]" succeeds there, because the engines all publish Windows wheels, so what you installed is not what decides it. mr-data asks the machine, and every local command refuses by name with that reason rather than running into a traceback later. One consequence is recorded rather than hidden: the credential mr-data login stores is written at mode 0600 on Linux and macOS, and on Windows is protected by the ACL your user profile directory already carries, which is a weaker statement.

mr-data chooses the lane by what is installed. With [local] present every invocation behaves exactly as it always has; --hosted on build, run, watch, status or deploy-dataset selects the backend instead. Without [local], the hosted lane is the only lane, and a command that runs on this machine refuses by name and says what to install rather than failing on a missing import.

Where a hosted command runs is not a variance a person has to track: a submission prints the dashboard address for the run it queued, and mr-data watch <run-id> follows that run's event log live, resuming from its last cursor when the connection is cut rather than replaying it.

Supported platforms

Platform	Installed capability
Anything CPython 3.13 runs on -- Linux, macOS, Windows	The hosted profile, complete: login, whoami, and every hosted command. This is the product, and it is the row that matters.
Ubuntu 24.04 x86_64, CPython 3.13.14	Additionally the [local] extra: every certified Reader over local files and public HTTPS, including bounded archives, workbooks, and optional GRIB. The Clean room uses a private network namespace and an operator-bounded cgroup root. This is the platform the hosted workers are, which is why it is the one that is tested.
macOS 14 arm64	The hosted profile, and [local] installs. Nothing under [local] is tested there any more: ADR 0021 retired parity-macos, sandbox-macos, and the macOS legs of the local-* workflows along with the local lane. Clean-room execution refuses -- SANDBOX_MEMORY_BOUNDARY for a Reader, SANDBOX_OS_BOUNDARY for anything else -- because the Seatbelt boundary that used to admit it is gone. A logged-in single-address credential-free public HTTPS acquisition still runs in Studio's reviewed Linux/amd64 hosted Clean room. Bounded collections require both local attestations, so they need a host that can produce them. Catalogue authoring, which is local by design, still works.
Windows	The hosted profile. pip install "mostlyright-data[local]" succeeds, because the engines all publish Windows wheels, so what you installed is not what decides it: every local command refuses by name and says why.

The GRIB Reader is optional. If a compatible mostlyright-grib binding is unavailable, first use fails with READER_DEPENDENCY_UNAVAILABLE. It accepts one extracted GRIB2 record and rejects a multi-record GFS or HRRR file.

Install from a checkout

Update the persistent uv tool and install the complete matching skill for both Codex and Claude Code with one command from the checkout:

uv run --no-project --python 3.13 python scripts/install_checkout.py

The installer resolves the checkout from its own path, reinstalls mr-data from that exact directory and installs the exact pinned generated-client wheel alongside it, copies the complete skills/mr-data-build tree to both personal skill directories, and then verifies mr-data --version, exact source-to-installed package bytes, and exact source-to-installed skill-tree parity. The separately owned generated Studio client is also exposed as the published distribution's hosted extra. The base Harness wheel remains client-free for credential-free Courier images. It also binds every installed launcher to the [project.scripts] entry point it declares.

PATH is checked last, after the verified generation is committed, because PATH belongs to your shell rather than to the installation and an ambient shell problem must never restore the code the install just replaced. If the uv tool bin directory is not on PATH at all, which is the ordinary first-install case, the installer keeps the install, exits 0, and tells you to run uv tool update-shell and open a new shell. If a different mr-data is found earlier on PATH, the installer keeps the install and exits 2 with a warning naming both paths: typing mr-data would otherwise still run the stale program. Put the uv tool bin directory earlier on PATH, or remove the other one.

So read the exit code rather than testing it for zero: 0 and 2 both mean the new generation is installed and verified, and only 1 means the install was refused and the prior generation restored. Rerunning on 2 produces the same 2 until PATH changes.

It preflights both skill targets before changing the uv tool. If an installed skill differs, it refuses with no mutation; inspect the named path and rerun with --force only when replacement is intended. The installer retains private backups of the exact prior uv tool environment, all Harness launchers, and both prior skill trees until final parity succeeds. Any failure restores that whole prior generation, or removes every new artifact when no prior generation existed. A single per-user lock serializes this entire operation with direct skill installs. Once exact final parity is verified, the new generation is successful; failed best-effort backup cleanup emits a warning and leaves the named private backup for manual recovery instead of undoing that generation.

For an isolated wheel smoke test instead of a persistent global tool update:

artifact_dir="$(mktemp -d)"
uv build --wheel --out-dir "$artifact_dir"
set -- "$artifact_dir"/mostlyright_data-*.whl
test "$#" -eq 1 && test -f "$1"
wheel="$1"
uv venv .work/harness-install --python 3.13
uv pip install --python .work/harness-install/bin/python "${wheel}[local]"
.work/harness-install/bin/mr-data --version
.work/harness-install/bin/mr-data inventory --json

Run ordinary CSV with the installed binary

mkdir -p .work
.work/harness-install/bin/mr-data init .work/installed-city-temperature \
  --run-id installed-city-temperature \
  --input-root examples/city-temperature \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --plan examples/city-temperature/question-plan.json \
  --semantics examples/city-temperature/semantics.json \
  --json
.work/harness-install/bin/mr-data plan .work/installed-city-temperature --json
.work/harness-install/bin/mr-data run .work/installed-city-temperature --json
.work/harness-install/bin/mr-data verify .work/installed-city-temperature --json

The workspace must not exist before init. run seals the Build under <workspace>/result and writes table.ipynb beside it.

Create a research directory, then start the local notebook viewer in a second terminal before init. Keep the future workspace absent:

mkdir -p .work/installed-city-temperature-research
.work/harness-install/bin/mr-data view \
  --workspace .work/installed-city-temperature \
  --research-dir .work/installed-city-temperature-research \
  --json

The receipt names url, research_notebook, and table_notebook. Codex and Claude Code agents always open url in their in-app browser and never pass --open; a host that is genuinely terminal-only and is not Codex or Claude Code may use --open. For a direct Build, use mr-data view --run-dir PATH --research-dir RESEARCH_DIR. view stays on loopback and starts before source selection or Recipe approval. It streams atomic research.ipynb updates and adds table.ipynb when the Build finishes. Keep an in-app viewer tab across approval turns. If a successful Build reports an unavailable sidecar, run mr-data notebook RUN_DIR --json; the viewer writes table.ipynb only when there is none and never replaces one that is there. mr-data watch remains an opt-in operational diagnostic.

Test the packaged Reader boundary

On Ubuntu 24.04 with cgroup v2, this command runs the installed Reader in the required memory and network boundaries:

uv sync --locked --extra dev
sudo sh -eu -c '
  echo "+memory +pids" > /sys/fs/cgroup/cgroup.subtree_control
  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test
  echo 536870912 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
  echo 0 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.swap.max
  echo 64 > /sys/fs/cgroup/mostlyright-stock-reader-test/pids.max
  echo 1 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.depth
  echo 2 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.descendants
  echo "+memory +pids" > \
    /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.subtree_control
  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
'
sudo chown "$(id -u):$(id -g)" \
  /sys/fs/cgroup/mostlyright-stock-reader-test \
  /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.procs
test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
test ! -w /sys/fs/cgroup/cgroup.procs
runner_uid="$(id -u)"
runner_gid="$(id -g)"
sudo env \
  GITHUB_WORKSPACE="$PWD" \
  RUNNER_UID="$runner_uid" \
  RUNNER_GID="$runner_gid" \
  bash <<'STOCK_READER'
set -eu
cd "${GITHUB_WORKSPACE}"
echo "$$" > /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
exec 3</sys/fs/cgroup/mostlyright-stock-reader-test
exec 4</proc/self/ns/net
exec unshare --net -- setpriv \
  --reuid="${RUNNER_UID}" \
  --regid="${RUNNER_GID}" \
  --init-groups \
  .venv/bin/python -m pytest -q tests/test_stock_reader_workflow.py
STOCK_READER
find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
  -type d -name 'mostlyright-sandbox-*' -exec sudo rmdir {} +
test -z "$(find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
  -type d -name 'mostlyright-sandbox-*' -print -quit)"
sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test
test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test

For development, run the same workflow from the checkout:

uv sync --locked --extra dev
mkdir -p .work
uv run --extra dev mr-data init .work/city-temperature \
  --run-id golden-city-temperature \
  --input-root examples/city-temperature \
  --question examples/city-temperature/question.json \
  --requirements examples/city-temperature/requirements.json \
  --source-proposals examples/city-temperature/source-proposals.json \
  --plan examples/city-temperature/question-plan.json \
  --semantics examples/city-temperature/semantics.json
uv run --extra dev mr-data plan .work/city-temperature
uv run --extra dev mr-data run .work/city-temperature
uv run --extra dev mr-data verify .work/city-temperature

Add --json for one JSON object on success or failure. Failures exit nonzero. The command result is the receipt; paths named by a receipt remain mutable after the command returns.

--input-root and --output may be relative paths. Relative paths are resolved from the current working directory.

Commands

Command	Behavior
mr-data catalog-search	Search a sealed source catalog and return one entry with its query trace.
mr-data catalog-fill	Stage and publish one bounded credential-free CKAN, STAC, or SDMX catalog.
mr-data catalog-harvest	Stage a resumable Data.gov provider sweep, or verify its key boundary without making a request.
mr-data catalog-author	Convert one completed private harvest into bounded immutable authoring shards.
mr-data catalog-publish	Validate harvest and authoring receipts, then publish and independently read back one packed catalog generation.
mr-data catalog-update	Install one published catalog release after checking its exact archive digest, its store digest, and its descent from the generation already installed.
mr-data index	Rebuild the private local Build index under explicit discovery and byte limits.
mr-data search	Search exact public-catalog and private-Build snapshots under caller-supplied retrieval limits.
mr-data source-audit	Validate the host's multi-lane discovery report, source comparisons, proposals, and citations without searching or fetching.
mr-data peek	Report columns, types, and sample rows for a local file or public HTTPS resource, including an exact certified Reader selected with --reader FAMILY@VERSION --reader-options JSON.
mr-data record	Seal messages already recorded from a live source into bounded digest-named batches a build can read, writing down each dropped connection. It opens no connection and reaches no network.
mr-data author	Convert YAML into a Recipe.
mr-data approve	Bind approval to an exact Recipe digest.
mr-data plan-check	Execute a dataset plan in a temporary directory and report the result.
mr-data show	Inspect one Build.
mr-data list	List Builds under a root.
mr-data diff	Compare two Builds.
mr-data review --local	Run local checks without granting approval; exit 1 when a check fails.
mr-data deploy-plan	Validate a reviewed Build and write a deployment request without deploying it.
mr-data deploy RUN_DIR	Stage an exact verified Build, record and confirm its internal approval as the authenticated Editor, then activate and bind the Cloud Dataset.
mr-data deploy-status RUN_DIR	Report the state Studio holds for the run that deployment queued, optionally waiting for it with --watch-seconds N. It reads; it changes nothing Studio holds.
mr-data deploy-dataset RUN_DIR	Bring the Build that deployment's run released back into RUN_DIR/hosted, checked against the fingerprints the release sealed and checked again here, and write its dataset notebook.
mr-data review-setup	Generate review enrollment, keys, role tokens, and anchor input.
mr-data notebook RUN_DIR	Create or refresh table.ipynb for an existing Harness Build.
mr-data view --workspace PATH / --run-dir PATH	Serve the local notebook viewer, optionally pairing it with an explicit --research-dir.
mr-data watch	Serve the read-only event view.
mr-data workflow	Print the supported workflow stages and their authority boundaries.
mr-data preflight	Report build blockers for the host, for named options, or for a named workload measured against the engine's own ceilings.
mr-data inventory	Report installed capabilities.
mr-data auth	Validate the effective device credential with Cloud; manage metadata-only device revocation, remote-first logout, safe rotation, and explicit recovery; explain the ephemeral token boundary.
mr-data login	Sign in to the Mostly Right cloud with a device code and store a device credential. login --force is the compatibility alias for safe auth rotate, never an in-place truncate.
mr-data whoami	Compatibility alias for the remotely validated mr-data auth status answer.

The hosted lane

Twenty-five commands run on the Mostly Right backend. On a [local] install they take --hosted to select that lane; on the hosted-only install they are what the command already does. Hosted parity is the full ledger, generated from the classification the software itself branches on; the table below is the submit-and-follow half of it, plus the commands that write something other than a run.

Command	Behavior
mr-data build --hosted / mr-data run --hosted	Queue one hosted initial build from an approved recipe proposal (--recipe-proposal, --recipe-digest, optionally --resource-class standard|large) and print the dashboard address to watch it at.
mr-data watch RUN_ID --hosted	Stream that run's event log live, rendering the facts the worker measured. It resumes from its last cursor across a stream cut instead of replaying, and stops when the run reaches a state that appends nothing further.
mr-data status RUN_ID --hosted	Report the state Studio holds for one run, and with --receipts its execution summary and the sealed evidence for the Build it produced.
mr-data deploy-dataset RUN_ID --hosted --output DIR	Fetch that run's dataset through a short-lived signed URL, refuse it unless the bytes hash to the digest the release sealed, and write it once into DIR.
mr-data acquire --hosted	Fetch one public address in Studio's clean room and print the Receipt Studio bound, naming under reader_binding which party resolved the pinned Reader. A source that refused answers status: "refused" with its own sealed code and exits 1.
mr-data acquire-cancel --workspace-id WORKSPACE_ID --crawler-session-id CRAWLER_SESSION_ID --hosted	Cancel one still-active public acquisition using the coordinate acquire printed. By default it writes a private, per-coordinate receipt journal under the local mr-data state directory; a transport-loss retry reuses its idempotency key, and success prints Studio's exact cancellation receipt. Use --journal FILE only to override that location.
mr-data approve --hosted / mr-data recipe-approve --hosted	Record the decision you are asking for against one approval Studio holds, print the address where the person who must grant it does so, then wait and read the decision receipt back. The command never decides, and exits 2 unless a human recorded the decision that was asked for.
mr-data note --run RUN_ID --hosted	Append one cell of the operator-facing decision record to that run's own log, where the rendered notebook and the live view both read it: --heading is its one outcome-led line, the body arrives on standard input or from --markdown-file, and --checkpoint places it against the run-event it explains. Writing the same --cell-id again revises the cell in place. A cell is unsealed prose and is never attestation evidence. --list reads them all back.
mr-data research-open --dataset DATASET_ID --question QUESTION_ID --source SOURCE_ID	Open one research session and print each state as its worker warms up: opening, warming, ready. A workspace already holding its bound of live sessions is refused with the retry window Studio stated, never waited through.
mr-data research-probe SESSION_ID --kind KIND --request JSON	Ask the session's warm worker one question about a source, checked against the probe vocabulary before it is sent, follow the session run's stream until the probe settles, and print the answer read from the probe record. Exits 0 only for an answered probe; a probe answer is exploration and never evidence.
mr-data research-status (SESSION_ID | --run RUN_ID) [--probe PROBE_ID]	Report Studio's own reading of one session, and of one probe with its answer. --run resolves the owning session directly in Studio; it never scans sessions client-side.
mr-data research-close (SESSION_ID | --run RUN_ID)	Close the session, cancelling any probe still queued, so its worker exits. --run asks Studio to resolve and close the owning session directly.

Current local workflow

The supported local workflow is:

question and output grain
  -> selected sources and rights evidence
  -> declarative dataset plan
  -> confined source snapshots
  -> deterministic transform and validation
  -> immutable Parquet Build
  -> two assigned reviews
  -> Checker-signed decision
  -> reviewed_candidate | fixes_required
  -> Studio source staging and immutable Recipe proposal
  -> authenticated Editor confirmation of the internal approval record
  -> Dataset activation Run and Cloud Current binding

For the controlled acceptance walkthrough that starts from a changing synthetic public CSV and, after the hosted rollout is separately marked ready, ends at Cloud Current after a scheduled Dataset Version, see Local Build to Cloud refresh demo. Local authoring does not require Docker; governed Linux execution remains in Studio.

The local engine supports string trimming, exact empty-string-to-null conversion, collision-safe rename, closed-type casts, stable left equality joins, output selection, and deterministic quality checks. local-plan.v2 runs a wider closed set of ordered operations over the same shared arithmetic: it adds exact decimal division, wide-to-long reshaping of columns the plan names, inner and anti joins, and bounded text normalization for keys that will not otherwise match. Every one of those is admitted by a written test that separates cleaning from modeling choices, and the same test is what refuses a general window, a filled-in value, and the reverse of the reshape. See Transforms a plan may perform.

local-plan.v2 also carries one retired operation, prediction_label@1.0.0. It named a value at a future date, and the horizon it required is a modeling choice rather than a property of the sources, so no newly authored plan may name it: authoring, plan checking, validation, export and approval all refuse one that does. It remains readable so that a Build sealed with it still verifies and a Recipe approved with it still runs. See Prediction labels.

The installed stock source-to-Reader path supports certified Readers for local files, credential-free public HTTPS sources, and bounded ordered public HTTPS collections. Each HTTPS retrieval, Reader decode, and parse shares one attested clean-room operation; JSON and NDJSON projection, ZIP, TAR, GZIP, XLSX, delimited text, and single-record GRIB2 are available under their exact Reader settings and budgets. A collection can combine 1–16 explicit objects or a fixed page-number range when every member uses the same Reader and normalized schema; see Deterministic source collections.

The review runtime handles initial Builds only. It records whether the Build passed review or needs fixes, rejects nonzero failed-cycle counts, and does not verify repaired successors or derive a human-direction status.

The package also contains a pure canonical empirical source-observation chain and deterministic cadence inference with shared cross-system vectors. Both governed paths record that evidence through one emitter: a hosted probe submits it to Studio, and mr-data acquire --cadence-history records a local probe in an append-only folder. For one acquisition outcome the two produce the same document byte for byte. It does not yet schedule probes or claim that a manually selected cron was learned; Studio owns persistence and adaptive scheduling.

The weather Reader opens one extracted GRIB2 record. It refuses a full GFS or HRRR file containing a collection of hundreds of records.

Recipes

frozen-recipe.v1 binds the question, requirements, selected source evidence, connector versions, query templates, credential-reference names, transformation plan, validation policy, Bookmark strategy, and engine versions. Its digest is lowercase SHA-256.

The checked-in example contains recipe.json and recipe-execution.json:

uv sync --locked --extra dev
mkdir -p .work/recipe
uv run --extra dev mr-data recipe-export \
  --recipe examples/city-temperature/recipe.json \
  --output .work/recipe/frozen-recipe.json
uv run --extra dev mr-data recipe-approve \
  --recipe .work/recipe/frozen-recipe.json \
  --approval-id approval-city-temperature-v1 \
  --approved-by local-operator \
  --approved-at 2026-07-19T12:00:00Z \
  --approval-policy-digest 0000000000000000000000000000000000000000000000000000000000000000 \
  --output .work/recipe/recipe-approval.json
uv run --extra dev mr-data recipe-run \
  --recipe .work/recipe/frozen-recipe.json \
  --approval .work/recipe/recipe-approval.json \
  --execution examples/city-temperature/recipe-execution.json \
  --input-root examples/city-temperature \
  --output .work/recipe/version-1 \
  --producer-principal local-recipe-producer \
  --producer-attempt initial-v1
uv run --extra dev mr-data recipe-verify .work/recipe/version-1

The output parent must exist. Export, approval, and run commands do not overwrite existing output. Refresh and backfill executions require --previous-run-dir and the exact predecessor Build.

Authority and security

• External agents may interpret questions, propose sources, author declarative inputs, and report findings. They do not calculate authoritative metrics or approve releases.
• The deterministic worker owns confinement, budgets, snapshots, transforms, checks, lineage, hashes, and artifact sealing.
• The trusted coordinator owns roots, assignments, attempt identities, enrolled signing keys, and immutable cycle lineage.
• The Harness owns deterministic public-catalog provider semantics, including Data.gov endpoint, pagination, catalog identity, revisions, normalization, and provenance. Catalog discovery never authorizes or acquires the underlying dataset resource.
• Studio owns hosted contracts, orchestration, persistence, scheduling, credentials, approval, and release.

Source bytes and dataset plans are untrusted. The local-file worker accepts normalized relative paths to regular, single-link CSV files below an approved input root. It rejects traversal, absolute paths, symlinks, hardlinks, special files, input/output overlap, source mutation, and budget violations.

Builds are content-addressed and non-overwritable. Canonical payloads exclude ambient clocks and absolute filesystem paths. Model output and source content cannot grant human approval.

The package does not provide local publication authority, local release authority, modeling, general feature engineering, prediction serving, authenticated source access, scheduling, or arbitrary code execution from an agent response. The one narrow exception to general feature engineering is deterministic, terminal-only prediction-label construction described above.

Hosted workers

mr-data export-hosted-candidate verifies a local Build and writes the canonical hosted Builder-input envelope:

mr-data export-hosted-candidate /absolute/path/to/run \
  --output /absolute/path/to/new-candidate-input.json \
  --json

The hosted executables are:

• mr-data-bootstrap: exchanges workload identity for one Studio job and starts the fixed role.
• mr-data-producer: reconstructs and verifies a Build, then uploads reservation-bound artifacts.
• mr-data-verifier: independently verifies artifacts and submits a signed report.
• mr-data-crawler-job: performs allowlisted public acquisition and passes bounded bytes to a networkless Clean room.
• mr-data-ingestion-job: redeems one opaque workload-identity-bound ingestion dispatch and runs only a reviewed bounded delivery processor; see Hosted ingestion delivery.

These commands cannot select or release a Build, create a dataset version, advance latest, or access Studio databases, queues, secrets, or storage metadata directly.

Verification

For ordinary edit-loop feedback, run make test-fast. This is a non-authoritative test tier that omits semantically expensive scale, isolated-install, and live-repository-scan coverage; make verify is required before review or merge.

uv sync --extra dev
make verify-offline-slice
make verify-hosted-workers
make verify-viewer
make verify-viewer-browser
make verify

verify-hosted-workers requires a sibling mostlyright-studio checkout. Set STUDIO_ROOT=/absolute/path when it is elsewhere.

make verify-thin-client proves the hosted profile the way a stranger meets it: it builds the base wheel, installs it into a fresh environment from an empty cache, refuses any compiled extension it finds there, and runs the hosted command surface. The same body runs in CI as the reusable thin-client-smoke workflow, fanned out over GitHub-hosted ubuntu, macOS and Windows: one py3-none-any wheel is built once and installed by all three legs, on every pull request and nightly.

There is no documented known-failing set; every failing node is a verification failure.

Documentation

• Architecture
• Offline Vertical Slice
• Recipes
• Sources and Acquisition
• Empirical source cadence
• Readers
• Deterministic source collections
• Notebook View
• Deployment request
• Local Build to Cloud refresh demo
• Hosted parity
• Catalogue release channel
• Hosted workers
• Hosted worker publication
• Research-session worker
• Transforms a plan may perform
• Review policy
• Contract ownership
• Engine governance

.planning/ contains repository work records. Its contents are not product contracts, merge authorization, approval, or release evidence.