rawctx 0.3.13

rawctx CLI scaffold

rawctx CLI

Python CLI and SDK for rawctx Hub.

Guides:

• ../docs/guides/package-workflow.md
• ../docs/guides/metricflow-native-workflow.md
• ../docs/diff.md

Commands

User:

• rawctx login [--registry URL] [--id-token JWT] [--token-name NAME] [--expires-in-days N] [--no-browser] [--json]
• rawctx logout [--local-only] [--json]
• rawctx search [QUERY] [--format F] [--source-format F] [--origin all|native|indexed] [--domain D] [--source S] [--tags CSV] [--sort recent|name] [--page N] [--size N] [--json] [--offline] [--registry URL]
• rawctx info PACKAGE_REF [--json] [--offline] [--registry URL]
• rawctx download PACKAGE_REF MODEL_PATH [--local-dir DIR] [--stdout] [--offline] [--force] [--json] [--registry URL]
• rawctx snapshot-download PACKAGE_REF [--local-dir DIR] [--offline] [--force] [--json] [--registry URL]
• rawctx validate [TARGET] [--format auto|manifest|osi] [--show-dataset-measures] [--json]
• rawctx pack [TARGET_DIR] [--output-dir DIR] [--json]
• rawctx convert --from metricflow --to osi INPUT_PATH --output DIR [--package-name @scope/name] [--package-version X.Y.Z] [--overwrite] [--json]
• rawctx publish [TARGET_DIR] [--registry URL]
• rawctx publish --from-dbt DBT_PROJECT_DIR [--native] [--emit-package DIR] [--package-name @scope/name] [--package-version X.Y.Z] [--registry URL]
• rawctx diff A B [--format text|json|github|markdown|junit] [--consumer sql|python|llm|all] [--severity breaking|behavioral|cosmetic|all] [--exit-code-on breaking|behavioral|none] [--output PATH]
• rawctx diff semantic A B [--format text|json|github|markdown|junit] [--consumer sql|python|llm|all] [--severity breaking|behavioral|cosmetic|all]
• rawctx diff prompt A B [--format text|json|github|markdown|junit] [--max-tokens N]
• rawctx diff eval A B --questions FILE [--format text|json|github|markdown|junit] [--runs N] [--model NAME]

Maintainer:

• rawctx claim PACKAGE_REF [--json] [--registry URL]

Ops:

• rawctx index dbt --seed-file PATH [--only owner/name] [--limit N] [--dry-run] [--json] [--registry URL]
• rawctx index git --repo owner/name --source-ref REF --package-version X.Y.Z [--package-name NAME] [--scope SCOPE] [--model-glob GLOB ...] [--dry-run] [--json] [--registry URL]

Supported Package Lanes

rawctx currently supports two published package formats:

• format=osi: packaged OSI YAML files
• format=metricflow: native MetricFlow/dbt snapshot packages

Both lanes support:

• rawctx info
• rawctx snapshot-download
• rawctx.load()
• rawctx.to_prompt()
• rawctx diff

download PACKAGE_REF MODEL_PATH also works for both lanes, but only for files listed in manifest.models.

rawctx diff accepts three artifact inputs:

• @scope/name@version
• local package directories
• .rawctx.tar.gz archives

It compares artifacts only. It never queries a warehouse.

Compare Packages

Use rawctx diff when you need semantic-level change review instead of raw file diffs.

rawctx diff ./pkg-v1 ./pkg-v2
rawctx diff semantic ./pkg-v1 ./pkg-v2 --format json
rawctx diff prompt ./pkg-v1 ./pkg-v2 --max-tokens 2000
rawctx diff eval ./pkg-v1 ./pkg-v2 --questions questions.jsonl --runs 5 --model mock

The top-level command runs semantic + prompt. eval stays opt-in because it measures model behavior, not deterministic package structure.

Notebook / Code

Search uses the public Hub index first so CLI and SDK results match the logged-out web experience. If a search returns no public matches and you have a token configured, rawctx retries with authenticated search.

Notebook shell style:

!rawctx search "semantic model" --sort recent --json
!rawctx info @scope/name --json
!rawctx snapshot-download @scope/name --json
!rawctx download @scope/name models/customers.yml --json
!rawctx validate ./my-package --json

Python API:

import rawctx

result = rawctx.search("semantic model", registry="https://api.rawctx.dev", sort="recent")
pkg = rawctx.info("@scope/name", registry="https://api.rawctx.dev")
model = rawctx.load("@scope/name", registry="https://api.rawctx.dev")
prompt = rawctx.to_prompt(
    "@scope/name",
    datasets=["customers", "order_item"],
    max_tokens=2000,
    registry="https://api.rawctx.dev",
)

print(model.format_name)    # "osi" or "metricflow"
print(model.datasets)       # normalized dataset names
print(model.measures)       # [Measure(name="...", ...)]
print(model.dimensions)     # [Dimension(name="...", ...)]
print(model.relationships)  # [Relationship(name="...", ...)]
print(prompt)
print(pkg["model_paths"])

snapshot_dir = rawctx.snapshot_download("@scope/name", registry="https://api.rawctx.dev")
model_path = rawctx.download("@scope/name", "models/customers.yml", registry="https://api.rawctx.dev")
validation = rawctx.validate("./my-package")
semantic = rawctx.semantic_diff("./pkg-v1", "./pkg-v2")
prompt_diff = rawctx.prompt_diff("./pkg-v1", "./pkg-v2", max_tokens=2000)
combined = rawctx.diff_artifacts("./pkg-v1", "./pkg-v2")

Async Python API:

import asyncio
import rawctx

async def main():
    async with rawctx.AsyncRawctxClient(registry="https://api.rawctx.dev") as client:
        result = await client.search("semantic model", sort="recent")
        model = await client.load("@scope/name")
        prompt = await client.to_prompt("@scope/name", datasets=["customers", "order_item"])
        snapshot_dir = await client.snapshot_download("@scope/name")
        diff_report = await client.diff("./pkg-v1", "./pkg-v2")
        return result, model, prompt, snapshot_dir, diff_report

asyncio.run(main())

to_prompt() Behavior

rawctx.to_prompt() turns a package snapshot into compact LLM context. The top-level helper opens a short-lived client, then the client:

1. calls load() to materialize the snapshot and normalize OSI or native MetricFlow into LoadedPackage
2. calls info() to read package metadata such as domain
3. passes the loaded package, package info, datasets, and max_tokens into the prompt builder

The rendered prompt uses this section order:

Domain: {domain} ({package_name})

Models:
...

Datasets:
...

Metrics:
...

Relationships:
...

Dataset filters preserve the requested order, drop duplicates, and fail with UsageError: Unknown dataset(s): ... when a requested dataset is not present. Selecting a subset keeps metrics for those datasets and relationships touching those datasets; relationships between selected datasets are listed before relationships to external datasets.

max_tokens is an approximate cap based on ceil(len(text) / 4), not a model tokenizer. When the full prompt is too large, rawctx progressively removes detail: detailed ai_context and expressions, long measure/dimension lists, model and metric sections, trailing relationships, verbose dataset labels, then finally falls back to a minimal dataset-and-relationship prompt.

Download Behavior

• download fetches one file listed in manifest.models
• snapshot-download materializes the full extracted package tree
• for native MetricFlow packages, snapshot-download is the primary handoff because it restores the full dbt-style snapshot
• load() and to_prompt() normalize both OSI and native MetricFlow packages into the same typed Python structures
• when using snapshot-download --local-dir, prefer a new or empty directory. --force only replaces an existing rawctx snapshot directory and refuses to wipe the current working directory or unrelated folders
• indexed packages remain preview-only and cannot be downloaded directly

Validate / Pack / Publish

validate, pack, and publish all start from a local package directory.

• validate: checks the manifest and validates the package according to manifest.format
• pack: builds a deterministic local .rawctx.tar.gz
• publish: validates again, rebuilds a temporary archive, calculates the checksum, uploads bytes, and completes the version

Package directories are no longer OSI-only.

OSI package example:

my-osi-package/
  rawctx.yaml
  README.md
  models/
    sales_summary.osi.yaml
    customers.osi.yaml

Native MetricFlow package example:

my-metricflow-package/
  rawctx.yaml
  README.md
  dbt_project.yml
  models/
    customers.yml
    orders.yml

Native MetricFlow manifest example:

name: "@demo/jaffle-metrics"
version: "1.0.0"
format: "metricflow"
source_format: "metricflow"
description: "Native MetricFlow package"
models:
  - models/customers.yml
  - models/orders.yml
include:
  - dbt_project.yml
repository: "https://github.com/dbt-labs/jaffle-sl-template"

Notes:

• format supports osi and metricflow
• models must stay relative and must resolve inside the package directory
• include is optional and is mainly useful for native packages that need extra project files such as dbt_project.yml
• standalone file validation is still limited to manifest files and OSI files, so rawctx validate models/customers.yml is not a native MetricFlow file validator by itself

Convert Workflow

Inspect-first OSI flow:

rawctx convert --from metricflow --to osi ./my-dbt-project --output ./dist/pkg
rawctx validate ./dist/pkg --json
rawctx pack ./dist/pkg --output-dir ./dist --json

Publish Directly From dbt

Convert to OSI and publish:

rawctx login
rawctx publish --from-dbt ./my-dbt-project --emit-package ./dist/pkg

Publish a native MetricFlow package:

rawctx login
rawctx publish --from-dbt ./my-dbt-project --native --emit-package ./dist/native-pkg
rawctx publish --from-dbt ./my-dbt-project --native --package-name @your-scope/jaffle-shop --package-version 1.2.3

Use --emit-package when you want the generated package directory to remain on disk after the publish run.

Auth Flow (Auto + Fallback)

1. Run rawctx login.
2. CLI opens or prints the OAuth URL from POST /api/auth/login and falls back to the legacy GitHub endpoint if needed.
3. Complete login in the browser.
4. CLI automatically polls OAuth session status and captures id_token when the registry supports it.
5. CLI calls POST /api/auth/token and stores the API token in ~/.rawctx/config.yaml.

Manual fallback:

• rawctx login --id-token '<JWT>'

Config and Environment

Config file (default): ~/.rawctx/config.yaml

registry: "https://api.rawctx.dev"
auth:
  token: "rxctx_..."
  token_id: "uuid"
  token_name: "rawctx-cli"
  issued_at: "2026-02-28T00:00:00+00:00"
profile:
  username: "owner"

Environment overrides:

• RAWCTX_CONFIG (config path)
• RAWCTX_REGISTRY (registry URL)
• RAWCTX_TOKEN (auth token)

Priority: CLI option > env var > config > default.

Offline Mode

--offline is supported for:

• search
• info
• download
• snapshot-download

Cache paths:

• index: ~/.rawctx/cache/packages.json
• archives: ~/.rawctx/cache/archives/@scope/name/<version>.rawctx.tar.gz
• snapshots: ~/.rawctx/packages/@scope/name/<version>/