rawctx 0.3.18

rawctx CLI scaffold

rawctx CLI

Python CLI and SDK for rawctx Hub.

Guides:

• ../docs/guides/package-workflow.md
• ../docs/guides/metricflow-native-workflow.md
• ../docs/diff.md

Commands

User:

• rawctx login [--registry URL] [--id-token JWT] [--token-name NAME] [--expires-in-days N] [--no-browser] [--json]
• rawctx logout [--local-only] [--json]
• rawctx search [QUERY] [--format F] [--source-format F] [--origin all|native|indexed] [--domain D] [--source S] [--tags CSV] [--sort recent|name] [--page N] [--size N] [--json] [--offline] [--registry URL]
• rawctx info PACKAGE_REF [--json] [--offline] [--registry URL]
• rawctx download PACKAGE_REF MODEL_PATH [--local-dir DIR] [--stdout] [--offline] [--force] [--json] [--registry URL]
• rawctx snapshot-download PACKAGE_REF [--local-dir DIR] [--offline] [--force] [--json] [--registry URL]
• rawctx to-prompt PACKAGE_REF [--datasets CSV] [--max-tokens N] [--offline] [--mode agent_context|strict_metric] [--metrics CSV] [--question TEXT] [--budget-policy compact|error_if_required_missing] [--render-format text|xml] [--json] [--registry URL]
• rawctx validate [TARGET] [--format auto|manifest|osi] [--show-dataset-measures] [--json]
• rawctx pack [TARGET_DIR] [--output-dir DIR] [--json]
• rawctx convert --from metricflow --to osi INPUT_PATH --output DIR [--package-name @scope/name] [--package-version X.Y.Z] [--overwrite] [--json]
• rawctx publish [TARGET_DIR] [--private] [--org ORG] [--registry URL]
• rawctx publish --from-dbt DBT_PROJECT_DIR [--native] [--emit-package DIR] [--package-name @scope/name] [--package-version X.Y.Z] [--private] [--org ORG] [--registry URL]
• rawctx diff A B [--format text|json|github|markdown|junit] [--consumer sql|python|llm|all] [--severity breaking|behavioral|cosmetic|all] [--exit-code-on breaking|behavioral|none] [--max-tokens N] [--output PATH]
• rawctx diff semantic A B [--format text|json|github|markdown|junit] [--consumer sql|python|llm|all] [--severity breaking|behavioral|cosmetic|all]
• rawctx diff prompt A B [--format text|json|github|markdown|junit] [--max-tokens N]
• rawctx diff eval A B --questions FILE [--format text|json|github|markdown|junit] [--runs N] [--model NAME]

Maintainer:

• rawctx claim PACKAGE_REF [--json] [--registry URL]

Ops:

• rawctx index dbt --seed-file PATH [--only owner/name] [--limit N] [--dry-run] [--json] [--registry URL]
• rawctx index git --repo owner/name --source-ref REF --package-version X.Y.Z [--package-name NAME] [--scope SCOPE] [--model-glob GLOB ...] [--dry-run] [--json] [--registry URL]

Supported Package Lanes

rawctx currently supports two published package formats:

• format=osi: packaged OSI YAML files
• format=metricflow: native MetricFlow/dbt snapshot packages

Both lanes support:

• rawctx info
• rawctx snapshot-download
• rawctx.load()
• rawctx.to_prompt()
• rawctx diff

download PACKAGE_REF MODEL_PATH also works for both lanes, but only for files listed in manifest.models.

rawctx diff accepts three artifact inputs:

• @scope/name@version
• local package directories
• .rawctx.tar.gz archives

It compares artifacts only. It never queries a warehouse.

Compare Packages

Use rawctx diff when you need semantic-level change review instead of raw file diffs.

rawctx diff ./pkg-v1 ./pkg-v2
rawctx diff semantic ./pkg-v1 ./pkg-v2 --format json
rawctx diff prompt ./pkg-v1 ./pkg-v2 --max-tokens 2000
rawctx diff eval ./pkg-v1 ./pkg-v2 --questions questions.jsonl --runs 5 --model mock

The top-level command runs semantic + prompt. eval stays opt-in because it measures model behavior, not deterministic package structure.

Notebook / Code

Search uses the public Hub index first so CLI and SDK results match the logged-out web experience. If a search returns no public matches and you have a token configured, rawctx retries with authenticated search.

Notebook shell style:

!rawctx search "semantic model" --sort recent --json
!rawctx info @scope/name --json
!rawctx snapshot-download @scope/name --json
!rawctx download @scope/name models/customers.yml --json
!rawctx to-prompt @scope/name --datasets customers,order_item --max-tokens 2000
!rawctx validate ./my-package --json

Python API:

import rawctx

result = rawctx.search("semantic model", registry="https://api.rawctx.dev", sort="recent")
pkg = rawctx.info("@scope/name", registry="https://api.rawctx.dev")
model = rawctx.load("@scope/name", registry="https://api.rawctx.dev")
prompt = rawctx.to_prompt(
    "@scope/name",
    datasets=["customers", "order_item"],
    max_tokens=2000,
    registry="https://api.rawctx.dev",
)

print(model.format_name)    # "osi" or "metricflow"
print(model.datasets)       # normalized dataset names
print(model.measures)       # [Measure(name="...", ...)]
print(model.dimensions)     # [Dimension(name="...", ...)]
print(model.relationships)  # [Relationship(name="...", ...)]
print(prompt)
print(pkg["model_paths"])

snapshot_dir = rawctx.snapshot_download("@scope/name", registry="https://api.rawctx.dev")
model_path = rawctx.download("@scope/name", "models/customers.yml", registry="https://api.rawctx.dev")
validation = rawctx.validate("./my-package")
semantic = rawctx.semantic_diff("./pkg-v1", "./pkg-v2")
prompt_diff = rawctx.prompt_diff("./pkg-v1", "./pkg-v2", max_tokens=2000)
combined = rawctx.diff_artifacts("./pkg-v1", "./pkg-v2")

Async Python API:

import asyncio
import rawctx

async def main():
    async with rawctx.AsyncRawctxClient(registry="https://api.rawctx.dev") as client:
        result = await client.search("semantic model", sort="recent")
        model = await client.load("@scope/name")
        prompt = await client.to_prompt("@scope/name", datasets=["customers", "order_item"])
        snapshot_dir = await client.snapshot_download("@scope/name")
        diff_report = await client.diff("./pkg-v1", "./pkg-v2")
        return result, model, prompt, snapshot_dir, diff_report

asyncio.run(main())

to_prompt() Behavior

rawctx.to_prompt() turns a package snapshot into compact LLM context. It uses the same normalized semantic objects as load(), then applies package metadata, dataset filters, and prompt budget settings to render agent-ready text.

The same prompt compiler is available from the CLI:

rawctx to-prompt @scope/name --datasets customers,order_item --max-tokens 2000
rawctx to-prompt @scope/name --mode strict_metric --metrics mrr --render-format xml
rawctx to-prompt @scope/name --json

The rendered prompt keeps a predictable section shape:

Domain: {domain} ({package_name})

Models:
...

Datasets:
...

Metrics:
...

Relationships:
...

Dataset filters preserve the requested order, drop duplicates, and fail with UsageError: Unknown dataset(s): ... when a requested dataset is not present. Selecting a subset keeps context focused on the requested datasets and their relevant relationships.

max_tokens is a practical size target, not a model-specific tokenizer guarantee. When the budget is tight, rawctx prioritizes high-signal semantic context and compacts lower-priority detail. Use return_context=True when you need the selected objects, estimated size, render hash, omissions, and warnings for logging or review.

Download Behavior

• download fetches one file listed in manifest.models
• snapshot-download materializes the full extracted package tree
• for native MetricFlow packages, snapshot-download is the primary handoff because it restores the full dbt-style snapshot
• load() and to_prompt() normalize both OSI and native MetricFlow packages into the same typed Python structures
• when using snapshot-download --local-dir, prefer a new or empty directory. --force only replaces an existing rawctx snapshot directory and refuses to wipe the current working directory or unrelated folders
• indexed packages remain preview-only and cannot be downloaded directly

Validate / Pack / Publish

validate, pack, and publish all start from a local package directory.

• validate: checks the manifest and validates the package according to manifest.format
• pack: builds a deterministic local .rawctx.tar.gz
• publish: validates again, rebuilds a temporary archive, calculates the checksum, uploads bytes, and completes the version

Package directories are no longer OSI-only.

OSI package example:

my-osi-package/
  rawctx.yaml
  README.md
  models/
    sales_summary.osi.yaml
    customers.osi.yaml

Native MetricFlow package example:

my-metricflow-package/
  rawctx.yaml
  README.md
  dbt_project.yml
  models/
    customers.yml
    orders.yml

Native MetricFlow manifest example:

name: "@demo/jaffle-metrics"
version: "1.0.0"
format: "metricflow"
source_format: "metricflow"
description: "Native MetricFlow package"
models:
  - models/customers.yml
  - models/orders.yml
include:
  - dbt_project.yml
repository: "https://github.com/dbt-labs/jaffle-sl-template"

Notes:

• format supports osi and metricflow
• models must stay relative and must resolve inside the package directory
• include is optional and is mainly useful for native packages that need extra project files such as dbt_project.yml
• standalone file validation is still limited to manifest files and OSI files, so rawctx validate models/customers.yml is not a native MetricFlow file validator by itself

Convert Workflow

Inspect-first OSI flow:

rawctx convert --from metricflow --to osi ./my-dbt-project --output ./dist/pkg
rawctx validate ./dist/pkg --json
rawctx pack ./dist/pkg --output-dir ./dist --json

Publish Directly From dbt

Convert to OSI and publish:

rawctx login
rawctx publish --from-dbt ./my-dbt-project --emit-package ./dist/pkg

Publish a native MetricFlow package:

rawctx login
rawctx publish --from-dbt ./my-dbt-project --native --emit-package ./dist/native-pkg
rawctx publish --from-dbt ./my-dbt-project --native --package-name @your-scope/jaffle-shop --package-version 1.2.3

Use --emit-package when you want the generated package directory to remain on disk after the publish run.

Auth Flow (Auto + Fallback)

1. Run rawctx login.
2. CLI opens or prints the OAuth URL from POST /api/auth/login and falls back to the legacy GitHub endpoint if needed.
3. Complete login in the browser.
4. CLI automatically polls OAuth session status and captures id_token when the registry supports it.
5. CLI calls POST /api/auth/token and stores the API token in ~/.rawctx/config.yaml.

Manual fallback:

• rawctx login --id-token '<JWT>'

Config and Environment

Config file (default): ~/.rawctx/config.yaml

registry: "https://api.rawctx.dev"
auth:
  token: "rxctx_..."
  token_id: "uuid"
  token_name: "rawctx-cli"
  issued_at: "2026-02-28T00:00:00+00:00"
profile:
  username: "owner"

Environment overrides:

• RAWCTX_CONFIG (config path)
• RAWCTX_REGISTRY (registry URL)
• RAWCTX_TOKEN (auth token)

Priority: CLI option > env var > config > default.

Offline Mode

--offline is supported for:

• search
• info
• download
• snapshot-download

Cache paths:

• index: ~/.rawctx/cache/packages.json
• archives: ~/.rawctx/cache/archives/@scope/name/<version>.rawctx.tar.gz
• snapshots: ~/.rawctx/packages/@scope/name/<version>/