Cosmos Semantic

Repository: cosmos-semantic-platform. License: open core, with the core under Apache 2.0 (ADR 0005).

A semantic layer for AI agents and applications, with its own engine: its own versioned YAML format with the dbt Semantic Layer vocabulary and a JSON Schema, SQL generation for each data source, and native interoperability with Apache Ossie (formerly Open Semantic Interchange, OSI). Cube and dbt are import sources and export targets, not dependencies.

Status: phases 0 to 3 are implemented: the engine, the ontology (glossary and business rules), the REST API, the MCP server, OIDC identity, verifiable numbers (evidence, replay, reconciliation, impact, bootstrap) and business context (rule versions, usage and cost, knowledge graph, Power BI import, AI-assisted discovery, the retail pack, SKOS export). The whole test suite passes on DuckDB, Postgres, SQL Server and Databricks. See the roadmap.

In one sentence

Define entities, metrics, business concepts and business rules once, and any AI agent or application queries the same number, with SQL generated and run directly on your data source, and the model portable to Ossie, Cube and dbt.

Quickstart

pip install "cosmos-semantic[duckdb,testdata]"
cosmos init demo && cd demo
cosmos demo-data
cosmos query revenue average_order_value --by orders.status --evidence
cosmos explain rule:active_customer

That is a governed metric with its evidence (formula, rules, policies, as_of and hashes) in about a minute, on a local DuckDB file. The quickstart continues with business rules, row filters, personal data and connecting an AI agent over MCP. Extras: duckdb, postgres, sqlserver, databricks, server (REST and MCP), ai (cosmos suggest), testdata.

Development

Requires Python 3.11+ and uv. From a clone of this repository:

uv sync --all-extras

# Generate synthetic e-commerce data into a local DuckDB (data/ecommerce.duckdb)
uv run cosmos demo-data -c examples/ecommerce/connections.yml   # --locale pt_BR for Brazilian data with CPF

# Validate the example model
uv run cosmos validate -m examples/ecommerce/semantic

# Revenue and average order value by status
uv run cosmos query revenue average_order_value --by orders.status \
  -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# Metrics from different models, with the user's row filter; --show-sql prints the generated SQL
uv run cosmos query revenue paid_value customer_count --by customers.region -u region=Midwest --show-sql \
  -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# Business rules as a filter and as a grouping; --as-of pins "today"
uv run cosmos query customer_count --where "{rule.active_customer}" --by customers.segment \
  -u region=Northeast --as-of 2026-10-22 -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# Evidence envelope: formulas, rules, policies, as_of and hashes; then replay the audited query
export COSMOS_AUDIT_LOG=data/audit.jsonl
uv run cosmos query average_order_value --by customers.segment --where "{rule.active_customer}"   -u region=Northeast --evidence -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml
uv run cosmos replay <query_id> -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# Reconcile a governed metric with legacy SQL or another model, and explain the divergence
uv run cosmos reconcile revenue --by orders.status --sql legacy_revenue.sql   -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml
uv run cosmos reconcile --cases examples/ecommerce/divergence_zoo.yml   -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# What a change affects, and who queried it (also: --changes-from <previous model>)
uv run cosmos impact rule:active_customer -m examples/ecommerce/semantic --audit data/audit.jsonl

# Past numbers under the rule in effect then, or under today's rule
uv run cosmos query customer_count -w "{rule.active_customer}" --as-of 2026-06-30 --rules-as-of 2026-10-23   -u region=Midwest -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# Usage and estimated cost per metric, user, agent and data source (rates: `cost` in connections.yml)
uv run cosmos usage --audit data/audit.jsonl -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml

# The business knowledge graph: owners, source systems and processes (Mermaid, DOT or JSON)
uv run cosmos graph --focus process:order_to_cash -m examples/ecommerce/semantic
uv run cosmos impact system:payment_gateway -m examples/ecommerce/semantic

# AI-assisted discovery: concepts, rules, metrics and descriptions, validated by the engine, for review
uv run cosmos suggest --source local -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml -o suggestions.yml

# Industry pack mapped onto your tables by name (draft + report for review)
uv run cosmos pack list
uv run cosmos pack apply retail --source local --schema public -c examples/ecommerce/connections.yml -o semantic/retail.yml

# The business glossary as SKOS/RDF for data catalogs (turtle, jsonld or ntriples)
uv run cosmos export skos -m examples/ecommerce/semantic -o glossary.ttl --base https://data.example.com/glossary/

# Draft a model from the database catalog (keys, joins, measures, LGPD suggestions), then review it
uv run cosmos bootstrap -c examples/ecommerce/connections.yml -o semantic/draft.yml

# Business rule examples (on DuckDB, without touching the data sources) and reference queries
uv run cosmos test -m examples/ecommerce/semantic
uv run cosmos reference

# Import models from other tools
uv run cosmos import cube path/to/model -o semantic/imported.yml
uv run cosmos import dbt path/to/dbt/models --schema analytics -o semantic/dbt.yml
uv run cosmos import ossie model.yaml -o semantic/ossie.yml
uv run cosmos import powerbi model.bim -o semantic/powerbi.yml   # or a TMDL folder

REST API and MCP

cosmos serve starts the REST API at /v1 and the MCP server at /mcp in the same process; interactive docs are at /docs.

# Production: tokens from the OIDC provider (Cosmos only validates, never issues)
export COSMOS_OIDC_ISSUER=https://login.microsoftonline.com/<tenant>/v2.0
export COSMOS_OIDC_AUDIENCE=api://cosmos-semantic
# Development: API keys bound to a fixed context (a file with `keys: {<key>: {subject, user}}`)
export COSMOS_DEV_API_KEYS=dev-keys.yml

uv run cosmos serve -m examples/ecommerce/semantic -c examples/ecommerce/connections.yml
curl -X POST localhost:8080/v1/query -H "X-API-Key: <key>" -H "content-type: application/json" \
  -d '{"metrics": ["revenue"], "dimensions": ["customers.segment"]}'

Remote MCP clients use /mcp as an OAuth protected resource: requests without a valid token get HTTP 401 with a WWW-Authenticate header pointing to /.well-known/oauth-protected-resource, which names the identity provider (set COSMOS_PUBLIC_URL behind a proxy).

For local MCP clients (Claude Desktop, Claude Code), use the stdio transport: cosmos mcp -m ... -c ... with COSMOS_API_KEY set. The tools are search_concepts, explain, list_members, query and show_sql. Every call runs as the user who owns the token; personal data never reaches agents.

Data sources

Local Postgres: docker compose -p cosmos-semantic -f deploy/docker-compose.yml up -d, then cosmos demo-data --target postgres and cosmos reference --source postgres. To also run the tests on Postgres, set COSMOS_PG_URI=postgresql://cosmos:cosmos@localhost:15432/cosmos.

Local SQL Server: docker compose -p cosmos-semantic -f deploy/docker-compose.yml --profile sqlserver up -d and COSMOS_MSSQL_URI='mssql+pymssql://sa:Cosmos!Passw0rd@localhost:14330/master'. Every data source accepts a schema_map in connections.yml (for example public: dbo) to use the same model with different schemas.

Databricks: set DATABRICKS_HTTP_PATH (SQL warehouse) and DATABRICKS_TOKEN with DATABRICKS_SERVER_HOSTNAME, or DATABRICKS_CONFIG_PROFILE with a Databricks CLI profile (databricks auth login --profile <name>). The tests load the example data into the catalog in DATABRICKS_CATALOG (default cosmos_demo, which must exist). On Git Bash for Windows, also set MSYS_NO_PATHCONV=1, or the shell rewrites /sql/1.0/... into a Windows path and the warehouse answers 404.

Editors validate and autocomplete model files through the JSON Schema in schema/, referenced on the first line of each file (# yaml-language-server: $schema=...). After changing spec.py, regenerate it with uv run cosmos schema -o schema/cosmos-semantic.v1.schema.json.

Principles

1. The engine is ours. The semantic model, metric planner and SQL generation are the core of the product.
2. Compatible, not dependent. Cube, dbt and Ossie are import and export formats.
3. Query the source directly. No intermediate infrastructure in the MVP; caching comes later.
4. Same number everywhere. The same request returns the same result on every supported database.
5. One differentiator per phase. Scope only grows after it is validated.
6. Open core. Everything that defines or computes a number is Apache 2.0; the paid part covers enterprise identity, auditing, managed operations and support.

Documentation

Repository layout

src/cosmos_semantic/
  spec.py            v1 authoring format (Pydantic); source of the JSON Schema
  loader.py          reading YAML files
  compiler.py        validation and semantic graph, with joins inferred from entities
  expressions.py     {member} and $user references, and protection of caller filters
  planner.py         query plan and per-dialect SQL (SQLGlot)
  connectors.py      DuckDB, Postgres (ADBC), Databricks and SQL Server (SQLAlchemy), always Arrow
  engine.py          model + connections -> executed queries
  ontology.py        business rules (virtual dimensions and tests) and SKOS glossary
  catalog.py         members, concept search and explanations
  auth.py            OIDC token validation and development keys
  evidence.py        evidence envelope: formulas, rules, policies, hashes
  reconcile.py       reconciliation with legacy SQL or another model, and divergence probes
  impact.py          impact analysis over the model graph and the audit log
  bootstrap.py       draft model from the database catalog
  service.py         querying with evidence, auditing and replay, shared by REST, MCP and CLI
  api.py             REST API (FastAPI)
  mcp_server.py      MCP server
  interchange/       dbt, Ossie and Cube import
  demo_data.py       synthetic data (en_US default, pt_BR optional)
  reference.py       running the reference queries
  cli.py             the `cosmos` command
schema/              JSON Schema of the format
examples/ecommerce/  example model, connections and the divergence zoo
reference/           reference queries (basic and advanced)
tests/fixtures/      example models in dbt, Ossie and Cube
deploy/              local docker compose
docs/                documentation and ADRs

License and contributing

The core is licensed under Apache 2.0 (ADR 0005). Contributions are welcome under the Developer Certificate of Origin; see CONTRIBUTING.md. Report vulnerabilities privately as described in SECURITY.md.