reflekt 0.6.0

A CLI tool to define event schemas, lint them to enforce conventions, publish to a schema registry, and build corresponding data artifacts (e.g., dbt sources/models/docs).

Reflekt

/rəˈflek(t)/ - to embody or represent in a faithful or appropriate way

Reflekt helps teams define, govern, and model event data for warehouse-first product analytics. It integrates with Customer Data Platforms, Schema Registries, Data Warehouses, and dbt. Events are defined using JSON schema, version controlled, and updated by pull requests (PRs), enabling:

• Branches for parallel development and testing.
• Reviews and discussion amongst teams and stakeholders.
• CI/CD suites to:
  • reflekt lint schemas against naming and metadata rules.
  • reflekt push schemas to deploy them to a schema registry for event data validation.

Stop writing SQL & YAML. reflekt build a dbt package to model, document, and test events in the warehouse.

To learn more about reflekt, checkout:

• Getting Started
• CLI Command Reference
• Integrations
• reflekt-jaffle-shop and this demo

Getting Started

Install

Reflekt is available on PyPI. Install it with pip or package manager, preferably in a virtual environment:

❯ source /path/to/venv/bin/activate  # Activate virtual environment
❯ pip install reflekt                # Install Reflekt
❯ reflekt --version                  # Confirm installation
Reflekt CLI Version: 0.6.0

Create a Reflekt Project

To create a new Reflekt project, make a directory, initialize a Git repo, and run reflekt init.

❯ mkdir ~/Repos/my-reflekt-project  # Create a new directory for the project
❯ cd ~/Repos/my-reflekt-project     # Navigate to the project directory
❯ git init                          # Initialize a new Git repo (REQUIRED)
❯ reflekt init                      # Initialize a new Reflekt project in the current directory

# Follow the prompts to configure the project

You now have a Reflekt project with the structure:

my-reflekt-project
├── .logs/                # CLI command logs
├── .reflekt_cache/       # Local cache used by reflekt CLI
├── artifacts/            # Data artifacts created by `reflekt build`
├── schemas/              # Event schema definitions
├── .gitignore
├── README.md
└── reflekt_project.yml   # Project configuration

Configure a Reflekt Project

Reflekt uses 3 files to define and configure a Reflekt project.

reflekt_project.yml

Defines project settings, event and metadata conventions, data artifact generation, and optional registry config (Avo only).

Example: reflekt_project.yml (click to expand)

# Example reflekt_project.yml
# GENERAL CONFIG ----------------------------------------------------------------------
version: 1.0

name: reflekt_demo              # Project name
vendor: com.company_name        # Default vendor for schemas in reflekt project
default_profile: dev_reflekt    # Default profile to use from reflekt_profiles.yml
profiles_path: ~/.reflekt/reflekt_profiles.yml  # Path to reflekt_profiles.yml

# SCHEMAS CONFIG ----------------------------------------------------------------------
schemas:                        # Define schema conventions
  conventions:
    event:
      casing: title             # title | snake | camel | pascal | any
      numbers: false            # Allow numbers in event names
      reserved: []              # Reserved event names
    property:
      casing: snake             # title | snake | camel | pascal | any
      numbers: false            # Allow numbers in property names
      reserved: []              # Reserved property names
    data_types: [               # Allowed data types
        string, integer, number, boolean, object, array, any, 'null'
    ]

# REGISTRY CONFIG ---------------------------------------------------------------------
registry:                       # Additional config for schema registry if needed
  avo:                          # Avo specific config
    branches:                   # Provide ID for Avo branches for `reflekt pull` to work
      staging: AbC12dEfG        # Safe to version control (See Avo docs to find branch ID: https://bit.ly/avo-docs-branch-id)
      main: main                # 'main' always refers to the main branch

# ARTIFACTS CONFIG -----------------------------------------------------------------------
artifacts:                      # Configure how data artifacts are built
  dbt:                          # dbt package config
    sources:
      prefix: __src_            # Source files will start with this prefix
    models:
      prefix: stg_              # Model files will start with this prefix
    docs:
      prefix: _stg_             # Docs files will start with this prefix
      in_folder: false          # Docs files in separate folder?
      tests:                    # dbt tests to add based on column name (can be empty dict {})
        id: [unique, not_null]

reflekt_profiles.yml

This file defines connections to schema registries and data warehouse connections.

Example: reflekt_profiles.yml (click to expand)

# Example reflekt_profiles.yml
version: 1.0

dev_reflekt:                                         # Profile name (multiple allowed)
  registry:                                          # Schema registry connection details (multiple allowed)
    - type: segment
      api_token: segment_api_token                   # https://docs.segmentapis.com/tag/Getting-Started#section/Get-an-API-token

    - type: avo
      workspace_id: avo_workspace_id                 # https://www.avo.app/docs/public-api/export-tracking-plan#endpoint
      service_account_name: avo_service_account_name # https://www.avo.app/docs/public-api/authentication#creating-service-accounts
      service_account_secret: avo_service_account_secret

  source:                          # Data warehouse connection details (multiple allowed)
    - id: snowflake                # ID must be unique per profile
      type: snowflake              # Specify details where raw event data is stored
      account: abc12345
      database: raw
      warehouse: transforming
      role: transformer
      user: reflekt_user           # Create reflekt_user with access to raw data (permissions: USAGE, SELECT)
      password: reflekt_user_password

    - id: redshift                 # ID must be unique per profile
      type: redshift               # Specify details where raw event data is stored
      host: example-redshift-cluster-1.abc123.us-west-1.redshift.amazonaws.com
      database: analytics
      port: 5439
      user: reflekt_user           # Create reflekt_user with access to raw data (permissions: USAGE, SELECT)
      password: reflekt_user_password

    - id: bigquery                 # ID must be unique per profile
      type: bigquery               # Specify details where raw event data is stored
      project: raw-data
      dataset: jaffle_shop_segment
      keyfile_json:                # Create a reflekt-user service account with access to BigQuery project where raw event data lands (permissions: BigQuery Data Viewer, BigQuery Job User). Include JSON keyfile fields below.
        type: "service_account"
        project_id: "foo-bar-123456"
        private_key_id: "abc123def456ghi789"
        private_key: "-----BEGIN PRIVATE KEY-----\nmy-very-long-private-keyF\n\n-----END PRIVATE KEY-----\n"
        client_email: "[e-mail]"
        client_id: "123456789101112131415161718"
        auth_uri: "https://accounts.google.com/o/oauth2/auth"
        token_uri: "https://oauth2.googleapis.com/token"
        auth_provider_x509_cert_url: "https://www.googleapis.com/oauth2/v1/certs"
        client_x509_cert_url: "https://www.googleapis.com/robot/v1/metadata/x509/reflekt-user%40foo-bar-123456.iam.gserviceaccount.com"

schemas/.reflekt/meta/1-0.json

A meta-schema used by reflekt lint to ensure all events in schemas/ follow the Reflekt format. Can also be used to define gloablly required metadata for all event schemas.

Example: schemas/.reflekt/meta/1-0.json (click to expand)

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": ".reflekt/meta/1-0.json",
    "description": "Meta-schema for all Reflekt events",
    "self": {
        "vendor": "reflekt",
        "name": "meta",
        "format": "jsonschema",
        "version": "1-0"
    },
    "type": "object",
    "allOf": [
        {
            "$ref": "http://json-schema.org/draft-07/schema#"
        },
        {
            "properties": {
                "self": {
                    "type": "object",
                    "properties": {
                        "vendor": {
                            "type": "string",
                            "description": "The company, application, team, or system that authored the schema (e.g., com.company, com.company.android, com.company.marketing)"
                        },
                        "name": {
                            "type": "string",
                            "description": "The schema name. Describes what the schema is meant to capture (e.g., pageViewed, clickedLink)"
                        },
                        "format": {
                            "type": "string",
                            "description": "The format of the schema",
                            "const": "jsonschema"
                        },
                        "version": {
                            "type": "string",
                            "description": "The schema version, in MODEL-ADDITION format (e.g., 1-0, 1-1, 2-3, etc.)",
                            "pattern": "^[1-9][0-9]*-(0|[1-9][0-9]*)$"
                        },
                        "metadata": {  // EXAMPLE: Defining required metadata (code_owner, product_owner)
                            "type": "object",
                            "description": "Required metadata for all event schemas",
                            "properties": {
                                "code_owner": {"type": "string"},
                                "product_owner": {"type": "string"}
                            },
                            "required": ["code_owner", "product_owner"],
                            "additionalProperties": false
                        },
                    },
                    "required": ["vendor", "name", "format", "version"],
                    "additionalProperties": false
                },
                "properties": {},
                "tests": {},
            },
            "required": ["self", "metadata", "properties"]
        }
    ]
}

Defining Event Schemas

Events in a Reflekt project are defined using the JSON schema specification and are stored in the schemas/ directory of the project. Click to expand the Order Completed example below.

Example: my-reflekt-project/schemas/jaffle_shop/Order_Completed/1-0.json (click to expand)

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": "jaffle_shop/Order_Completed/1-0.json",  // Unique ID for schema (relative to `schemas/` dir)
    "description": "User completed an order.",      // Event description (REQUIRED)
    "self": {
        "vendor": "com.thejaffleshop",                         // Company, application, or system that authored the schema
        "name": "Order Completed",                             // Name of the event
        "format": "jsonschema",                                // Format of the schema
        "version": "1-0",                                      // Version of the schema
        "metadata": {                                          // Metadata for the event
            "code_owner": "@the-jaffle-shop/frontend-guild",
            "product_owner": "[e-mail]",
        }
    },
    "type": "object",
    "properties": {                                            // Event properties (REQUIRED, but can be empty)
        "coupon": {
            "description": "Coupon code used for the order.",  // Property description (REQUIRED)
            "type": [
                "string",
                "null"                                         // Allow null values
            ]
        },
        "currency": {
            "description": "Currency for the order.",
            "type": "string"
        },
        "discount": {
            "description": "Total discount for the order.",
            "type": "number"
        },
        "order_id": {
            "description": "Unique identifier for the order.",
            "type": "string"
        },
        "products": {
            "description": "List of products in the cart.",
            "type": "array",                                    // Array type
            "items": {                                          // Items in the array
                "type": "object",
                "properties": {                                 // Properties of the items
                    "category": {
                        "description": "Category of the product.",
                        "type": "string"
                    },
                    "name": {
                        "description": "Name of the product.",
                        "type": "string"
                    },
                    "price": {
                        "description": "Price of the product.",
                        "type": "number"
                    },
                    "product_id": {
                        "description": "Unique identifier for the product.",
                        "type": "string"
                    },
                    "quantity": {
                        "description": "Quantity of the product in the cart.",
                        "type": "integer"
                    },
                    "sku": {
                        "description": "Stock keeping unit for the product.",
                        "type": "string"
                    }
                },
                "required": [                                  // Required properties for the items
                    "product_id",
                    "sku",
                    "category",
                    "name",
                    "price",
                    "quantity"
                ],
                "additionalProperties": false,                // Are additional item properties allowed?
            }
        },
        "revenue": {
            "description": "Total revenue for the order.",
            "type": "number"
        },
        "session_id": {
            "description": "Unique identifier for the session.",
            "type": "string"
        },
        "shipping": {
            "description": "Shipping cost for the order.",
            "type": "number"
        },
        "subtotal": {
            "description": "Subtotal for the order (revenue - discount).",
            "type": "number"
        },
        "tax": {
            "description": "Tax for the order.",
            "type": "number"
        },
        "total": {
            "description": "Total cost for the order (revenue - discount + shipping + tax = subtotal + shipping + tax).",
            "type": "number"
        }
    },
    "required": [                  // Required properties (can be empty)
        "session_id",
        "order_id",
        "revenue",
        "coupon",
        "discount",
        "subtotal",
        "shipping",
        "tax",
        "total",
        "currency",
        "products"
    ],
    "additionalProperties": false  // Are additional properties allowed?
}

Schema $id and version

Schemas in a Reflekt project are identified and --selected by their $id, which is their path relative to the schemas/ directory. For example:

File Path to Schema	Schema $id
~/repos/my-reflekt-project/schemas/jaffle_shop/Cart_Viewed/1-0.json	jaffle_shop/Cart_Viewed/1-0.json
~/repos/my-reflekt-project/schemas/jaffle_shop/Cart_Viewed/2-1.json	jaffle_shop/Cart_Viewed/2-1.json

Each schema has a version (e.g., 1-0, 2-1), used to indicate changes to data collection requirements. New event schemas start at 1-0 and follow a MAJOR-MINOR version spec, as shown in the table below.

Type	Description	Example	Use Case
MAJOR	Breaking change incompatible with previous data.	1-0, 2-0 (ends in `-0)	- Add/remove/rename a required property - Change a property from optional to required - Change a property's type
MINOR	Non-breaking change compatible with previous data.	1-1, 2-3	- Add/remove/rename an optional property - Change a property from required to optional

[!NOTE] For MINOR schema versions (non-breaking changes), you can either:

• Update the existing schema and increment the MINOR version number.
• Create a new .json file with the updated schema and increment the MINOR version number.

For MAJOR schema versions (breaking changes), you MUST:

• Create a new .json file with the updated schema. This way, an application/product/feature can begin using the new schema while others continue to use the old schema (migrating later).

Linting Event Schemas

Schemas can be linted to test if they follow the naming conventions in your [reflekt_project.yml] and metadata conventions in schemas/.reflekt/meta/1-0.json.

❯ reflekt lint --select schemas/jaffle_shop
[18:31:12] INFO     Running with reflekt=0.5.0
[18:31:12] INFO     Searching for JSON schemas in: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop
[18:31:12] INFO     Found 9 schema(s) to lint
[18:31:12] INFO     1 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Order_Completed/1-0.json
[18:31:19] INFO     2 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Identify/1-0.json
[18:31:20] INFO     3 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Clicked/1-0.json
[18:31:24] INFO     4 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Cart_Viewed/1-0.json
[18:31:26] INFO     5 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Removed/1-0.json
[18:31:32] INFO     6 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Added/1-0.json
[18:31:37] INFO     7 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Viewed/1-0.json
[18:31:40] INFO     8 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Completed/1-0.json
[18:31:44] INFO     9 of 9 Linting /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Page_Viewed/1-0.json
[18:31:51] INFO     Completed successfully

Sending Event Schemas to a Schema Registries

In order to validate events as they flow from Application -> Registry -> Customer Data Platform (CDP) -> Data Warehouse, we need to send a copy of our event schemas to a schema registry (see supported registries). This is done with the reflekt push command.

❯ reflekt push --registry segment --select schemas/jaffle_shop
[18:41:05] INFO     Running with reflekt=0.5.0
[18:41:06] INFO     Searching for JSON schemas in: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop
[18:41:06] INFO     Found 9 schemas to push
[18:41:06] INFO     1 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Order_Completed/1-0.json
[18:41:06] INFO     2 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Identify/1-0.json
[18:41:06] INFO     3 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Clicked/1-0.json
[18:41:06] INFO     4 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Cart_Viewed/1-0.json
[18:41:06] INFO     5 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Removed/1-0.json
[18:41:06] INFO     6 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Added/1-0.json
[18:41:06] INFO     7 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Viewed/1-0.json
[18:41:06] INFO     8 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Completed/1-0.json
[18:41:06] INFO     9 of 9 Pushing /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Page_Viewed/1-0.json
[18:41:08] INFO     Completed successfully

Building dbt Packages to Model Event Data

Modeling event data in dbt is a lot of work. Everyone wants staging models that are clean, documented, and tested. But who wants to write and maintain SQL and YAML for hundreds of events?

You don't have to choose. Put reflekt build to work for you - staging models, documentation, even tests - all in a dbt package ready for you to use in your dbt project.

❯ reflekt build --artifact dbt --select schemas/jaffle_shop --source snowflake.raw.jaffle_shop_segment --sdk segment
[18:56:25] INFO     Running with reflekt=0.5.0
[18:56:26] INFO     Searching for JSON schemas in: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop
[18:56:26] INFO     Found 9 schemas to build
[18:56:27] INFO     Building dbt package:
                        name: jaffle_shop
                        dir: /Users/gclunies/Repos/reflekt/artifacts/dbt/jaffle_shop
                        --select: jaffle_shop
                        --sdk_arg: segment
                        --source: snowflake.raw.jaffle_shop_segment
[18:56:27] INFO     Building dbt source 'jaffle_shop_segment'
[18:56:27] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Order_Completed/1-0.json
[18:56:28] INFO     Building dbt table 'order_completed' in source 'jaffle_shop_segment'
[18:56:28] INFO     Building staging model 'stg_jaffle_shop_segment__order_completed.sql'
[18:56:28] INFO     Building dbt documentation '_stg_jaffle_shop_segment__order_completed.yml'
[18:56:28] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Identify/1-0.json
[18:56:29] INFO     Building dbt table 'identifies' in source 'jaffle_shop_segment'
[18:56:29] INFO     Building staging model 'stg_jaffle_shop_segment__identifies.sql'
[18:56:29] INFO     Building dbt documentation '_stg_jaffle_shop_segment__identifies.yml'
[18:56:29] INFO     Building dbt artifacts for schema: Segment 'users' table
[18:56:29] INFO     Building dbt table 'users' in source 'jaffle_shop_segment'
[18:56:29] INFO     Building staging model 'stg_jaffle_shop_segment__users.sql'
[18:56:29] INFO     Building dbt documentation '_stg_jaffle_shop_segment__users.yml'
[18:56:29] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Clicked/1-0.json
[18:56:29] INFO     Building dbt table 'product_clicked' in source 'jaffle_shop_segment'
[18:56:29] INFO     Building staging model 'stg_jaffle_shop_segment__product_clicked.sql'
[18:56:29] INFO     Building dbt documentation '_stg_jaffle_shop_segment__product_clicked.yml'
[18:56:29] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Cart_Viewed/1-0.json
[18:56:29] INFO     Building dbt table 'cart_viewed' in source 'jaffle_shop_segment'
[18:56:29] INFO     Building staging model 'stg_jaffle_shop_segment__cart_viewed.sql'
[18:56:29] INFO     Building dbt documentation '_stg_jaffle_shop_segment__cart_viewed.yml'
[18:56:29] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Removed/1-0.json
[18:56:30] INFO     Building dbt table 'product_removed' in source 'jaffle_shop_segment'
[18:56:30] INFO     Building staging model 'stg_jaffle_shop_segment__product_removed.sql'
[18:56:30] INFO     Building dbt documentation '_stg_jaffle_shop_segment__product_removed.yml'
[18:56:30] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Product_Added/1-0.json
[18:56:30] INFO     Building dbt table 'product_added' in source 'jaffle_shop_segment'
[18:56:30] INFO     Building staging model 'stg_jaffle_shop_segment__product_added.sql'
[18:56:30] INFO     Building dbt documentation '_stg_jaffle_shop_segment__product_added.yml'
[18:56:30] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Viewed/1-0.json
[18:56:30] INFO     Building dbt table 'checkout_step_viewed' in source 'jaffle_shop_segment'
[18:56:30] INFO     Building staging model 'stg_jaffle_shop_segment__checkout_step_viewed.sql'
[18:56:30] INFO     Building dbt documentation '_stg_jaffle_shop_segment__checkout_step_viewed.yml'
[18:56:30] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Checkout_Step_Completed/1-0.json
[18:56:30] INFO     Building dbt table 'checkout_step_completed' in source 'jaffle_shop_segment'
[18:56:30] INFO     Building staging model 'stg_jaffle_shop_segment__checkout_step_completed.sql'
[18:56:30] INFO     Building dbt documentation '_stg_jaffle_shop_segment__checkout_step_completed.yml'
[18:56:30] INFO     Building dbt artifacts for schema: /Users/gclunies/Repos/reflekt/schemas/jaffle_shop/Page_Viewed/1-0.json
[18:56:30] INFO     Building dbt table 'pages' in source 'jaffle_shop_segment'
[18:56:30] INFO     Building staging model 'stg_jaffle_shop_segment__pages.sql'
[18:56:30] INFO     Building dbt documentation '_stg_jaffle_shop_segment__pages.yml'
[18:56:30] INFO     Building dbt artifacts for schema: Segment 'tracks' table
[18:56:31] INFO     Building dbt table 'tracks' in source 'jaffle_shop_segment'
[18:56:31] INFO     Building staging model 'stg_jaffle_shop_segment__tracks.sql'
[18:56:31] INFO     Building dbt documentation '_stg_jaffle_shop_segment__tracks.yml'
[18:56:31] INFO     Copying dbt package from temporary path /Users/gclunies/Repos/reflekt/.reflekt_cache/artifacts/dbt/jaffle_shop to /Users/gclunies/Repos/reflekt/artifacts/dbt/jaffle_shop
[18:56:31] INFO     Successfully built dbt package

---

CLI Commands

A description of commands can be seen by running reflekt --help. The help page for each CLI command is shown below.

reflekt init

❯ reflekt init --help
[11:17:16] INFO     Running with reflekt=0.6.0

 Usage: reflekt init [OPTIONS]

 Initialize a Reflekt project.

╭─ Options ──────────────────────────────────────────────────────╮
│ --dir                        TEXT  [default: .]                │
│ --verbose    --no-verbose          [default: no-verbose]       │
│ --help                             Show this message and exit. │
╰────────────────────────────────────────────────────────────────╯

reflekt debug

❯ reflekt debug --help
[11:18:07] INFO     Running with reflekt=0.6.0

 Usage: reflekt debug [OPTIONS]

 Check Reflekt project configuration.

╭─ Options ───────────────────────────────────╮
│ --help          Show this message and exit. │
╰─────────────────────────────────────────────╯

reflekt lint

❯ reflekt lint --help
[11:20:29] INFO     Running with reflekt=0.6.0

 Usage: reflekt lint [OPTIONS]

 Lint schema(s) to test for naming and metadata conventions.

╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --select   -s      TEXT  Schema(s) to lint. Starting with 'schemas/' is optional. [default: None] [required] │
│    --verbose  -v            Verbose logging.                                                                        │
│    --help                   Show this message and exit.                                                             │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

reflekt push

❯ reflekt push --help
[11:22:37] INFO     Running with reflekt=0.6.0

 Usage: reflekt push [OPTIONS]

 Push schema(s) to a schema registry.

╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --registry  -r      [avo|segment]  Schema registry to push to. [default: None] [required]                                          │
│ *  --select    -s      TEXT           Schema(s) to push to schema registry. Starting with 'schemas/' is optional. [default: None] │
│    --delete    -D                     Delete schema(s) from schema registry. Prompts for confirmation                                 │
│    --force     -F                     Force command to run without confirmation.                                                      │
│    --profile   -p      TEXT           Profile in reflekt_profiles.yml to use for schema registry connection.                          │
│    --verbose   -v                     Verbose logging.                                                                                │
│    --help                             Show this message and exit.                                                                     │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

reflekt pull

❯ reflekt pull --help
[11:25:18] INFO     Running with reflekt=0.6.0

 Usage: reflekt pull [OPTIONS]

 Pull schema(s) from a schema registry.

╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --registry  -r      [avo|segment]  Schema registry to pull from. [default: None] [required]                                                                         │
│ *  --select    -s      TEXT           Schema(s) to pull from schema registry. If registry uses tracking plans, starting with the plan name. [default: None] [required] │
│    --profile   -p      TEXT           Profile in reflekt_profiles.yml to use for schema registry connection.                                                           │
│    --verbose   -v                     Verbose logging.                                                                                                                 │
│    --help                             Show this message and exit.                                                                                                      │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

reflekt build

❯ reflekt build --help
[11:31:36] INFO     Running with reflekt=0.6.0

 Usage: reflekt build [OPTIONS]

 Build data artifacts based on schemas.

╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --artifact  -a      [dbt]      Type of data artifact to build. [default: None] [required]                                                                                                         │
│ *  --select    -s      TEXT       Schema(s) to build data artifacts for. Starting with 'schemas/' is optional. [default: None] [required]                                                            │
│ *  --sdk               [segment]  The SDK used to collect the event data. [default: None] [required]                                                                                                 │
│ *  --source            TEXT       The <source_id>.<database>.<schema> storing raw event data. <source_id> must be a data warehouse source defined in reflekt_profiles.yml [default: None] [required] │
│    --profile   -p      TEXT       Profile in reflekt_profiles.yml to look for the data source specified by the --source option. Defaults to default_profile in reflekt_project.yml                   │
│    --verbose   -v                 Verbose logging.                                                                                                                                                   │
│    --help                         Show this message and exit.                                                                                                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

reflekt report

❯ reflekt report --help
[08:20:09] INFO     Running with reflekt=0.6.0

 Usage: reflekt report [OPTIONS]

 Generate Markdown report(s) for schema(s).

╭─ Options ───────────────────────────────────────────────────────────────────────────────────────╮
│ *  --select   -s      TEXT  Schema(s) to generate Markdown report(s) for. Starting with         │
│                             'schemas/' is optional.                                             │
│                             [default: None]                                                     │
│                             [required]                                                          │
│    --to-file  -f            Write report(s) to file instead of terminal.                        │
│    --verbose  -v            Verbose logging.                                                    │
│    --help                   Show this message and exit.                                         │
╰─────────────────────────────────────────────────────────────────────────────────────────────────╯

---

Integrations

Customer Data Platform (CDP)

Reflekt understands how Customer Data Platforms (CDPs) collect event data and load them into data warehouses, allowing it to:

• Parse the schemas in a Reflekt project.
• Find matching tables for events and columns for properties in the data warehouse.
• Build a dbt package with sources, staging models, and documentation for event data.

CDP	Supported
Segment	✅
Rudderstack	🚧 Coming Soon 🚧
Amplitude	🚧 Coming Soon 🚧

Schema Registry

Schema registries store and serve schemas. When a schema is registered in a regsitry, it can be used to validate events as they flow through your data collection infrastructure. Reflekt works with schema registries from CDPs, SaaS vendors, and open-source projects, letting teams to decide between managed and self-hosted solutions.

Registry	Open Source	Schema Versions	Recommended Workflow
Segment Protocols	❌	MAJOR only	Manage schemas in Reflekt. reflekt push to Protocols for event validation. reflekt build --artifact dbt to build dbt package.
Avo	❌	MAJOR only	Manage schemas in Avo. reflekt pull to get schemas. reflekt build --artifact dbt to build dbt package.
reflekt-registry 🚧 Coming Soon 🚧	✅	MAJOR & MINOR	Manage schemas in Reflekt. reflekt push to reflekt-registry. reflekt build --artifact dbt to build dbt package.

Data Warehouse

In order to build dbt packages, Reflekt needs to connect to a cloud data warehouse where raw event data is stored.

Data Warehouse	Supported
Snowflake	✅
Redshift	✅
BigQuery	✅

Reflekt NEVER copies, moves, or modifies events in the data warehouse. It ONLY reads table and column names for templating.

dbt

dbt enables anyone that knows SQL to transform data in a cloud data warehouse. When modeling in dbt, it is best practice to:

• Define sources pointing to the raw data.
• Write staging models that rename, recast, or usefully reconsider columns into a consistent format.
• Document and test the staging models.

But as the number of events intsrumented in your product grow or change as the product evolves, mainting this best practice can be burdensome and boring. This is where reflekt build steps in.

Contribute

• Source Code: github.com/GClunies/reflekt
• Issue Tracker: github.com/GClunies/reflekt/issues
• Pull Requests: github.com/GClunies/reflekt/pulls

License

This project is licensed under the Apache-2.0 License.