meltano-state-backend-snowflake 0.2.0

Meltano State Backend for Snowflake

meltano-state-backend-snowflake

This is a Meltano extension that provides a Snowflake state backend.

Installation

This package needs to be installed in the same Python environment as Meltano.

From GitHub

With uv

uv tool install --with git+https://github.com/meltano/meltano-state-backend-snowflake.git meltano

With pipx

pipx install meltano
pipx inject meltano git+https://github.com/meltano/meltano-state-backend-snowflake.git

Configuration

To store state in Snowflake, set the state_backend.uri setting to snowflake://<user>:<password>@<account>/<database>/<schema>.

State will be stored in two tables that Meltano will create automatically:

• meltano_state - Stores the actual state data
• meltano_state_locks - Manages concurrency locks

To authenticate to Snowflake, you'll need to provide:

state_backend:
  uri: snowflake://my_user:my_password@my_account/my_database/my_schema
  snowflake:
    warehouse: my_warehouse  # Required: The compute warehouse to use
    role: my_role           # Optional: The role to use for the connection

Alternatively, you can provide credentials via individual settings:

state_backend:
  uri: snowflake://my_account/my_database/my_schema
  snowflake:
    account: my_account
    user: my_user
    password: my_password
    warehouse: my_warehouse
    database: my_database
    schema: my_schema      # Defaults to PUBLIC if not specified
    role: my_role          # Optional

Connection Parameters

• account: Your Snowflake account identifier (e.g., myorg-account123)
• user: The username for authentication
• password: The password for authentication
• warehouse: The compute warehouse to use (required)
• database: The database where state will be stored
• schema: The schema where state tables will be created (defaults to PUBLIC)
• role: Optional role to use for the connection

Security Considerations

When storing credentials:

• Use environment variables for sensitive values in production
• Consider using Snowflake key-pair authentication (future enhancement)
• Ensure the user has CREATE TABLE, INSERT, UPDATE, DELETE, and SELECT privileges

Example using environment variables:

export MELTANO_STATE_BACKEND_SNOWFLAKE_PASSWORD='my_secure_password'
meltano config set meltano state_backend.uri 'snowflake://my_user@my_account/my_database'
meltano config set meltano state_backend.snowflake.warehouse 'my_warehouse'

Development

Setup

uv sync

Run tests

Run all tests, type checks, linting, and coverage:

uvx --with tox-uv tox run-parallel

Bump the version

Using the GitHub CLI:

gh release create v<new-version>