Skip to main content

The Materialize adapter plugin for dbt (data build tool)

Project description

dbt-materialize

dbt adapter for Materialize.

Note: this plugin is a work in progress, and not yet suitable for production.

Installation

This package can be installed via pip:

pip install dbt-materialize

or locally from the Materialize Github repository:

git clone https://github.com/MaterializeInc/materialize
pip install materialize/dbt-materialize

Configuring your profile

Materialize is based on the Postgres database protocols, so use the dbt postgres settings in your connection profile, with the following alterations:

  • type: materialize
  • user: materialize

Complete sample profiles can be found in sample_profiles.yml.

Supported Features

Materializations

Type Supported? Details
view YES Creates a view.
materializedview YES Creates a materialized view.
table YES Creates a materialized view. (Actual table support pending #5266)
ephemeral YES Executes queries using CTEs.
incremental NO Use the materializedview materialization instead! dbt's incremental models are valuable because they only spend your time and money transforming your new data as it arrives. Luckily, this is exactly what Materialize's materialized views were built to do! Better yet, our materialized views will always return up-to-date results without manual or configured refreshes. For more information, check out our documentation.

Additional macros

dbt only supports a limited set of materialization types. To create other types of objects in Materialize via dbt, use the following Materialize-specific macros:

Macro Purpose
mz_generate_name(identifier) Generates a fully-qualified name (including the database and schema) given an object name.
mz_create_source(sql) Given a CREATE SOURCE SQL statement, creates the source in the Materialize instance.
mz_drop_source(name, if_exists, cascade) Drops the named source in Materialize.
mz_create_index(obj_name, default, idx_name, col_refs, with_options) Creates an index in Materialize.
mz_drop_index(name, default, if_exists, cascade) Drops an index in Materialize.
mz_create_sink(name) Given a CREATE SINK SQL statement, creates a sink in the Materialize instance.
mz_drop_sink(name, if_exists) Drops a sink in Materialize.

Additional macro details

mz_drop_source(name, if_exists, cascade)

Argument Type Detail
name string Name of the source to drop
if_exists boolean Does not return an error if the named source does not exist
cascade boolean Drops the source and its dependent objects

mz_create_index(obj_name, name, default, col_refs, with_options)

Argument Type Detail
obj_name string Name of the source or view to create an index on
name optional string If not a default index, the name of the index to create
default boolean True will create a default index on the named source or view
col_refs optional list of strings The columns to use as the key into the index
with_options optional list of strings The name of index parameters to set as val

mz_drop_index(name, default, if_exists, cascade)

Argument Type Detail
name string Name of the index to drop
default boolean True will drop the default index
if_exists boolean Does not return an error if the named index does not exist
cascade boolean Drops the index and its dependent objects

Seeds

dbt seed will create a static materialized view from a CSV file. You will not be able to add to or update this view after it has been created.

Hooks

Not tested.

Custom Schemas

Not tested.

Sources

Not tested.

Testing and Documentation

dbt docs and dbt test commands are supported.

Snapshots

Not supported, will likely not be supported in the near term.

Contributors

A huge thank you to Josh Wills, who originally created this adapter.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt-materialize-0.18.1.post4.tar.gz (11.4 kB view hashes)

Uploaded Source

Built Distribution

dbt_materialize-0.18.1.post4-py2-none-any.whl (26.2 kB view hashes)

Uploaded Python 2

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page