The MySQL adapter plugin for dbt (data build tool)
Project description
dbt-mysql
This plugin ports dbt functionality to MySQL 8.0.
This is an experimental plugin. Please read these docs carefully and use at your own risk. Issues and PRs welcome!
Untested
We have not tested it against older versions of MySQL or storage engines other than the default of InnoDB.
Compatiblity with other dbt packages (like dbt_utils) is also untested.
Installation
This plugin can be installed via pip:
# Install dbt-mysql from PyPi:
$ pip install dbt-mysql
dbt-mysql creates connections via an ODBC driver that requires pyodbc
.
See https://github.com/mkleehammer/pyodbc/wiki/Install for more info about installing pyodbc
.
Supported features
Supported? | Feature |
---|---|
✅ | Table materialization |
✅ | View materialization |
✅ | Incremental materialization |
✅ | Ephemeral materialization |
✅ | Seeds |
✅ | Sources |
✅ | Custom data tests |
✅ | Docs generate |
✅ | Snapshots |
Configuring your profile
A dbt profile can be configured to run against MySQL using the following configuration:
Option | Description | Required? | Example |
---|---|---|---|
schema | Specify the schema (database) to build models into | Required | analytics |
server | The server (hostname) to connect to | Required | yourorg.mysqlhost.com |
username | The username to use to connect to the server | Required | dbt_admin |
password | The password to use for authenticating to the server | Required | correct-horse-battery-staple |
driver | ODBC DSN configured | Required | MySQL ODBC 8.0 ANSI Driver |
Example entry for profiles.yml:
your_profile_name:
target: dev
outputs:
dev:
type: mysql
server: localhost
schema: analytics
username: your_mysql_username
password: your_mysql_password
driver: MySQL ODBC 8.0 ANSI Driver
Notes
MySQL also has two-part relation names (similar to SQLite and Spark) whereas many database management systems have three-part relation names.
MySQL, dbt, and the ANSI-standard information_schema
use different terminology for analagous concepts. Note that MySQL does not implement the top-level concept of the information_schema "catalog" (which dbt calls a "database").
This adapter handles the two-part relation names in MySQL similarly to the dbt-spark and dbt-sqlite adapters.
This is as a cross-walk between each concept:
information_schema | MySQL | dbt |
---|---|---|
catalog | undefined / not implemented | database |
schema | database | schema |
table/view | table/view | relation |
column | column | column |
Running Tests
- Modify
test/mysql.dbtspec
with yourserver
,username
, andpassword
- Install the
pytest-dbt-adapter
package - Run the test specs in this repository
pip install pytest-dbt-adapter
pytest test/mysql.dbtspec
Reporting bugs and contributing code
- Want to report a bug or request a feature? See the contributing guidelines, or open an issue.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dbt_mysql-0.18.0rc2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2395be76056a3238cc389bed977b523cda6e355060678c460d491d9ca0db89a5 |
|
MD5 | 433b13c1e663d2a25ae7dd800ad5889a |
|
BLAKE2b-256 | e8be7f3249e8814de50e8366d8c26eb57899729746f4e9df9d2d976f94535207 |