Skip to main content

Docstring dbt test & documentation in SQL file

Project description

Build Status

dbt_docstring

Docstring dbt test & documentation in SQL file

What is it?

dbt has a test and documentation feature where models/schema.yml is the definition file. While this is already a big help for testing and documentation in ELT, not being able to keep documentation in the source code may cause more documentats out of sync with the source.

dbtdocstr lets you write docment in a docstring style directly in .sql files.

Install

pip install dbt_docstring

How does it work?

dbtdocstr command scans .sql files under dbt's models directories and look for a block that begins with ```dbt and end with ```. Inside the block you can write the content of the models section of schema.yml corresponding to the current table as specified in dbt document:

Example (<dbt_root>/models/api_key_status.sql)

/*
# API key status

This table lists the API keys with the status.

```dbt
columns:
  - name: api_key
    description: API key
    tests:
      - unique
  - name: enabled
    description: True if API key is enabled status
  - name: update_datetime
    description: Last update date and time
```
*/
SELECT
   api_key,
   enabled,
   update_datetime
FROM {{ ref('my_api_key_list') }}

Then run:

dbtdocstr <dbt_project_root_directory>

These two files will be auto-generated from each .sql file in the dbt project:

models/docs.md:

# This file was auto-generated by dbtdocstr.
# Don't manually update.
# https://github.com/anelendata/dbt_docstring

{% docs api_key_status %}
# API key status

This table lists the API keys with the status.
{% enddocs %}

models/schema.yml:

# This file was auto-generated by dbtdocstr.
# Don't manually update.
# https://github.com/anelendata/dbt_docstring

version: 2
models:
  - name: api_key_status
    description: '{{ docs("api_key_status") }}'
    columns:
      - name: api_key
        description: API key
        tests:
          - unique
      - name: enabled
        description: True if API key is enabled status
      - name: update_datetime
        description: Last update date and time
  - name: ...

To see the document generation, use dbt command:

dbt docs generate
dbt docs serve

Notes

  • The doc must be a SQL comment block comment that begins with '/*' and ends with '*/'
  • The first comment block will be extracted.
  • The dbt block is searched within the first comment block.
  • Any text after the dbt block will be ignored.
  • dbt's Docs Blocks feature can be used only for table & view description. Not column descriptions.
  • dbtdocstr --backup <dbt_root_directory> to create backup files of schema.yml and docs.yml if they exsit.

Original repository


Copyright © 2020 Anelen Co., LLC

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dbt-docstring, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size dbt_docstring-0.1.1-py3-none-any.whl (8.8 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size dbt_docstring-0.1.1.tar.gz (4.2 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page