dbt-upsolver 0.1.8

The Upsolver adapter plugin for dbt

dbt-upsolver

Using Upsolver udapter for dbt

What is Upsolver

Upsolver enables you to use familiar SQL syntax to quickly build and deploy data pipelines, powered by a stream processing engine designed for cloud data lakes.

SQLake

SQLake is Upsolvers new UI and SQL console allowing to execute commands and monitor pipelines in the UI. It also includes freee trial and access to variety of examples and tutorials.

What is dbt

dbt is a transformation workflow that helps you get more work done while producing higher quality results.

What is dbt Core

dbt Core is an open-source tool that enables data teams to transform data using analytics engineering best practices. You can install and use dbt Core on the command line.

Getting started

Install dbt-upsolver adapter :

 pip install  dbt-upsolver

Make sure the adapter is installed:

dbt --version

Expect to see:

Core:
  - installed: <version>
  - latest:    <version>
Plugins:
  - upsolver: <version>

Register Upsolver account

To register just navigate to SQL Lake Sign Up form. You'll have access to SQL workbench with examples and tutorials after completing the registration.

Create API token

After login navigate to "Settings" and then to "API Tokens" You will need API token and API Url to access Upsolver programatically. Settings -> API Tokens -> Generate Then click "Generate" new token and save it for future use.

Get your user name, database and schema

For user name navigate to Settings -> User details and copy user name For database and schema navigate to Worksheets and click New. You will find database name and schema(catalog) name under Entities panel

Create new dbt-upsolver project

dbt init <project_name>

Prompt:

Which database would you like to use? [1] upsolver

Enter a number:
api_url (your api_url): https://mt-api-prod.upsolver.com
token (your token): <token>
user (dev username): <username>
database (default database): <database>
schema (default schema): <schema>
threads (1 or more) [1]: <number>

profiles.yml should look like this:

profiles.yml location is something like /Users//.dbt/profiles.yml

project_name:
  target: dev
  outputs:
    dev:
      api_url: https://mt-api-prod.upsolver.com
      database: ...
      schema: ...
      threads: 1
      token: ...
      type: upsolver
      user: ...

Check connection

dbt debug

Run all models

dbt run

Run the specific model

dbt run --select <model name>

Supported dbt commands:

• dbt init
• dbt debug
• dbt run
• dbt compile

!not supported Upsolver feature Delete data from target table

Further reading

Projects samples examples Upsolver sqlake documentation Upsolver sqlake documentation