Skip to main content

dbt adapter for Polnor — run dbt models on a Polnor SQL warehouse (DuckDB reads + Spark Iceberg writes).

Project description

dbt-polnor

dbt adapter for Polnor — run your dbt models on a Polnor SQL warehouse with Iceberg tables, the same way you'd run them on Databricks or Snowflake.

Install

pip install dbt-polnor

This installs dbt-core>=1.7, polnor>=1.0, and the adapter glue.

Configure

In ~/.dbt/profiles.yml:

my_project:
  target: dev
  outputs:
    dev:
      type: polnor
      host: https://api.polnor.net
      token: "{{ env_var('POLNOR_TOKEN') }}"
      warehouse_id: wh-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      workspace_id: ws-xxxxxxxx                # optional
      database: polnor                          # Spark catalog name (= Lakekeeper warehouse)
      schema: demo                              # default namespace for unqualified models
      threads: 4

Or interactively :

dbt init my_project
# select polnor as the database

Use

Anywhere your dbt project does :

dbt run
dbt test
dbt build
dbt docs generate

Models with materialized='table' get CREATE TABLE ... USING iceberg through the Spark sidecar; materialized='view' becomes a SQL view via Spark. Incremental models work via Spark's MERGE INTO.

What's supported (V0.1)

Feature Status
materialized='table' (Iceberg)
materialized='view'
materialized='incremental' 🟡 append + merge (no delete+insert yet)
Sources + ref()
Tests (generic + singular)
Snapshots 🟡 V2
Seeds
Macros
dbt docs generate
Python models (Spark) ⏳ V2

Limitations (V1)

  • No transaction rollback — Polnor's engine is auto-commit per statement.
  • No transient / temporary tables — Iceberg doesn't support them.
  • dbt seed uses INSERT batches (1k rows / chunk) — for ≫100k rows prefer copying CSV to S3 then CREATE TABLE FROM directly.

Links

License

Apache-2.0.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_polnor-0.1.0.tar.gz (10.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dbt_polnor-0.1.0-py3-none-any.whl (11.0 kB view details)

Uploaded Python 3

File details

Details for the file dbt_polnor-0.1.0.tar.gz.

File metadata

  • Download URL: dbt_polnor-0.1.0.tar.gz
  • Upload date:
  • Size: 10.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for dbt_polnor-0.1.0.tar.gz
Algorithm Hash digest
SHA256 c0b0cc85f3d3c4100de85f209c419b6e97fe49c0727e78ef0bb47002460b88e3
MD5 cdeaacaf24f71becd946136c7f9813c6
BLAKE2b-256 8d4b96a852eb0e879fa75ea138145192ab036dac886021cd17bbd5e3ec565901

See more details on using hashes here.

File details

Details for the file dbt_polnor-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: dbt_polnor-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 11.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for dbt_polnor-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 da01dfc26e6edaf54928c649fc257895cf29352ea166005646d7366d9e21a82e
MD5 72559d93d43cd2f2d23110cb869a5d71
BLAKE2b-256 bc72aeefc8a1fa1a4d0fa71fa99d171495066a789e9bad85cb5c5d5a5fa5eb0b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page