dbt-polars
dbt-polars integrates the flexibility and performance of Polars with the data governance and lineage capabilities of dbt.
Key features of dbt-polars:
- Run anywhere: Polars runs on any compute platform, so you’re not locked into an expensive data warehouse.
- Flexible: Available catalogs let you connect to local storage, Iceberg catalogs, Databricks, and more.
- Extensible: Create a custom catalog to fit your unique storage and warehouse needs.
- First-class Python models: Run Python models with the same performance and capabilities as SQL models.
- Mix Python and SQL: Choose the language that best fits the transformation.
- dbt-native: Bring lineage, documentation, and data testing into your workflow.
- Ingestion: Build efficient, incremental API ingestion with Python models and keep the entire pipeline visible in the lineage graph.
Installation
pip install dbt-polars
Extra dependencies are required for cloud and Iceberg backends:
pip install 'dbt-polars[azure]' # Azure Blob Storage
pip install 'dbt-polars[databricks]' # Azure Blob Storage
pip install 'dbt-polars[iceberg]' # Apache Iceberg (SQLite, REST, Databricks, …)
Catalogs
dbt-polars uses catalogs to define where data is stored. Each catalog maps to a storage backend. You can configure multiple catalogs in a single profile and reference them from your models.
| Catalog | Backend | Status |
|---|---|---|
local |
Local filesystem | Usable |
iceberg with SQLite |
Iceberg + SQLite metastore | Usable |
iceberg with REST + Databricks |
Iceberg REST API (Unity Catalog) | Experimental |
iceberg with other backends |
Any pyiceberg-supported backend | Experimental |
azure |
Azure Blob Storage / ADLS Gen2 | Experimental |
databricks |
Unity Catalog REST API + credential vending | Experimental |
Quick start
See the Getting started guide.
A minimal profiles.yml for local development is:
my_project:
target: dev
outputs:
dev:
type: polars
schema: dev
catalog_type: local
root: ./data
Documentation
1 maintainer
Metadata
1 maintainer
Release files for dbt-polars-catalog-s3 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| dbt_polars_catalog_s3-0.3.1.tar.gz | 7.1 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dbt_polars_catalog_s3-0.3.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 12.3 kB
Release files / dbt_polars_catalog_s3-0.3.1.tar.gz
| Download URL | dbt_polars_catalog_s3-0.3.1.tar.gz |
|---|---|
| Size | 7.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a0b13a6f3c8f8b5c5de4b639e2ff47a3e2f285748308fbd4aaf8b560f3bead53
|
|
BLAKE2b-256 checksum How to use checksums |
162f202b3a56597598c9b4f310e43adc619ef0d49cff65b5f71f960093244d69
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / dbt_polars_catalog_s3-0.3.1-py3-none-any.whl
| Download URL | dbt_polars_catalog_s3-0.3.1-py3-none-any.whl |
|---|---|
| Size | 5.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a78a0db0ad8c30aeb9e35f0293038e3725e5c3aa1ccc1ccf87a9078ba395d9f1
|
|
BLAKE2b-256 checksum How to use checksums |
6f7480c327d35ebf322675e8cfa56df7c3d2ac8e4b024d717b2ad4d1a4ee99be
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for dbt-polars-catalog-s3 0.3.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
