dbtenv is a version manager for dbt, automatically installing and switching to the needed adapter and version of dbt.
Project description
dbtenv
dbtenv is a version manager for dbt, automatically installing and switching to the needed adapter and version of dbt.
Installation
- Install pipx if you haven't already.
- Run
pipx install dbtenv
.
Basic example usage
Run dbtenv execute -- build
in a dbt project, and dbtenv will automatically download the required adapter and version for the project and run dbt build
. We recommend setting up an alias to dbtenv so that you can run dbt commands as normal but through dbtenv.
How it works
Run dbtenv --help
to see some overall documentation for dbtenv, including its available sub-commands, and run dbtenv <sub-command> --help
to see documentation for that sub-command.
dbtenv uses pip to install dbt versions from the Python Package Index into Python virtual environments within ~/.dbt/versions
.
Installing dbt versions
You can run dbtenv versions
to list the versions of dbt available to install, and run dbtenv install <version>
to install a specific version.
dbtenv will automatically install the required version of dbt for the current project by default. To disable this behaviour, set the environment variable DBTENV_AUTO_INSTALL
to false
.
Some tips:
- You can customize where the dbt package-version-specific Python virtual environments are created by setting the
DBTENV_VENVS_DIRECTORY
environment variable. - You can have dbtenv only install Python packages that were actually available on the date the dbt version was released by setting a
DBTENV_SIMULATE_RELEASE_DATE=true
environment variable, or specifying--simulate-release-date
when runningdbtenv install
. This can help if newer versions of dbt's dependencies are causing installation problems. - By default dbtenv uses whichever Python version it was installed with to install dbt, but that can be changed by setting a
DBTENV_PYTHON
environment variable to the path of a different Python executable, or specifying--python <path>
when runningdbtenv install
.
Switching between dbt versions
Adapter type
If a dbtenv command is invoked from within a dbt project, dbtenv will try to determine the in-use adapter from the default set for the project's profile in profiles.yml
. If the --adapter
argument is set in the dbt command passed to dbtenv execute
, dbtenv will use that adapter's type instead. If running outside of a dbt project, a pip specifier should be passed to dbtenv execute's --dbt
argument so that dbtenv knows which adapter to use.
dbt version
dbtenv determines which dbt version to use by trying to read it from the following sources, in this order, using the first one it finds:
- The
dbtenv execute
command's optional--dbt <version>
argument. - A
DBT_VERSION
environment variable. - A
.dbt_version
file in the dbt project directory. - The dbt version requirements of the dbt project.
- If the dbt version requirements specify a range of versions rather than an exact version, then dbtenv will try to read a preferred dbt version from the sources below and will use that version if it's compatible with the requirements.
- The first
.dbt_version
file found by searching the dbt project's parent directories. - The
~/.dbt/version
file. - The max installed dbt version (preferring stable versions).
- The max installable dbt version (preferring stable versions).
You can:
- Run
dbtenv version
to show which dbt version dbtenv determines dynamically based on the current environment. - Run
dbtenv which
to show the full path to the executable of the dbt version dbtenv determines dynamically based on the current environment. - Run
dbtenv version --global <version>
to set the dbt version globally in the~/.dbt/version
file. The<version>
can be either a dbt version (e.g. 1.0.0) or full pip specifier (e.g. dbt-snowflake==1.0.0). dbtenv will attempt to automatically detect the required adapter or version from the environment if not specified. - Run
dbtenv version --local <version>
to set the dbt version for the current directory in a.dbt_version
file. The<version>
can be either a dbt version (e.g. 1.0.0) or full pip specifier (e.g. dbt-snowflake==1.0.0). dbtenv will attempt to automatically detect the required adapter or version from the environment if not specified.
Running dbt through dbtenv
Run dbtenv execute -- <dbt arguments>
to execute the dbt version determined dynamically based on the current environment, or run dbtenv execute --dbt <version> -- <dbt arguments>
to execute the specified dbt version.
For example:
dbtenv execute -- run
will executedbt run
using the version determined dynamically based on the current environment.dbtenv execute --dbt 1.0.0 -- run
will executedbt run
using dbt 1.0.0, automatically detecting the required adapter from the default target inprofiles.yml
.dbtenv execute --dbt 1.0.0 -- run --target prod
will executedbt run
using dbt 1.0.0, using the required adapter for the 'prod' target inprofiles.yml
.dbtenv execute --dbt 1.0.0==dbt-bigquery -- run
will executedbt run
using dbt-bigquery==1.0.0.
Important: It's highly recommended to put two dashes with spaces on both sides before the list of dbt arguments (as shown in the examples above) so that dbtenv doesn't try to interpret the dbt arguments itself.
Running dbt with dbtenv more seamlessly
For a more seamless experience you can define a dbt
alias or function in your shell to run dbtenv execute -- <dbt arguments>
and dynamically determine which dbt version to use whenever you type dbt commands like dbt run
or dbt test
.
Some examples:
- In bash you could add the following alias in your
~/.bash_profile
file:alias dbt='dbtenv execute --'
- In Windows PowerShell aliases can't include additional arguments, but you could add the following function in your
~\Documents\PowerShell\Microsoft.PowerShell_profile.ps1
file:function dbt { dbtenv execute -- @Args }
Note that after adding such a dbt
alias/function to your shell profile you'll need to reload the profile to activate it (e.g. by running . ~/.bash_profile
in bash, or . $PROFILE
in PowerShell).
Uninstalling dbt versions
You can run dbtenv versions --installed
to list the versions of dbt that dbtenv has installed in Python virtual environments and/or with Homebrew, and then run dbtenv uninstall <version>
to uninstall a version.
Development
Development setup
- Clone this repository onto your computer.
- Install Poetry
pipx install poetry
(What is pipx?) - Install this project into a virtual environment
poetry install
- Activate the virtual environment
poetry shell
- Any
dbtenv
commands will run using the local version of the project.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.