turbovault4dbt 0.2.7

CLI for Turbovault 4 DBT

TurboVault4dbt_cli

---

What is TurboVault4dbt_cli?

TurboVault4dbt_cli is an open-source CLI tool that automatically generates dbt models according to datavault4dbt templates. It uses a metadata input of your Data Vault 2.0 from one of the supported databases and creates ready-to-process dbt-models.

---

Prerequisites

• Python 3.8+
• Metadata prepared in one of the supported formats (see below)
• dbt project
• datavault4dbt dbt package

---

Supported Metadata Formats

• Excel files: .xls, .xlsx
• CSV files: folder of CSVs (one per sheet/table)

---

How does my metadata need to look like?

Your metadata needs to be stored in the following tables/worksheets/files:

• Source Data
• Standard Hubs
• Standard Links
• Non-Historized Links
• Standard Satellites
• Non-Historized Satellites
• Multi-Active Satellites
• Point-In-Time Tables
• Reference Tables

---

Installation

You can install TurboVault4dbt_cli directly from PyPI:

pip install turbovault4dbt

Or install from source for development:

git clone https://github.com/ScalefreeCOM/turbovault4dbt.git
cd turbovault4dbt
pip install -e .

---

Publishing to PyPI

1. Build the package (inside your project directory):

   python -m build
   

2. Upload to PyPI using Twine:

   pip install twine  # if not already installed
   twine upload dist/*
   

3. (Optional) Test upload to TestPyPI first:

   twine upload --repository testpypi dist/*
   

4. After upload, install your package from PyPI:

   pip install turbovault4dbt
   

For more details, see the official Python packaging docs.

---

Quickstart: Using the CLI

1. Prepare your metadata

• Prepare your metadata as Excel (.xls or .xlsx) or as a folder of CSV files (one CSV per sheet/table, filenames matching required sheets).
• (See metadata_ddl/ for example templates.)

2. Run TurboVault4dbt_cli

Basic usage:

# List all nodes in your metadata (Excel or CSV)
turbovault list -f xlsx path/to/your.xlsx
turbovault list -f csv path/to/your_csv_folder

# Generate dbt models for all nodes
turbovault run -f xlsx path/to/your.xlsx
turbovault run -f csv path/to/your_csv_folder

# Generate dbt models for selected nodes
turbovault run -f xlsx path/to/your.xlsx -s hub1 link1 sat1
turbovault run -f csv path/to/your_csv_folder -s '+hub1' '@sat1'

# Specify output directory (all generated files will go here unless overridden by metadata)
turbovault run -f xlsx path/to/your.xlsx --output-dir my_output_dir

Command reference:

• turbovault run -f {xls|xlsx|csv} <input> [-s <selectors>] [--output-dir <dir>]
  Generate dbt models for all or selected nodes.
• turbovault list -f {xls|xlsx|csv} <input> [-s <selectors>]
  List resolved nodes for a selector (dry run).

Arguments:

• -f, --format: Input format. Must be one of: xls, xlsx, csv
• input: Path to Excel file (.xls/.xlsx) or folder containing CSV files (for csv)
• -s, --select: (Optional) Node selectors (space-separated).
  Examples: hub1, +sat1, hub2+, @masat3
• --output-dir: (Optional) Output directory for all generated files. By default, all models will be placed in this directory. If your metadata (Excel/CSV) includes an output_dir column for an asset, that asset's file will be placed in a subdirectory under --output-dir (e.g., --output-dir models and output_dir column value 01_RawVault/Sales results in models/01_RawVault/Sales/Hub1.sql). If output_dir is not present in your metadata, all files go directly into the main output directory. Both / and \ are supported as path separators and absolute paths are automatically sanitized to prevent writing outside the output directory.

Selector syntax:

• A+ — node A and all descendants
• +A — node A and all ancestors
• @A — node A, all ancestors, and all descendants
• Multiple selectors can be space-separated

---

Regression Testing

To run the regression test suite:

pip install -r requirements-test.txt
pytest tests/

• Add new test cases by creating folders in tests/ with input.xlsx and expected_output/.
• Negative test cases (expected failures) are also supported.

---

Project Structure

• src/turbovault4dbt/ — all source code
• tests/ — regression test cases
• pyproject.toml, requirements.txt, etc. — project config in root

---

Releases

See PyPI Releases

---

License

See LICENSE

---

Need Help?

• Open an issue
• Wiki