acc-cli 0.1.3

ACC-02: Databricks pipeline project scaffolding and build CLI

ACC CLI — Databricks Project Generator

A command-line tool to scaffold production-ready Databricks pipeline projects from templates. Built with Typer and Cookiecutter.

---

Table of Contents

• Features
• Project Structure
• Installation
• Usage
• Available Templates
• Storage Options
• Development Setup
• Publishing a New Version
• Versioning Convention

---

Features

• 🚀 Scaffold a Databricks ETL pipeline project in seconds
• 📁 Generates a fully structured Python package with src/ layout
• ☁️ Supports multiple storage backends: UC, DBFS, S3, ADLS
• 🔧 Includes Makefile, pyproject.toml, configs, Databricks job JSON, and tests out of the box
• 📦 Installable via pip — works as a global CLI tool

---

Project Structure

python-deployment-package/
├── .gitignore
├── README.md
├── pyproject.toml           # Package metadata & entry point
├── requirements.txt         # Runtime dependencies
├── dist/                    # Built distributions (wheel + sdist)
├── docs/
│   ├── cli-spec.md
│   ├── project-structure.md
│   └── tech-decisions.md
├── src/
│   └── acc_cli/             # Main CLI package
│       ├── __init__.py
│       ├── cli.py           # Typer app & entry point
│       ├── config.py        # Templates & storage config
│       ├── commands/
│       │   ├── __init__.py
│       │   └── init.py      # `acc init` command
│       ├── utils/
│       │   ├── __init__.py
│       │   └── utils.py     # Input collection, cookiecutter wrapper
│       └── templates/
│           └── etl/         # ETL pipeline cookiecutter template
│               ├── cookiecutter.json
│               └── {{cookiecutter.project_slug}}/
│                   ├── Makefile
│                   ├── pyproject.toml
│                   ├── README.md
│                   ├── requirements.txt
│                   ├── requirements-dev.txt
│                   ├── .gitignore
│                   ├── configs/
│                   │   ├── dev.yaml
│                   │   └── prod.yaml
│                   ├── jobs/
│                   │   └── databricks_job.json
│                   ├── src/{{cookiecutter.project_slug}}/
│                   │   ├── main.py
│                   │   ├── pipelines/
│                   │   ├── tasks/
│                   │   └── utils/
│                   └── tests/
└── tests/
    ├── __init__.py
    └── test_cli.py

---

Installation

pip install acc-cli

Note (macOS): If acc is not found after install, add Python's bin directory to your PATH:

echo 'export PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

---

Usage

Start the scaffolder (default command)

acc

Explicit init subcommand

acc init

Help

acc --help
acc init --help

Interactive prompts

When you run acc init, you will be prompted for:

Prompt	Description	Default
Project name	Name of your new project	—
Version	Initial version	0.1.0
Author	Your name	—
Template	Project template to use	etl
Storage	Cloud storage backend	uc
Databricks workspace URL	Your workspace URL	—
Output directory	Where to create the project	current directory

Example session

ACC Project Scaffolder

Available templates:
  etl        — Extract-Transform-Load pipeline
  ml         — Machine learning pipeline
  utility    — Utility / helper library

Project name: my-sales-pipeline
Version [0.1.0]:
Author: Jane Doe
Template [etl / ml / utility] [etl]:
Storage [uc / dbfs / s3 / adls] [uc]: s3
Databricks workspace URL: https://adb-xxxx.azuredatabricks.net
Output directory [...]: /Users/jane/projects

Project created successfully!

Location:   /Users/jane/projects/my_sales_pipeline
Storage:    s3://bucket/wheels
Wheel path: s3://bucket/wheels/my-sales-pipeline-0.1.0-py3-none-any.whl

Next steps:
  cd /Users/jane/projects/my_sales_pipeline
  make install && make test

---

Available Templates

Key	Description	Status
etl	Extract-Transform-Load pipeline	✅ Available
ml	Machine learning pipeline	🚧 Coming soon
utility	Utility / helper library	🚧 Coming soon

---

Storage Options

Key	Resolved path
uc	dbfs:/Volumes/catalog/schema/volume
dbfs	dbfs:/FileStore/wheels
s3	s3://bucket/wheels
adls	abfss://[e-mail]/wheels

---

Development Setup

1. Clone the repository

git clone https://github.com/your-org/python-deployment-package.git
cd python-deployment-package

2. Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

3. Install in editable mode

pip install -e .

4. Verify the CLI works

acc --help

5. Run tests

pytest tests/ -v

---

Publishing a New Version

⚠️ PyPI does not allow re-uploading the same version. Always bump the version before building.

Step 1 — Bump the version in pyproject.toml

[project]
version = "0.1.2"   # ← change this

Step 2 — Clean previous builds

rm -rf dist/ build/ src/acc_cli.egg-info

Step 3 — Build the distribution

python3 -m build

This creates two files inside dist/:

• acc_cli-<version>-py3-none-any.whl — binary wheel (fast install)
• acc_cli-<version>.tar.gz — source distribution

Step 4 — Upload to PyPI

python3 -m twine upload dist/*

Twine will use credentials from ~/.pypirc. If that file doesn't exist, it will prompt for your API token.

Setting up ~/.pypirc (one-time)

[distutils]
index-servers = pypi

[pypi]
username = __token__
password = pypi-YOUR_API_TOKEN_HERE

Get your API token at https://pypi.org/manage/account/token/ The username must always be __token__ (literally).

Step 5 — Verify on PyPI

https://pypi.org/project/acc-cli/

---

Versioning Convention

This project follows Semantic Versioning (MAJOR.MINOR.PATCH):

Change type	Example
Bug fix / small restructure	0.1.1 → 0.1.2
New feature / new template	0.1.2 → 0.2.0
Breaking change	0.2.0 → 1.0.0

---

Dependencies

Package	Purpose
typer>=0.12	CLI framework
rich>=13	Terminal formatting
cookiecutter>=2.6	Project scaffolding from templates

---

License

Proprietary — All rights reserved.