governor-bq 0.3.7

BigQuery cost optimization platform — detect opportunities, generate solutions, create PRs

Governor

BigQuery cost optimization platform for dbt projects. Governor syncs your warehouse metadata, detects cost and performance issues, generates solutions (deterministic or AI-assisted), and opens pull requests with the fixes.

Install

pip install governor-bq

Requires Python 3.14+ and Docker (for automatic PostgreSQL).

Quick start

governor init

That's it. The wizard walks you through:

1. Database — starts PostgreSQL via Docker (or uses yours)
2. Google Cloud — opens browser for Google login (or reads a service account file)
3. GitHub — opens browser for GitHub auth (or paste a token, or skip)
4. Project — picks your BigQuery project and dbt manifest
5. Launch — starts the server and opens http://localhost:8000

What Governor detects

Category	Issues
Performance	shuffle spill, slot contention, join explosion, partition pruning
Dead code	dead CTEs, dead columns, dead window expressions, unused aggregations
SQL cleanup	redundant ORDER BY, unused JOINs
Storage	billing model optimization

What Governor does about it

• Deterministic fixes (no LLM) — add partition/cluster, remove dead code, switch billing model
• AI-assisted fixes (Gemini) — complex rewrites with safety validation
• Pull requests — opens PRs on your dbt repo with the proposed changes
• Verified savings — compares BigQuery costs before and after merge

Day-to-day commands

governor start          # start server + database
governor stop           # stop server
governor stop --all     # stop server + database container
governor status         # check what's running
governor sync           # trigger a manual sync
governor auth refresh   # re-authenticate Google or GitHub

Main surfaces

Page	What it shows
/dashboard	Global spend, opportunities, savings
/opportunities	Cross-project opportunity explorer
/projects/{id}	Project-level dashboard
/projects/{id}/opportunities/{id}	Opportunity detail, diff, verification SQL
/admin/configurations	Project config, rules, guardrails
/prs	Pull request tracking and cost verification

Install from source (contributors)

git clone https://github.com/simple-machines/governor.git
cd governor
uv pip install -e .
governor init

The governor init command handles everything interactively:

1. Database — starts a PostgreSQL container via Docker (or uses yours)
2. Google Cloud — opens your browser for Google login (or reads a service account file)
3. GitHub — opens your browser for GitHub device flow (or paste a token, or skip)
4. Project — creates your first project from a local dbt manifest
5. Launch — starts the server and opens http://localhost:8000

After the first run, use these to manage your instance:

governor start          # start server + database
governor stop           # stop server
governor stop --all     # stop server + database container
governor status         # check what's running
governor sync           # trigger a manual sync
governor auth refresh   # re-authenticate Google or GitHub

See SETUP.md for the full setup guide including Cloud Run deployment.

Local development (Devbox)

For contributors working on Governor itself, local development is managed with Devbox.

Prerequisites

• Devbox: https://www.jetify.com/devbox
• Docker (for database, or bring your own PostgreSQL)

Enter the environment

devbox shell

The Devbox shell runs uv sync automatically.

Database initialization

Initialize PostgreSQL from your host shell:

devbox run -- initdb
devbox services start postgresql
devbox run -- createdb -h localhost -p 5432 governor

Then, inside devbox shell, run migrations:

make migrate

Start the app

Build CSS and run the server:

make run

If you want the Tailwind watcher in a separate process:

bun run tailwind:watch

If you prefer to start background services together:

devbox services up
process-compose up

If you need to override the database connection:

DATABASE_URL="postgresql+psycopg://localhost:5432/governor" make run

Common commands

make migrate
make run
make test
make test-serial
uv run ruff format
uv run ruff check

Seed an admin user

uv run python scripts/seed_admin.py --email [e-mail]

You will be prompted for a password unless you pass --password.

GitHub integration setup

Governor integrates with GitHub to create pull requests and manage repository content. This requires creating a GitHub App and configuring Governor with its credentials.

Prerequisites

• GitHub organization administrator access
• Governor deployment with environment variable access

Step 1: Create GitHub App

1. Navigate to your organization's GitHub App settings: https://github.com/organizations/{your-org}/settings/apps

2. Click New GitHub App

3. Fill in the required fields:

   • GitHub App name: Governor - {YourOrgName} (must be globally unique)
   • Homepage URL: Your Governor instance URL (e.g., https://governor.example.com)

4. Under Webhook:

   • Uncheck "Active" (webhooks are not required)

5. Under Permissions → Repository permissions:

   • Contents: Read and write
   • Pull requests: Read and write
   • Metadata: Read-only (auto-selected)

6. Under Where can this GitHub App be installed?:

   • Select Only on this account

7. Click Create GitHub App

Step 2: Generate Private Key

1. After creation, you'll be on the app settings page
2. Note the App ID displayed near the top (e.g., 123456)
3. Scroll down to Private keys section
4. Click Generate a private key
5. A .pem file will download automatically
6. Store this file securely - it cannot be retrieved again

Step 3: Install App to Organization

1. In the left sidebar, click Install App
2. Click Install next to your organization
3. Choose repository access:
   • All repositories or
   • Only select repositories (choose repos Governor will manage)
4. Click Install
5. Note the Installation ID from the URL after installation:
   • URL format: https://github.com/organizations/{org}/settings/installations/{installation_id}

Step 4: Configure Governor

Set the following environment variables in your Governor deployment:

Shell export:

# GitHub App ID from Step 2
export GITHUB_APP_ID="123456"

# Contents of the .pem file from Step 2
export GITHUB_APP_PRIVATE_KEY="$(cat /path/to/private-key.pem)"

# Installation ID from Step 3
export GITHUB_APP_INSTALLATION_ID="12345678"

.env file format:

GITHUB_APP_ID=123456
GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA...
...
-----END RSA PRIVATE KEY-----"
GITHUB_APP_INSTALLATION_ID=12345678

Required Permissions Reference

Permission	Level	Purpose
Contents	Read & Write	Create branches, commit changes
Pull requests	Read & Write	Create and manage pull requests
Metadata	Read-only	Access basic repository information

Verifying Configuration

1. Log in to Governor as an admin
2. Navigate to Admin → System Verification
3. Click Run Verification
4. Confirm the GitHub check shows Pass status

Troubleshooting

Check shows "Configuration incomplete"

Cause: One or more environment variables not set.

Solution: Verify all three variables are set:

echo $GITHUB_APP_ID
echo $GITHUB_APP_INSTALLATION_ID
# Don't echo the private key for security
test -n "$GITHUB_APP_PRIVATE_KEY" && echo "Private key is set"

Check shows "Private key is malformed"

Cause: Private key format is incorrect.

Solution:

• Ensure key starts with -----BEGIN RSA PRIVATE KEY----- or -----BEGIN PRIVATE KEY-----
• Check for proper newlines (not literal \n strings in shell)
• Verify no extra whitespace or truncation

Check shows "Authentication failed"

Cause: App ID doesn't match private key.

Solution:

• Verify GITHUB_APP_ID matches the app settings page
• Generate a new private key if needed
• Ensure you're using the correct app's key

Check shows "App is suspended"

Cause: GitHub App has been suspended.

Solution:

1. Go to GitHub organization settings → GitHub Apps
2. Find your app and check its status
3. Go to Advanced and click "Unsuspend" if available

Check shows "Rate limit exceeded"

Cause: Too many API requests.

Solution:

• Wait for the rate limit to reset (typically 1 hour)
• Check for other processes using the same App ID

Check shows "Connection timeout"

Cause: Network connectivity issues.

Solution:

• Verify Governor can reach api.github.com port 443
• Check firewall rules and proxy settings
• Test with: curl -I https://api.github.com