botmaro-secrets-manager 0.2.0

A standalone secret management tool for multi-environment deployments with Google Secret Manager

Botmaro Secrets Manager

A standalone, environment-aware secret management tool built on Google Secret Manager (GSM). Designed for multi-environment deployments with support for both GitHub Actions workflows and local development.

Features

• 🔐 Multi-environment support - Manage secrets across staging, prod, dev, and custom environments
• 🎯 Project scoping - Organize secrets by project within environments
• 🔄 Version control - Leverage GSM's built-in versioning
• 🚀 CI/CD ready - Bootstrap secrets in GitHub Actions or any CI/CD pipeline
• 🛠️ CRUD operations - Full create, read, update, delete support via CLI
• 📦 Pip installable - Install as a standalone package
• 🎨 Rich CLI - Beautiful, user-friendly command-line interface
• 🔒 IAM integration - Automatic service account access management

Installation

From source (development)

# Clone the repository
git clone https://github.com/B9ice/botmaro-gcp-secret-manager.git
cd botmaro-gcp-secret-manager

# Install in development mode
pip install -e .

From PyPI (when published)

pip install botmaro-secrets-manager

Quick Start

1. Create a configuration file

Create a secrets.yml file defining your environments and secrets:

version: "1.0"

environments:
  staging:
    name: staging
    gcp_project: your-gcp-project-staging
    global_secrets:
      - name: API_KEY
        required: true
      - name: DATABASE_URL
        required: true

See secrets.example.yml for a complete example.

2. Bootstrap your environment

# Load all secrets for staging environment
secrets-manager bootstrap staging

# Bootstrap with project scope
secrets-manager bootstrap staging --project myapp

# Export to a .env file
secrets-manager bootstrap staging --output .env.staging

3. Manage secrets

# Set a secret
secrets-manager set staging.API_KEY --value "sk-123456"

# Set a project-scoped secret
secrets-manager set staging.myapp.DATABASE_URL --value "postgres://..."

# Get a secret
secrets-manager get staging.API_KEY --reveal

# List all secrets
secrets-manager list staging

# Delete a secret
secrets-manager delete staging.OLD_KEY --force

Usage

Command-Line Interface

Bootstrap Command

Load all secrets for an environment:

secrets-manager bootstrap <environment> [OPTIONS]

Options:
  --project, -p TEXT      Project name to scope secrets
  --config, -c TEXT       Path to secrets config file
  --export/--no-export    Export secrets to environment variables [default: True]
  --runtime-sa TEXT       Runtime service account to grant access
  --deployer-sa TEXT      Deployer service account to grant access
  --output, -o TEXT       Output file for .env format
  --verbose, -v           Verbose output

Examples:

# Bootstrap staging
secrets-manager bootstrap staging

# Bootstrap with service account access grants
secrets-manager bootstrap staging \
  --runtime-sa [e-mail] \
  --deployer-sa [e-mail]

# Save to .env file
secrets-manager bootstrap prod --output .env.production

Set Command

Create or update a secret:

secrets-manager set <target> [OPTIONS]

Target format:
  - env.SECRET_NAME              (environment-scoped)
  - env.project.SECRET_NAME      (project-scoped)

Options:
  --value, -v TEXT     Secret value (or read from stdin)
  --config, -c TEXT    Path to secrets config file
  --grant, -g TEXT     Service account to grant access (can be repeated)

Examples:

# Set an environment-scoped secret
secrets-manager set staging.API_KEY --value "sk-123456"

# Set a project-scoped secret
secrets-manager set staging.orchestrator.DATABASE_URL --value "postgres://..."

# Read from stdin
echo "secret-value" | secrets-manager set staging.MY_SECRET

# Grant access to service accounts
secrets-manager set staging.API_KEY --value "sk-123" \
  --grant [e-mail] \
  --grant [e-mail]

Get Command

Retrieve a secret value:

secrets-manager get <target> [OPTIONS]

Options:
  --version TEXT       Secret version to retrieve [default: latest]
  --config, -c TEXT    Path to secrets config file
  --reveal             Show the full secret value

Examples:

# Get latest version (masked by default)
secrets-manager get staging.API_KEY

# Reveal full value
secrets-manager get staging.API_KEY --reveal

# Get specific version
secrets-manager get staging.API_KEY --version 2 --reveal

List Command

List all secrets in an environment:

secrets-manager list <environment> [OPTIONS]

Options:
  --project, -p TEXT   Project name to filter by
  --config, -c TEXT    Path to secrets config file
  --reveal             Show secret values

Examples:

# List all staging secrets
secrets-manager list staging

# List project-specific secrets
secrets-manager list staging --project orchestrator

# Show values (masked by default)
secrets-manager list staging --reveal

Delete Command

Delete a secret:

secrets-manager delete <target> [OPTIONS]

Options:
  --config, -c TEXT    Path to secrets config file
  --force, -f          Skip confirmation

Examples:

# Delete with confirmation prompt
secrets-manager delete staging.OLD_API_KEY

# Force delete
secrets-manager delete staging.OLD_API_KEY --force

Grant Access Command

Grant access to all secrets in an environment or project:

secrets-manager grant-access <target> [OPTIONS]

Target format:
  - env                  (all environment-level secrets)
  - env.project          (all project-scoped secrets)

Options:
  --sa TEXT            Service account email to grant access (can be repeated)
  --config, -c TEXT    Path to secrets config file
  --force, -f          Skip confirmation prompt

Examples:

# Grant access to all staging secrets
secrets-manager grant-access staging \
  --sa [e-mail]

# Grant to multiple service accounts
secrets-manager grant-access staging \
  --sa [e-mail] \
  --sa [e-mail]

# Grant access to all project-specific secrets
secrets-manager grant-access staging.myapp \
  --sa [e-mail]

# Skip confirmation
secrets-manager grant-access staging --sa [e-mail] --force

Note: This command grants the secretmanager.secretAccessor role, allowing the service account to read secret values.

Check Command

Validate secrets configuration and state:

secrets-manager check <environment> [OPTIONS]

Options:
  --project, -p TEXT      Project name to scope secrets
  --config, -c TEXT       Path to secrets config file
  --workflows, -w TEXT    Path to workflow file or .github/workflows directory
  --verbose, -v           Show detailed findings

This command performs comprehensive validation:

• ✅ All required secrets exist in GSM
• ✅ No placeholder values in secrets (e.g., "PLACEHOLDER_", "TODO", "changeme")
• ✅ No placeholder service accounts
• ✅ Service accounts have proper access to secrets
• ✅ Workflow secrets are defined in config (when --workflows provided)

Examples:

# Check all staging secrets
secrets-manager check staging

# Check with project scope
secrets-manager check staging --project myapp

# Validate against workflow files
secrets-manager check staging --workflows .github/workflows

# Check specific workflow file
secrets-manager check prod --workflows .github/workflows/deploy.yml --verbose

# Use in CI/CD to prevent deployment with placeholders
secrets-manager check prod && echo "Secrets validated, deploying..."

Exit codes:

• 0 - All checks passed
• 1 - Validation failed (missing secrets, placeholders, or access issues)

Example output:

Validation Summary:
⚠️  2 placeholder secrets
❌ 1 missing secrets
❌ 1 service account access issues

⚠️  Placeholder Secrets (2):
  • OPENAI_API_KEY: PLACEHOLDER_OPENAI_...
  • DATABASE_URL: TODO-replace-me

❌ Missing Secrets (1):
  • STRIPE_SECRET_KEY

❌ Missing Service Account Access (1):
  • API_KEY → [e-mail]

❌ Validation failed with errors

Use cases:

• Pre-deployment validation in CI/CD pipelines
• Verify secrets before running bootstrap
• Audit service account access
• Ensure workflow secrets are properly configured
• Catch configuration issues early

GitHub Actions Integration

Example Workflow

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install secrets manager
        run: pip install -e .

      - name: Authenticate to Google Cloud
        uses: google-github-actions/auth@v1
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}

      - name: Bootstrap secrets
        run: |
          secrets-manager bootstrap staging \
            --runtime-sa [e-mail] \
            --config secrets.yml

      - name: Deploy application
        run: |
          # Your deployment commands here
          # All secrets are now available as environment variables
          echo "Deploying with SUPABASE_URL=$SUPABASE_URL"

Setting up initial secrets

Before running in GitHub Actions, you need to populate secrets using this tool's CLI.

Note: This tool is your interface to Google Secret Manager. You only need gcloud for authentication, not for managing secrets.

# 1. Authenticate to GCP (required for API access)
gcloud auth application-default login

# 2. Use secrets-manager CLI to create and manage secrets
secrets-manager set staging.SUPABASE_URL --value "https://xxx.supabase.co"
secrets-manager set staging.SUPABASE_ANON_KEY --value "eyJxxx..."
secrets-manager set staging.SUPABASE_SERVICE_ROLE_KEY --value "eyJxxx..."

# 3. Verify secrets were created
secrets-manager list staging --reveal

The secrets-manager CLI automatically creates secrets in Google Secret Manager with proper naming conventions and IAM permissions.

Configuration Reference

Environment Configuration

environments:
  <env-name>:
    name: string              # Environment name
    gcp_project: string       # GCP project ID
    prefix: string            # Optional: secret name prefix (default: botmaro-{env})

    # Optional: Service accounts that need access to all environment-level secrets
    # These will be automatically granted secretAccessor role during bootstrap
    service_accounts:
      - [e-mail]
      - [e-mail]

    global_secrets:           # Environment-level secrets
      - name: string          # Secret name
        description: string   # Optional description
        required: boolean     # Whether secret is required (default: true)
        default: string       # Optional default value

    projects:                 # Project-specific secrets
      <project-name>:
        project_id: string

        # Optional: Project-specific service accounts
        # If not specified, environment-level service accounts will be used
        service_accounts:
          - [e-mail]

        secrets:
          - name: string
            description: string
            required: boolean
            default: string

Automatic Service Account Access Grants

The tool automatically grants secretAccessor role to service accounts configured in secrets.yml during bootstrap. This ensures that your runtime and deployment service accounts always have access to the secrets they need.

Configuration Example:

environments:
  staging:
    name: staging
    gcp_project: my-project-staging

    # These service accounts will automatically get access to all staging secrets
    service_accounts:
      - [e-mail]
      - [e-mail]

    global_secrets:
      - name: API_KEY
        required: true

    projects:
      myapp:
        project_id: myapp

        # Additional service accounts for this project's secrets
        service_accounts:
          - [e-mail]

        secrets:
          - name: DATABASE_URL
            required: true

How it works:

1. During secrets-manager bootstrap, the tool checks if configured service accounts have access to each secret
2. If access is missing, it automatically grants the secretAccessor role
3. If access already exists, it skips (idempotent operation)
4. For project-scoped secrets, both environment-level and project-level service accounts get access

Manual override:

You can also specify additional service accounts at runtime:

secrets-manager bootstrap staging \
  --runtime-sa [e-mail] \
  --deployer-sa [e-mail]

These will be granted access in addition to the service accounts configured in secrets.yml.

Secret Naming Convention

Secrets are stored in GSM using a double-hyphen (--) convention for hierarchical separation, while single hyphens (-) or underscores (_) are used within component names:

• Environment-scoped: {prefix}--{SECRET_NAME}

  • Example: botmaro-staging--API_KEY
  • Example: my-longer-prefix--SUPABASE_URL

• Project-scoped: {prefix}--{project}--{SECRET_NAME}

  • Example: botmaro-staging--orchestrator--DATABASE_URL
  • Example: my-company-prod--my-service--API_KEY

Where {prefix} defaults to botmaro-{environment} but can be customized.

Why double-hyphen?

• Provides clear, unambiguous hierarchical separation
• Easy to parse: secret_id.split('--')
• GSM-compliant (only allows letters, numbers, -, and _)
• Visually distinct from naming hyphens

Local Development

Setup

# Install dependencies
pip install -e ".[dev]"

# Set up secrets config
cp secrets.example.yml secrets.yml
# Edit secrets.yml with your configuration

# Authenticate to GCP
gcloud auth application-default login

Running locally

# Bootstrap local environment
secrets-manager bootstrap dev

# Or use a custom config
secrets-manager bootstrap dev --config ./my-secrets.yml

Python API

You can also use the library programmatically:

from secrets_manager import SecretsManager, SecretsConfig

# Load from config file
config = SecretsConfig.from_file("secrets.yml")
manager = SecretsManager(config)

# Bootstrap environment
secrets = manager.bootstrap(env="staging", export_to_env=True)

# Set a secret
manager.set_secret(
    env="staging",
    secret="API_KEY",
    value="sk-123456",
    grant_to=["[e-mail]"]
)

# Get a secret
value = manager.get_secret(env="staging", secret="API_KEY")

# List secrets
secrets = manager.list_secrets(env="staging", project="myapp")

# Grant access to all secrets in an environment
result = manager.grant_access_bulk(
    env="staging",
    service_accounts=["[e-mail]", "[e-mail]"]
)

IAM Permissions

The service account running the secrets manager needs:

• secretmanager.admin - To create/update/delete secrets
• iam.serviceAccountUser - To grant access to other service accounts

Grant these roles:

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:[e-mail]" \
  --role="roles/secretmanager.admin"

Runtime service accounts need:

• secretmanager.secretAccessor - To read secrets (granted automatically by the tool)

Troubleshooting

Authentication errors

Make sure you're authenticated to GCP:

gcloud auth application-default login

For CI/CD, use service account key or Workload Identity.

Secret not found

Check that:

1. The secret exists in GSM: gcloud secrets list --project PROJECT_ID
2. Your config file matches the environment name
3. The GCP project ID is correct

Permission denied

Ensure your service account has the required IAM roles listed above.

Releasing

This project uses tag-based releases. When you're ready to publish a new version:

1. Update version and changelog

# Update version in pyproject.toml
vim pyproject.toml  # Change version = "0.1.0" to "0.1.1"

# Update CHANGELOG.md
vim CHANGELOG.md    # Add release notes under [Unreleased]

2. Commit changes

git add pyproject.toml CHANGELOG.md
git commit -m "Bump version to 0.1.1"
git push origin main

3. Create and push tag

# Create an annotated tag
git tag -a v0.1.1 -m "Release v0.1.1: Description of changes"

# Push the tag (this triggers PyPI publish!)
git push origin v0.1.1

What happens automatically:

1. Version verification - Ensures tag matches pyproject.toml version
2. Build package - Creates wheel and source distribution
3. Publish to PyPI - Uploads to PyPI (requires PYPI_API_TOKEN secret)
4. Create GitHub Release - Creates release page with changelog and assets

Manual publish (if needed)

You can also manually trigger the publish workflow from GitHub Actions UI:

• Go to Actions → Publish to PyPI → Run workflow

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details.

Support

For issues and questions:

• GitHub Issues: https://github.com/B9ice/botmaro-gcp-secret-manager/issues
• Documentation: https://github.com/B9ice/botmaro-gcp-secret-manager