bicep-whatif-advisor 3.5.0

AI-powered Azure Bicep What-If deployment advisor with automated safety reviews

bicep-whatif-advisor

AI-powered deployment safety gate for Azure Bicep and ARM templates. Pipe Azure What-If output through an LLM (Anthropic Claude, Azure OpenAI, or Ollama) to detect infrastructure drift, validate PR intent alignment, and evaluate custom risk dimensions. Integrates into GitHub Actions and Azure DevOps with zero-config platform detection, automatic PR comments, and configurable risk thresholds. Also works as a standalone CLI for local What-If analysis.

How It Works

Pipe Azure What-If output to the tool in your CI/CD pipeline. It auto-detects the platform, collects PR metadata and code diff, then sends everything to the LLM for analysis across two built-in risk categories:

• Infrastructure Drift - Detects changes not in your code (out-of-band modifications)
• PR Intent Alignment - Ensures changes match PR description

You can also add custom risk assessment agents via markdown files to evaluate additional dimensions like compliance, cost, risky operations, or naming conventions. See Custom Agents below.

Known Azure What-If noise (etag, provisioningState, etc.) is filtered before LLM analysis. Results are posted as a PR comment and the pipeline exits with code 0 (safe) or 1 (unsafe) based on configurable thresholds.

Example PR Comment:

production What-If Summary (non-blocking)

Risk Assessment	Risk Level	Key Concerns
Infrastructure Drift	High	Existing Paris Key Vault being modified when not present in code diff
PR Intent Alignment	High	PR intends to add Berlin branch but What-If shows Paris branch changes

Summary: The What-If output shows modifications to existing Paris branch resources rather than creating new Berlin branch resources. Most changes appear to be template refactoring (hardcoded to dynamic references) and What-If noise (retention policies, metadata). The only significant change is disabling public network access on an existing Key Vault.

📁 View changed resources (1 High Confidence)

...

⚠️ Potential Azure What-If Noise (10 Low Confidence)

...

📂 Raw What-If Output

...

Verdict: ❌ UNSAFE

Reasoning: This deployment is unsafe due to a complete mismatch between PR intent (adding Berlin branch) and What-If output (modifying existing Paris Key Vault). The changes indicate both infrastructure drift and unintended modifications to existing resources not mentioned in the PR description.

---

Generated by bicep-whatif-advisor

Quick Start

GitHub Actions:

- name: Deployment Safety Gate
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: |
    pip install bicep-whatif-advisor[anthropic]
    az deployment group what-if \
      --resource-group ${{ vars.AZURE_RESOURCE_GROUP }} \
      --template-file main.bicep \
      --exclude-change-types NoChange Ignore \
      | bicep-whatif-advisor

Azure DevOps:

- script: |
    pip install bicep-whatif-advisor[anthropic]
    az deployment group what-if \
      --resource-group $(RESOURCE_GROUP) \
      --template-file main.bicep \
      --exclude-change-types NoChange Ignore \
      | bicep-whatif-advisor
  env:
    ANTHROPIC_API_KEY: $(ANTHROPIC_API_KEY)
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Configuration

Risk Thresholds

Each risk bucket has an independent threshold: low, medium, high (default: high — most permissive).

# Block on medium or high risk
bicep-whatif-advisor \
  --drift-threshold medium \
  --intent-threshold medium

Skip individual buckets with --skip-drift or --skip-intent.

Custom Agents

Add custom risk assessment dimensions using markdown files with YAML frontmatter. Each file defines a new risk bucket that runs alongside the built-in ones:

---
id: compliance
display_name: Compliance Review
default_threshold: high
---

**Compliance Risk:**
Evaluate whether changes comply with organizational policies.

Risk levels for compliance:
- high: Encryption disabled, public access enabled, data residency violations
- medium: Policy assignment changes, unapproved resource types
- low: Tag updates for compliance metadata, monitoring additions

Point the tool at a directory of agent files:

bicep-whatif-advisor --ci \
  --agents-dir ./agents/ \
  --agent-threshold compliance=medium

Custom agents support --agent-threshold for per-agent thresholds and --skip-agent to disable individual agents. See the User Guide for details.

LLM Providers

Defaults to Anthropic Claude. Also supports Azure OpenAI and Ollama:

# Azure OpenAI
pip install bicep-whatif-advisor[azure]
bicep-whatif-advisor --provider azure-openai

# Local Ollama
pip install bicep-whatif-advisor[ollama]
bicep-whatif-advisor --provider ollama --model llama3.1

Config File

Define settings in a YAML file instead of passing many CLI flags. CLI flags always override config values:

# whatif-config.yaml
provider: azure-openai
ci: true
drift_threshold: medium
agents_dir: ./agents

bicep-whatif-advisor --config-file whatif-config.yaml

Additional Options

# Output formats: table (default), json, markdown
bicep-whatif-advisor --format json

# Include raw What-If output in PR comment
bicep-whatif-advisor --include-whatif

# Label comments for multi-environment pipelines
bicep-whatif-advisor --comment-title "Production"

# Non-blocking mode (report without failing pipeline)
bicep-whatif-advisor --no-block

See the User Guide for all CLI flags and the CI/CD Integration Guide for complete pipeline setup.

Documentation

• Quick Start - Get running in 5 minutes
• User Guide - Complete feature reference and CLI flags
• CI/CD Integration - Pipeline setup for GitHub Actions, Azure DevOps, etc.
• Risk Assessment - Deep dive into AI risk evaluation
• Technical Specifications - Architecture, design, and module specs

Support

• Issues: Report bugs or request features via repository issues
• Contributing: Pull requests welcome!
• License: MIT - see LICENSE for details