ddutil 0.2.3

A CLI tool for managing DataDog AWS integrations

DataDog AWS Integration CLI

A command-line tool for managing DataDog AWS integrations with ease.

Features

• 🚀 Easy setup of DataDog integrations for AWS accounts
• 🔧 Automated IAM role and policy creation/management
• ✅ Comprehensive status checking and validation
  • IAM role and policy verification
  • DataDog account configuration validation
  • Settings comparison (regions, services, metrics, resources)
• 🔄 Update existing integrations (regions, services, metrics, resources)
• 🗑️ Clean deletion of integrations
• 🎨 Beautiful terminal output with tables and colors
• ⚙️ Flexible configuration via .env files or CLI arguments
• 🔒 Dry-run mode to preview changes before applying
• 📝 Verbose logging for debugging
• 📊 JSON output support for automation
• 🔐 Support for multiple AWS partitions (standard, GovCloud, China)
• ⚡ No complex YAML configuration required

Build Status

Installation

Using UV (Recommended)

UV is a blazing-fast Python package manager.

# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh
# or on Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Clone the repository
git clone <repository-url>
cd datadog-utility

# Create a virtual environment and install dependencies
uv venv
uv pip install -e .

Using pip

# Clone the repository
git clone <repository-url>
cd datadog-utility

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install the CLI tool
pip install -e .

After installation, the ddutil command will be available globally.

Quick Start

1. Set Up Environment Variables

Copy the example .env file and fill in your credentials:

cp .env.example .env

Edit .env with your values:

# Required
DD_API_KEY=your_datadog_api_key
DD_APP_KEY=your_datadog_app_key
AWS_ACCOUNT_ID=123456789012

# Optional but commonly used
DD_ACCOUNT_ID=your_dd_account_id
AWS_PROFILE=default
DD_IAM_ROLE_NAME=datadog-integration-role

2. Preview the Setup (Dry-Run)

ddutil setup --dry-run

3. Apply the Changes

ddutil setup

That's it! The tool will:

• Create the necessary IAM role and policies
• Configure the DataDog integration
• Update the IAM role with the external ID from DataDog

Usage

Available Commands

ddutil --help              # Show all available commands
ddutil --version           # Show version information
ddutil --license           # Show license information
ddutil -v, --verbose       # Enable verbose output for any command
ddutil -q, --quiet         # Suppress non-error output

Commands:

• setup - Set up DataDog integration for an AWS account
• status - Check the status of DataDog integration and validate configuration
• update - Update an existing DataDog integration
• delete - Delete DataDog integration for an AWS account

Quick Reference

# Setup with defaults from .env
ddutil setup

# Preview setup without making changes
ddutil setup --dry-run

# Check configuration status and validation
ddutil status

# Update configuration
ddutil update --regions us-east-1,us-west-2

# Delete integration (with confirmation)
ddutil delete --confirm

# Any command with verbose output
ddutil --verbose <command>

Setup Command

Set up DataDog integration for an AWS account:

# Using environment variables (.env file)
ddutil setup

# Override with CLI arguments
ddutil setup --account-id 123456789012 --dd-account-id YOUR_DD_ACCOUNT_ID

# Specify AWS profile
ddutil setup --profile aws-prod

# Dry-run mode (preview changes without applying)
ddutil setup --dry-run

# Verbose output (shows debug logs)
ddutil --verbose setup

# Specify regions and services
ddutil setup --regions us-east-1,us-west-2 --services AWS/Lambda,AWS/EC2

# Configure metric and resource collection
ddutil setup --metric-automute true --metric-collect-cloudwatch true \
  --resource-collect-cspm false --resource-collect-extended true

# Specify AWS partition (for GovCloud or China regions)
ddutil setup --partition aws-us-gov

Status Command

Check the status of an existing integration and validate configuration:

# Basic status check
ddutil status

# Check specific account
ddutil status --account-id 123456789012

# Use specific AWS profile
ddutil status --profile aws-prod

# JSON output for programmatic use
ddutil status --output json

# Verbose output with debug info
ddutil --verbose status

The status command validates:

• ✅ IAM role existence and policies
• ✅ DataDog account registration
• ✅ Configuration sync (regions, services, partition)
• ✅ Metric settings (automute, CloudWatch, custom metrics)
• ✅ Resource settings (CSPM, extended collection)
• ✅ Role name and External ID matching

Update Command

Update an existing integration:

# Update services monitored
ddutil update --services AWS/Lambda,AWS/EC2,AWS/RDS

# Update regions
ddutil update --regions us-west-2,eu-west-1

# Update metric settings
ddutil update --metric-collect-custom true --metric-automute false

# Update resource collection
ddutil update --resource-collect-cspm true

# Preview changes before applying
ddutil update --regions us-west-2,eu-west-1 --dry-run

# Update with verbose output
ddutil --verbose update --services AWS/Lambda

Delete Command

Remove DataDog integration:

# With confirmation prompt (interactive)
ddutil delete

# Specify account ID
ddutil delete --account-id 123456789012

# Skip confirmation prompt (for automation)
ddutil delete --confirm

# Delete with specific role name
ddutil delete --role-name custom-datadog-role --confirm

# Use specific AWS profile
ddutil delete --profile aws-prod --confirm

Configuration

Configuration Priority

Configuration values are resolved in the following order (highest to lowest priority):

1. CLI arguments - Values passed directly via command-line flags
2. Environment variables - Values from .env file or system environment

Environment Variables

Create a .env file in your project directory:

# Copy the example file
cp .env.example .env

# Edit with your values
nano .env  # or your favorite editor

Required Environment Variables

# DataDog API Credentials
DD_API_KEY=your_datadog_api_key_here    # DataDog API key (required)
DD_APP_KEY=your_datadog_app_key_here    # DataDog application key (required)

# AWS Configuration
AWS_ACCOUNT_ID=123456789012             # Your AWS account ID (required)

Optional Environment Variables

# Application Settings
LOG_LEVEL=INFO                         # Logging level (DEBUG, INFO, WARNING, ERROR)

# DataDog Configuration
DD_ACCOUNT_ID=                         # DataDog account ID (obtained from DataDog)
DD_SITE=datadoghq.com                  # DataDog site (datadoghq.com, datadoghq.eu, etc.)
DD_PARTITION=aws                       # AWS partition (aws, aws-cn, aws-us-gov)
DATADOG_VERIFY_SSL=false               # SSL verification for DataDog API

# AWS Configuration
AWS_PROFILE=default                    # AWS CLI profile name

# IAM Configuration
DD_IAM_ROLE_NAME=datadog-integration-role  # IAM role name
DD_MANAGED_POLICIES=                   # Comma-separated managed policy ARNs
                                       # Default: ReadOnlyAccess, SecurityAudit
DD_POLICY_ACTIONS=                     # Comma-separated additional IAM actions
                                       # Default: 32 standard actions (see .env.example)

# Monitoring Configuration
DD_REGIONS=                            # Comma-separated AWS regions (empty = all)
DD_SERVICES=                           # Comma-separated AWS services (empty = default)
DD_TRACES=                             # Comma-separated services for X-Ray tracing

# Metric Settings
DD_METRIC_AUTOMUTE=true                # Auto-mute monitors (true/false)
DD_METRIC_COLLECT_CLOUDWATCH=true      # Collect CloudWatch alarms (true/false)
DD_METRIC_COLLECT_CUSTOM=false         # Collect custom metrics (true/false)
DD_METRIC_COLLECT_METRICS=true         # Enable metric collection (true/false)
DD_METRIC_ENABLE=true                  # Enable metrics globally (true/false)

# Resource Settings
DD_RESOURCE_COLLECT_CSPM=false         # Cloud Security Posture Management (true/false)
DD_RESOURCE_COLLECT_EXTENDED=true      # Extended resource collection (true/false)

Examples

Basic Setup with .env File

Create .env:

DD_API_KEY=abc123def456...
DD_APP_KEY=xyz789ghi012...
AWS_ACCOUNT_ID=123456789012
AWS_PROFILE=production

Run setup:

# Preview changes
ddutil setup --dry-run

# Apply configuration
ddutil setup

# Check status after setup
ddutil status

Multi-Region Setup

# Monitor specific regions only
ddutil setup --regions us-east-1,us-west-2,eu-west-1

# Or set in .env
DD_REGIONS=us-east-1,us-west-2,eu-west-1
ddutil setup

Specific Services Only

# Monitor specific AWS services
ddutil setup --services AWS/Lambda,AWS/EC2,AWS/RDS

# With X-Ray tracing for Lambda
ddutil setup --services AWS/Lambda,AWS/EC2 --traces AWS/Lambda

Metric and Resource Configuration

# Enable Cloud Security Posture Management
ddutil setup --resource-collect-cspm true

# Enable custom metrics collection
ddutil setup --metric-collect-custom true

# Configure multiple metric settings
ddutil setup \
  --metric-automute true \
  --metric-collect-cloudwatch true \
  --metric-collect-custom false \
  --metric-enable true

Override with CLI Arguments

# Override environment variables for one-off operations
ddutil setup --account-id 999888777666 --profile dev-account

# Use different AWS partition
ddutil setup --partition aws-us-gov --profile govcloud

Custom IAM Configuration

# Custom role name and policies
ddutil setup \
  --role-name custom-datadog-role \
  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess \
  --policy-actions logs:PutSubscriptionFilter,s3:GetBucketNotification

# Update existing role policies
ddutil update \
  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess,arn:aws:iam::aws:policy/SecurityAudit

Validation and Status Checking

# Check if configuration matches .env settings
ddutil status

# Get detailed JSON output for monitoring
ddutil status --output json > status.json

# Validate specific account configuration
ddutil --verbose status --account-id 123456789012

Development

Project Structure

datadog-utility/
├── src/
│   └── ddutil/          # Main package
│       ├── __init__.py  # Package initialization
│       ├── cli.py       # CLI command definitions
│       └── common/      # Shared utilities
│           ├── aws/     # AWS-related modules
│           │   ├── auth.py      # AWS authentication
│           │   └── iam.py       # IAM role management
│           ├── datadog/         # DataDog-related modules
│           │   └── aws.py       # DataDog API interactions
│           ├── logs.py          # Logging configuration
│           └── utils.py         # Utility functions
├── pyproject.toml       # Package configuration
├── .env.example         # Example environment variables
├── .python-version      # Python version for UV
├── requirements.txt     # Python dependencies
└── responses/           # Sample API responses

Troubleshooting

Common Issues

Missing required environment variables:

# Check if all required variables are set with dry-run
ddutil setup --dry-run

# Set missing variables in .env file or export them
export AWS_ACCOUNT_ID=123456789012
export DD_API_KEY=your_api_key
export DD_APP_KEY=your_app_key

AWS authentication errors:

# Ensure your AWS profile is configured
aws configure --profile your-profile-name

# Or specify profile explicitly
ddutil setup --profile your-profile-name

# Or use environment variables
export AWS_PROFILE=your-profile-name

# Test AWS connectivity
aws sts get-caller-identity --profile your-profile-name

DataDog API errors:

# Verify your API keys are set
echo $DD_API_KEY
echo $DD_APP_KEY

# Make sure they're in your .env file
cat .env | grep DD_

# Test DataDog API connectivity
ddutil --verbose status

Configuration mismatch errors:

# Use status command to see what doesn't match
ddutil status

# Status will show:
# - IAM role and policy status
# - DataDog account configuration
# - Mismatches between actual and expected settings
# - Specific issues with regions, services, metrics, resources

# Fix mismatches by updating
ddutil update --regions us-east-1,us-west-2
ddutil update --metric-collect-cloudwatch true

Debug with verbose and dry-run:

# Always test with dry-run first
ddutil setup --dry-run

# Use verbose for detailed debugging
ddutil --verbose setup --dry-run

# Check logs for detailed error messages
tail -f ~/.ddutil/logs/ddutil.log  # if logging to file

Roadmap

See ROADMAP.md for planned features and improvements.

License

MIT License - See LICENSE file for details.