cfddb: CloudFormation Parser for DynamoDB Local

cfddb is a specialized Infrastructure-as-Code (IaC) deployment engine designed to bridge the gap between AWS CloudFormation and DynamoDB Local.

While AWS CloudFormation works perfectly for production, dynamodb-local (the Docker image) does not natively understand CFN templates. This tool parses your YAML, generates a deployment plan, and applies it to your local container using boto3, mimicking the behavior of a real stack deployment.

🚀 Key Features

CloudFormation Parsing: Native support for CFN Intrinsics (!Ref, !Sub, !GetAtt, !If, !Select, !Split, etc.).

Idempotent Deployment: Runs a "diff" against the local database. It only creates missing tables, adds missing GSIs, or updates TTLs.

Local Limitation Workarounds: The latest DynamoDB Local image available on Docker Hub does not yet support Multi-Attribute Global Secondary Indexes (GSIs). Because of this limitation, attempting to create tables with such GSIs locally can result in “Key Schema too big” errors. To handle this, the system automatically detects when Multi-Attribute GSIs are present in the template and applies a workaround. Currently, these GSIs are excluded during local table creation to avoid the error (a more robust solution is being explored). This behavior is limited to DynamoDB Local only. Once AWS updates the DynamoDB Local image to support Multi-Attribute GSIs, this workaround will no longer be needed and the full template will work automatically without any changes.

Drift Detection & Pruning: Detects ghost GSIs that exist locally but not in your template and optionally deletes them (--prune).

Safety & Locking: Uses file-based locking to prevent concurrent executions and includes interactive confirmation prompts.

🛠️ Prerequisites

Python 3.10+
Docker (to run amazon/dynamodb-local)

📦 Installation

1. Install via pip

You can install cfddb directly from PyPI:

pip install cfddb

2. Start DynamoDB Local

You need a running DynamoDB Local instance to execute deployment plans.

docker run -d -p 8000:8000 amazon/dynamodb-local

💻 Usage

Once installed, the cfddb command is available in your terminal.

Basic Run

Deploy a template to your local environment:

cfddb --template path/to/dynamodb.yaml

Command Line Arguments

Examples

Dry Run (See what will happen):

cfddb --template dynamodb.yaml --parameters AppEnv=local --plan

Clean up old indexes (Pruning):

cfddb --template dynamodb.yaml --prune

⚙️ How It Works

1. The Parser

The tool uses a custom YAML loader to process CloudFormation shorthand syntax.

Pseudo Parameters: Injects AWS::AccountId, AWS::Region, etc.
Conditions: Evaluates Fn::If, Fn::Or, Fn::Equals to determine if a resource should be created (e.g., enabling PITR only in prod).
Intrinsics:
!Ref -> Resolves parameters.
!Sub -> Interpolates strings. for example: (table-${Env}-users).
!GetAtt -> Generates mock ARNs for local consistency.

2. The Planner

It queries the local database and compares it against your YAML spec.
Tables: If missing, plans create.
GSIs: If missing, plans add_gsi. If extra, plans warn (or delete if --prune is set).
TTL: Checks Time-to-Live status and plans updates if drift is detected.

3. The Executor

This is where the magic happens. It applies the plan using boto3.

⚠️ Handling Multi-Attribute GSIs.
DynamoDB Local has a legacy limitation where the Key Schema cannot exceed 2 attributes (Hash + Range). The latest DynamoDB supports up to 8 attributes for GSIs only in production environments right now.
Once AWS updates the DynamoDB Local image to support Multi-Attribute GSIs, this workaround will no longer be needed and the full template will work automatically without any changes.
If your template uses modern Multi-Attribute keys, DynamoDB Local will crash with ValidationException: Key Schema too big, while cfddb handles this automatically by following steps:
It attempts to create the table.
If it catches the "Key Schema too big" error.
It identifies GSIs with > 2 keys.
Filters them out of the request, cleans up unused AttributeDefinitions.
Retries the creation.

Result: Your table is created successfully locally (minus the unsupported index), allowing you to continue developing without changing your production YAML.

📝 License

Apache License 2.0