terraback 0.5.0

Terraback: Universal infrastructure scanning and Terraform generation tool

🚀 Terraback

Multi-Cloud Infrastructure as Code Tool

Terraback is an advanced CLI tool that reverse-engineers existing cloud infrastructure into clean, production-ready Terraform code with intelligent dependency discovery.

Transform legacy cloud environments into infrastructure-as-code in minutes, not months.

---

✨ Why Terraback?

• 🎯 90% Faster: Reduce infrastructure documentation time from weeks to hours
• 🧠 Intelligent Discovery: Automatic dependency mapping across 50+ cloud services
• 🏢 Enterprise-Ready: Production-ready templates with lifecycle management
• Multi-Cloud: Full support for AWS, Azure, and GCP
• 🔒 Security-First: Read-only permissions, no credential storage, enterprise compliance

---

🌟 Features

📊 Comprehensive Cloud Coverage

Core Infrastructure (Community Edition - Free):

• AWS: EC2 Instances, VPCs, Subnets, Security Groups, S3 Buckets, IAM Roles
• Azure: Virtual Machines, Virtual Networks, Subnets, Network Security Groups, Storage Accounts
• GCP: Compute Instances, Networks, Subnets, Storage Buckets
• Basic Commands: list, import, scan
• ✨ Unlimited core resources
• Basic dependency mapping

Advanced Services (Professional & Enterprise):

AWS Services:

• Container Platform: ECS Clusters, Services, Task Definitions, ECR Repositories
• Advanced Networking: CloudFront CDN, Route 53 DNS, NAT/Internet Gateways, VPC Endpoints
• Database & Caching: RDS instances, ElastiCache Redis/Memcached clusters
• Monitoring & Security: CloudWatch, Auto Scaling, ACM Certificates
• Serverless & APIs: Lambda Functions, API Gateway, SQS, SNS
• Storage: EFS, EBS Volumes/Snapshots, S3 advanced features
• Load Balancing: ALB/NLB with advanced features, listener rules, SSL policies

Azure Services:

• Compute: Virtual Machines with OS detection, Managed Disks, SSH Keys
• Networking: Virtual Networks, Subnets with delegations, NSGs with rules
• Storage: Storage Accounts with blob properties, static websites, encryption
• Load Balancing: Application Gateway, Load Balancers
• Database: Azure SQL, Cosmos DB
• Container: AKS, Container Instances

GCP Services:

• Compute: VM Instances, Persistent Disks, Machine Images
• Networking: VPC Networks, Subnets, Firewall Rules
• Storage: Cloud Storage, Persistent Disks
• Container: GKE Clusters, Node Pools

🔗 Intelligent Dependency Discovery

The --with-deps flag automatically discovers and maps complete infrastructure stacks:

# AWS Example
terraback ec2 scan --with-deps
# Automatically finds and scans:
# ├── VPC and networking (subnets, security groups)
# ├── Load balancers (ALB, target groups, listeners)
# ├── Storage (EBS volumes, snapshots)
# ├── Security (IAM roles, certificates)
# ├── Monitoring (CloudWatch logs, alarms)
# └── All interconnected dependencies

# Azure Example
terraback vm scan --with-deps
# Automatically finds and scans:
# ├── Virtual Networks and Subnets
# ├── Network Security Groups with rules
# ├── Managed Disks and Snapshots
# ├── Network Interfaces and Public IPs
# ├── Storage Accounts
# └── All interconnected dependencies

🚀 Advanced Features

• Performance Optimization: API response caching, parallel scanning, smart dependency resolution
• Multi-Account/Subscription: Scan across multiple AWS accounts or Azure subscriptions
• Module Generation: Create reusable Terraform modules automatically
• State Management: Import existing resources directly into Terraform state
• Compliance Ready: Generate code following HashiCorp and cloud provider best practices
• Name Deduplication: Automatically append numeric suffixes to resource names when duplicates are detected
• SSH Key Sanitization: Generated public keys are collapsed to a single line automatically

---

📦 Installation

Prerequisites

• Python 3.8+ (for pip installation)
• Terraform 1.0+ (for import functionality)
• Cloud CLI tools:
  • AWS: aws CLI configured
  • Azure: az CLI configured and logged in
  • GCP: gcloud CLI configured
• (Optional) xxhash for faster cache hashing

Install via pip (Recommended)

pip install terraback

Install via Binary

Linux/macOS:

# Linux
curl -L https://dist.terraback.dev.io/releases/latest/terraback-linux -o terraback
chmod +x terraback
sudo mv terraback /usr/local/bin/

# macOS
curl -L https://dist.terraback.dev.io/releases/latest/terraback-macos -o terraback
chmod +x terraback
sudo mv terraback /usr/local/bin/

Windows: Download from releases

---

🚀 Quick Start

AWS Scanning

# Configure AWS credentials (if not already done)
aws configure

# Scan EC2 instances
terraback ec2 scan

# Scan VPC with all dependencies
terraback vpc scan --with-deps

# Scan specific region
terraback ec2 scan --region eu-west-1

# Scan with specific profile
terraback s3 scan --profile production

Azure Scanning

# Login to Azure (if not already done)
az login

# Scan Virtual Machines
terraback vm scan

# Scan entire resource group
terraback vm scan --resource-group production-rg

# Scan specific subscription
terraback vnet scan --subscription-id YOUR-SUBSCRIPTION-ID

# Scan with location filter
terraback storage scan --location westeurope

Multi-Cloud Commands

# Scan all resources (cloud-specific)
terraback scan-all aws --region us-east-1
terraback scan-all azure --resource-group my-rg
terraback scan-all gcp --region us-central1 --cache-ttl 30

# Check authentication status
terraback auth-check

# Use caching for large infrastructures
terraback scan-recursive ec2 --use-cache --cache-ttl 120
terraback scan-recursive vm --use-cache --parallel-workers 10

After scanning completes, Terraback automatically runs terraform fmt, terraform init and terraform validate on the generated code. Use --skip-check with any scan-all command to bypass this validation step. If you skip the checks, run terraback aws scan-all --check (or python -m terraback.utils.template_syntax_fixer <output_dir>) before invoking terraform init manually.

---

📋 Supported Resources

AWS Resources (Full Support)

Service	Resources	Status
EC2	Instances, Volumes, Snapshots, AMIs, Key Pairs, Launch Templates, Network Interfaces	✅ Full Support
VPC	VPCs, Subnets, Security Groups, Internet/NAT Gateways, Route Tables, VPC Endpoints	✅ Full Support
IAM	Roles, Policies, Instance Profiles	✅ Full Support
S3	Buckets, Versioning, Lifecycle, ACLs, Policies	✅ Full Support
RDS	Instances, Subnet Groups, Parameter Groups	✅ Full Support
Load Balancing	ALB, NLB, CLB, Target Groups, Listeners, SSL Policies	✅ Full Support
Lambda	Functions, Layers, Permissions	✅ Full Support
Route 53	Hosted Zones, Records	✅ Full Support
CloudWatch	Log Groups, Alarms, Dashboards	✅ Full Support
Auto Scaling	Groups, Launch Templates, Policies	✅ Full Support
ECS/ECR	Clusters, Services, Task Definitions, Repositories	✅ Full Support
CloudFront	Distributions, Origin Access Controls, Cache Policies	✅ Full Support
ElastiCache	Redis/Memcached Clusters, Parameter Groups	✅ Full Support
ACM	Certificates	✅ Full Support
EFS	File Systems, Mount Targets, Access Points	✅ Full Support
API Gateway	REST APIs, Resources, Methods, Deployments	✅ Full Support
SQS/SNS	Queues, Topics, Subscriptions	✅ Full Support
Secrets Manager	Secrets, Versions	✅ Full Support
Systems Manager	Parameters, Documents	✅ Full Support

Azure Resources

Service	Resources	Status
Compute	Virtual Machines, Managed Disks, Availability Sets, SSH Keys	✅ Full Support
Networking	Virtual Networks, Subnets, Network Security Groups, Network Interfaces	✅ Full Support
Storage	Storage Accounts, Blob Storage, File Shares	✅ Full Support
Load Balancing	Load Balancers, Application Gateways	✅ Full Support
Database	SQL Database, Cosmos DB	✅ Full Support
Container	AKS, Container Instances	✅ Full Support
Identity	Managed Identities, Service Principals	✅ Full Support

GCP Resources

Service	Resources	Status
Compute Engine	Instances, Disks, Images	✅ Full Support
VPC	Networks, Subnets, Firewalls	✅ Full Support
Cloud Storage	Buckets, Objects	✅ Full Support
GKE	Clusters, Node Pools	✅ Full Support

---

🎯 Advanced Usage

Dependency Scanning

# AWS: Scan EC2 with all dependencies
terraback ec2 scan --with-deps --output-dir ./infrastructure

# Azure: Scan VM with all dependencies  
terraback vm scan --with-deps --output-dir ./infrastructure

# Result: Complete infrastructure graph
# ├── compute/
# │   ├── instances.tf
# │   └── launch_templates.tf
# ├── networking/
# │   ├── vpc.tf
# │   ├── subnets.tf
# │   └── security_groups.tf
# └── storage/
#     ├── volumes.tf
#     └── snapshots.tf

Enterprise Module Generation

Transform generated Terraform into production-ready enterprise modules with standardized structure:

# Generate enterprise modules (requires Enterprise license)
terraback scan all aws --enterprise-modules

# Result: Organized module structure
# modules/
# ├── alb/
# │   ├── lb.tf
# │   ├── listeners.tf
# │   ├── global-accelerator.tf
# │   ├── variables.tf
# │   ├── outputs.tf
# │   └── locals.tf
# ├── ecs/
# │   └── service/
# │       ├── ecs-service.tf
# │       ├── ecs-task-definition.tf
# │       ├── autoscaling.tf
# │       ├── variables.tf
# │       ├── outputs.tf
# │       └── locals.tf
# ├── rds-aurora-postgresql/
# │   ├── cluster.tf
# │   ├── instances.tf
# │   ├── parameter-groups.tf
# │   ├── variables.tf
# │   ├── outputs.tf
# │   └── locals.tf
# └── s3-bucket/
#     ├── bucket.tf
#     ├── policy.tf
#     ├── versioning.tf
#     ├── lifecycle-configuration.tf
#     ├── variables.tf
#     ├── outputs.tf
#     └── locals.tf

# Works with all providers
terraback scan all azure --enterprise-modules
terraback scan all gcp --enterprise-modules

Features:

• Automatically groups related resources into logical modules
• Creates standard module files (variables.tf, outputs.tf, locals.tf)
• Follows Terraform best practices and naming conventions
• Supports 150+ AWS, Azure, and GCP resource types
• Maintains proper resource dependencies
• Production-ready module structure out of the box

Supported Module Types:

• AWS: ALB, ASG, ECS, RDS, Lambda, S3, CloudFront, OpenSearch, and 40+ more
• Azure: VMs, AKS, App Services, Databases, Networking, Storage, and 30+ more
• GCP: Compute, GKE, Cloud Run, Cloud SQL, Networking, Storage, and 30+ more

Caching & Performance

# Enable caching for large infrastructures
terraback scan-all aws --cache-ttl 45
terraback scan-recursive ec2 --use-cache --cache-ttl 60

# View cache statistics
terraback cache stats

# Clear cache
terraback cache clear

# ``--cache-ttl`` sets how long scan results stay in the on-disk cache
# (in minutes) before expiring. This can dramatically speed up repeated
# ``scan-all`` operations on very large environments.
# For even faster cache hashing install ``xxhash`` (optional). Without it
# Terraback gracefully falls back to ``blake2s``.

# Parallel scanning
terraback scan-recursive vm --parallel-workers 10

Import to Terraform

# List discovered resources
terraback list all

# Import specific resource
terraback ec2 import i-1234567890abcdef0

# Import Azure VM
terraback vm import /subscriptions/xxx/resourceGroups/my-rg/providers/Microsoft.Compute/virtualMachines/my-vm
# Bulk import all resources asynchronously with a progress bar
terraback import-all --async --progress
# Verify account and region before importing
terraback import-all --verify
# Set the Terraform lock timeout to 600 seconds
terraback import-all --lock-timeout 600
# Import using Terraform 1.5+ import blocks
terraback import-all --workflow import-blocks
# Automatically select the optimal workflow
terraback import-all --workflow auto --parallel=8
# Import via Terraform workspaces with 8 parallel workers
terraback import-all --workflow workspaces --parallel=8
# Import using enhanced import blocks in batches of 25
terraback import-all --workflow import-blocks-enhanced --batch-size=25
# Import using parallel workspaces with 10 workers
terraback import-all --workflow parallel-workspaces --parallel=10

Template Formatting Check

After scanning or when editing .tf files manually, run one of the following commands before running terraform init:

terraback aws scan-all --check
# or
python -m terraback.utils.template_syntax_fixer <output_dir>

Before executing these validation commands, ensure the base Python dependencies are installed. The template checker relies on packages from requirements.txt, including Jinja2. Install them with:

pip install -r requirements.txt

This prevents "Missing newline after argument" errors.

Example command sequence:

terraback scan-all aws --skip-check --output-dir ./generated
terraback aws scan-all --check
cd generated
terraform init

The --workflow option controls how state is imported:

• auto – automatically selects the fastest strategy based on your Terraform version.

• import-blocks – for Terraform 1.5+ with a local backend, Terraback generates import blocks that Terraform applies in a single workspace.

• import-blocks-enhanced – groups resources into batches and applies each with Terraform. Use --batch-size and --parallel to control batch size and parallelism.

• workspaces – for Terraform <1.5 or when using a remote state backend (S3, GCS, etc.), Terraback imports each resource in its own workspace.

• parallel-workspaces – runs workspace imports concurrently in temporary directories and merges their state automatically. Controlled by --parallel.

If --workflow is not specified, Terraback automatically chooses the best workflow based on your Terraform version.

Fast & Parallel Workspace Workflows

Terraback provides two high-speed import strategies. The fast workflow is the default. Resources are processed in batches (controlled by --batch-size) and the tool automatically chooses between Terraform import blocks (for Terraform 1.5+) and the temporary workspace approach. When workspaces are used, each batch is imported concurrently using the number of workers set by --parallel. The resulting state files are merged together with merge_states and all workspace_* folders along with plan/debug files are removed using clean_import_artifacts.

The parallel-workspaces workflow exposes the workspace method directly. Each resource is imported in its own temporary directory. Once all imports finish, the states are merged and the temporary directories are deleted.

# Fast workflow with batching and 8 parallel workers
terraback import-all --workflow fast --batch-size=50 --parallel=8

# Parallel workspaces with 10 workers
terraback import-all --workflow parallel-workspaces --parallel=10

---

⚖️ License & Pricing

Terraback uses a multi-tier licensing model:

Community Edition (Free)

• ✅ Core resources for AWS, Azure, GCP
• ✅ EC2, VPC, S3 (AWS)
• ✅ VMs, VNets, Storage (Azure)
• ✅ Compute, Networks, Storage (GCP)
• ✅ ✨ Unlimited core resources
• ✅ Basic dependency mapping
• ✅ Community support via GitHub
• ❌ Advanced services (RDS, Lambda, etc.)
• ❌ Multi-account support

Professional License ($499 one-time, lifetime access)

• All 50+ cloud services
• Unlimited resources and accounts
• Advanced dependency mapping
• Multi-account/subscription scanning
• RDS, Lambda, EKS, and more
• Module generation and best practices
• State file management
• Priority email support
• API access for automation
• Lifetime updates included

Enterprise Edition (Coming Soon)

• Everything in Professional
• ✅ Enterprise Module Generation - Automatic organization into production-ready modules
• ✅ Annual/multi-year licensing
• ✅ SSO integration (SAML, OIDC)
• ✅ Custom resource scanners
• ✅ On-premise deployment options
• ✅ SLA with guaranteed uptime
• ✅ Dedicated training & onboarding
• ✅ Dedicated customer success manager
• ✅ Volume licensing & team management
• ✅ Compliance reporting

---

📚 Documentation

• Full Documentation
• AWS Scanning Guide
• Azure Scanning Guide
• GCP Scanning Guide
• API Reference
• Examples & Tutorials
• AWS Secrets Manager templates
• Scanning Secrets Manager resources automatically creates a variables.tf file with a secret_value variable stub.
• The stub provides a default placeholder so Terraform doesn't prompt during imports.
• Be sure to update this value in a .tfvars file before applying.

---

Logging

Terraback respects two environment variables for logging:

• TERRABACK_LOG_LEVEL controls the log verbosity (e.g. DEBUG or INFO).
• TERRABACK_LOG_FILE can specify a path to write logs to a file.

Set these variables before running any command to adjust log output.

---

🐛 Troubleshooting

AWS Issues

# Check AWS credentials
aws sts get-caller-identity

# Use specific profile
export AWS_PROFILE=production

# Debug mode
terraback ec2 scan --debug

Azure Issues

# Check Azure login
az account show

# Set subscription
az account set --subscription "My Subscription"

# List available subscriptions
az account list --output table

Common Issues

1. Permission Denied: Ensure your cloud credentials have read access to resources
2. Rate Limiting: Use --use-cache flag for large infrastructures
3. Module Not Found: Install with pip install -e . for development

---

🚀 Roadmap

• AWS support (50+ services)
• Azure support (Core services)
• GCP support (Core services)
• Terraform module generation
• Web UI dashboard
• CI/CD integrations

🏗️ Template System

Terraback generates Terraform files from a collection of Jinja2 templates. All provider templates inherit from templates/common/base_resource.tf.j2, giving them a shared set of helper macros and a common block to override. To create a new template, extend the base file and implement the resources block:

{% extends "common/base_resource.tf.j2" %}
{% from "common/base_resource.tf.j2" import render_common_tags %}

{% block resources %}
resource "aws_my_service" "{{ item.name | tf_resource_name }}" {
  name = "{{ item.name }}"
  {{ render_common_tags(item, indent=2) }}
}
{% endblock %}

Complex-data macros

templates/common/macros.j2 exposes utilities for rendering lists and nested structures. Examples include:

• render_block_list – iterate over complex objects and emit nested blocks
• render_json_list – output JSON formatted lists
• render_depends_on – add a depends_on array when dependencies exist

These macros keep templates concise while handling deeply nested data.

Automatic dependency generation

During scanning Terraback records relationships between resources. When a template uses render_depends_on, the discovered dependencies are inserted automatically. For example:

resource "aws_launch_template" "example" {
  ...
{{ macros.render_depends_on(template.depends_on, indent=2) }}
}

If an IAM role and security group were detected as dependencies, the rendered Terraform code becomes:

depends_on = [
  aws_iam_role.example_role,
  aws_security_group.example_sg,
]

Terraback calculates these relationships automatically, but you can still supply additional items in the depends_on list to enforce custom ordering.

🧪 Development & Testing

Install the additional packages used by the test suite:

pip install -r requirements-dev.txt

Then run all tests from the project root:

pytest

---

📝 Line Endings

This project enforces LF line endings for all text files. Git uses .gitattributes to normalize line endings and a pre-commit hook (end-of-file-fixer) ensures files end with a single newline.

📞 Support

• Community: GitHub Discussions
• Issues & Feature Requests: GitHub Issues
• Professional Support: [e-mail]
• Enterprise Sales: [e-mail]

---

🙏 Acknowledgments

Built with ❤️ by DevOps engineers who understand the pain of manual cloud documentation.

---

Copyright © 2025 Terraback