Skip to main content

Terraform wrapper to manage state across multiple cloud providers

Project description

Terraform Remote State Manager(tfremote)

tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a defined structure for all cloud providers by removing the overheard of configuring and managing the path in storage buckets.

It works with:

:point_right: Google Storage Bucket

:point_right: AWS S3

:point_right: Azure Storage

❗️ Note Best practice is to make sure buckets are versioned.

Install package

pip install tfremote --upgrade

Environment setup

  • Install Python 3.6+

  • Using virtualenv is strongly recommended:

python3 -m venv <venv name>

Default log level is WARNING, to change:

export TF_LOG_LEVEL to any of these: 'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'

❗️ Important - Two variables are required for using tf package (used set creat path in remote storage):

  • teamid
  • prjid

Required variables can be defined using:

  • As inline variables e.g.: -var='teamid=demo-team' -var='prjid=demo-project'
  • Inside .tfvars file e.g.: -var-file=<tfvars file location>

Two optional variables:

workspace and state_key can be defined using:

  • -w=<workspace_name>. If no workspace is provided default workspace is used.

  • s=<state_key name>. If no key is provided terraform is used.

Path created in S3 backend: /<teamid>/<prjid>/<workspace>/<state-key>.tfstate

For more information refer to Terraform documentation

Setup environment variables

Workspace list file location TF_WORKSPACE_FILE_LOCATION

export TF_WORKSPACE_FILE_LOCATION=<workspace yml file location>

Reference file: link

AWS

❗️ Important - s3 bucket for remote state should reside in us-west-2

Set these env variables:

export TF_AWS_BUCKET=<your_remote_state_bucket_name>
export TF_AWS_BUCKET_REGION=us-west-2

One of below environment variable is required:

export TF_AWS_PROFILE=<aws profile to use>

or

export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>

Azure

To create storage for remote state there is handy script.

Run scripts/remote_state.sh (fill in the required information)

Set below env variables:

export TF_AZURE_STORAGE_ACCOUNT=<remote state storage account name>
export TF_AZURE_CONTAINER=<remote state container>
export ARM_ACCESS_KEY=<storage account access key>

GCP(gcloud)

https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform

Set below env variables:

export TF_GCLOUD_BUCKET=<remote state storage bucket name>
export TF_GCLOUD_CREDENTIALS=json credentials file path>

Usage

For GCP(gcloud):

tf plan -c=gcloud -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Google Storage Bucket:

alt text

For AWS:

tf plan -c=aws -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in AWS S3:

alt text

If you need to specify state_key in S3, specify -s=tryme-key

For Azure:

tf plan -c=azure -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Azure Storage:

alt text

For more available options:

tf --help
usage: tf [-h] [-var] [-var-file] [-c] [-w] [-wp] [-s] [-no-color] [-json] [-out] [-f] [-nf] [-v]

Terraform remote state wrapper package
--------------------------------------
Usage: Set below env variables to begin (more information: https://github.com/tomarv2/tfremote):
TF_WORKSPACE_FILE_LOCATION
aws: TF_AWS_BUCKET, TF_AWS_BUCKET_REGION=us-west-2, TF_AWS_PROFILE or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
azure: TF_AZURE_STORAGE_ACCOUNT, TF_AZURE_CONTAINER, ARM_ACCESS_KEY
gcloud: TF_GCLOUD_BUCKET, TF_GCLOUD_CREDENTIALS

optional arguments:
  -h, --help  show this help message and exit
  -var        Set Terraform configuration variable. This flag can be set multiple times
  -var-file   Set Terraform configuration variables from a file. This flag can be set multiple times
  -c          Specify cloud provider (default: 'aws'). Supported values: gcloud, aws, or azure
  -w          Specify existing workspace name(default: 'default')
  -wp         Overwrite workspace directory path structure
  -s          File name in remote state (default: 'terraform.tfstate')
  -no-color   Disables terminal formatting sequences in the output
  -json       Enables the machine readable JSON UI output
  -out        Writes the generated plan to the given filename in an opaque file format
  -f          Enable FIPS endpoints (default: True)
  -nf         Disable FIPS endpoints
  -v          show program's version number and exit

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for tfremote, version 0.0.36
Filename, size File type Python version Upload date Hashes
Filename, size tfremote-0.0.36-py3-none-any.whl (18.8 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size tfremote-0.0.36.tar.gz (18.5 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page