Skip to main content

Terraform wrapper to manage state across multiple cloud providers

Project description

Terraform Remote State Manager(tfremote)

tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a defined structure for all cloud providers by removing the overheard of configuring and managing the path in storage buckets.

It works with:

:point_right: Google Storage Bucket

:point_right: AWS S3

:point_right: Azure Storage

❗️ Note Best practice is to make sure buckets are versioned.

Install package

pip install tfremote --upgrade

Environment setup

  • Install Python 3.6+

  • Using virtualenv is strongly recommended:

python3 -m venv <venv name>

Default log level is WARNING, to change:

export TF_LOG_LEVEL to any of these: 'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'

❗️ Important - Two variables are required for using tf package (used set creat path in remote storage):

  • teamid
  • prjid

Required variables can be defined using:

  • As inline variables e.g.: -var='teamid=demo-team' -var='prjid=demo-project'
  • Inside .tfvars file e.g.: -var-file=<tfvars file location>

Two optional variables:

workspace and state_key can be defined using:

  • -w=<workspace_name>. If no workspace is provided default workspace is used.

  • s=<state_key name>. If no key is provided terraform is used.

Path created in S3 backend: /<teamid>/<prjid>/<workspace>/<state-key>.tfstate

For more information refer to Terraform documentation

Setup environment variables

Workspace list file location TF_WORKSPACE_FILE_LOCATION

export TF_WORKSPACE_FILE_LOCATION=<workspace yml file location>

Reference file: link

AWS

❗️ Important - s3 bucket for remote state should reside in us-west-2

Set these env variables:

export TF_AWS_BUCKET=<your_remote_state_bucket_name>
export TF_AWS_BUCKET_REGION=us-west-2

One of below environment variable is required:

export TF_AWS_PROFILE=<aws profile to use>

or

export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>

Azure

To create storage for remote state there is handy script.

Run scripts/remote_state.sh (fill in the required information)

Set below env variables:

export TF_AZURE_STORAGE_ACCOUNT=<remote state storage account name>
export TF_AZURE_CONTAINER=<remote state container>
export ARM_ACCESS_KEY=<storage account access key>

GCP(gcloud)

https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform

Set below env variables:

export TF_GCLOUD_BUCKET=<remote state storage bucket name>
export TF_GCLOUD_CREDENTIALS=json credentials file path>

Usage

For GCP(gcloud):

tf plan -c=gcloud -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Google Storage Bucket:

alt text

For AWS:

tf plan -c=aws -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in AWS S3:

alt text

If you need to specify state_key in S3, specify -s=tryme-key

For Azure:

tf plan -c=azure -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Azure Storage:

alt text

For more available options:

tf --help
usage: tf [-h] [-var] [-var-file] [-c] [-w] [-wp] [-s] [-no-color] [-json] [-out] [-f] [-nf] [-v]

Terraform remote state wrapper package
--------------------------------------
Usage: Set below env variables to begin (more information: https://github.com/tomarv2/tfremote):
TF_WORKSPACE_FILE_LOCATION
aws: TF_AWS_BUCKET, TF_AWS_BUCKET_REGION=us-west-2, TF_AWS_PROFILE or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
azure: TF_AZURE_STORAGE_ACCOUNT, TF_AZURE_CONTAINER, ARM_ACCESS_KEY
gcloud: TF_GCLOUD_BUCKET, TF_GCLOUD_CREDENTIALS

optional arguments:
  -h, --help  show this help message and exit
  -var        Set Terraform configuration variable. This flag can be set multiple times
  -var-file   Set Terraform configuration variables from a file. This flag can be set multiple times
  -c          Specify cloud provider (default: 'aws'). Supported values: gcloud, aws, or azure
  -w          Specify existing workspace name(default: 'default')
  -wp         Overwrite workspace directory path structure
  -s          File name in remote state (default: 'terraform.tfstate')
  -no-color   Disables terminal formatting sequences in the output
  -json       Enables the machine readable JSON UI output
  -out        Writes the generated plan to the given filename in an opaque file format
  -f          Enable FIPS endpoints (default: True)
  -nf         Disable FIPS endpoints
  -v          show program's version number and exit

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tfremote-0.0.36.tar.gz (18.5 kB view details)

Uploaded Source

Built Distribution

tfremote-0.0.36-py3-none-any.whl (18.8 kB view details)

Uploaded Python 3

File details

Details for the file tfremote-0.0.36.tar.gz.

File metadata

  • Download URL: tfremote-0.0.36.tar.gz
  • Upload date:
  • Size: 18.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.9.5

File hashes

Hashes for tfremote-0.0.36.tar.gz
Algorithm Hash digest
SHA256 2d4711786065e48422878217a6089f966f437b9015ebf5a70b27473d6cedb8e6
MD5 af10b309bd009419b476714766592290
BLAKE2b-256 69ee916a3cf38bfe45efc770857679663c03013cc0a2065cf43f9d6b3fa12572

See more details on using hashes here.

File details

Details for the file tfremote-0.0.36-py3-none-any.whl.

File metadata

  • Download URL: tfremote-0.0.36-py3-none-any.whl
  • Upload date:
  • Size: 18.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.9.5

File hashes

Hashes for tfremote-0.0.36-py3-none-any.whl
Algorithm Hash digest
SHA256 ae6fbe845459ac7dc8b43a8113a73acf55b7f8a0fbdff907fbf5001c63e3abae
MD5 661aeed1fc6e0a258c0e08586dfc0565
BLAKE2b-256 6ab8ca50ede15760b29a55d6fc5194255138b133e68e7901a5cc92dbc6bf81a8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page