Terraform Remote State Manager(tfremote)

tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a defined structure for all cloud providers by removing the overheard of configuring and managing the path in storage buckets.

It works with:

👉 Google Storage Bucket

👉 AWS S3

👉 Azure Storage

❗️ Note Best practice is to make sure buckets are versioned.

Install package

pip install tfremote --upgrade

Environment setup

Install Python 3.6+
Using virtualenv is strongly recommended:

python3 -m venv <venv name>

Terraform 0.14.0 and above (download: https://www.terraform.io/downloads.html)

Default log level is WARNING, to change:

export TF_LOG_LEVEL to any of these: 'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'

❗️ Important - Two variables are required for using tf package (used set creat path in remote storage):

teamid

prjid

Required variables can be defined using:

As inline variables e.g.: -var='teamid=demo-team' -var='prjid=demo-project'

Inside .tfvars file e.g.: -var-file=<tfvars file location>

Two optional variables:

workspace and state_key can be defined using:

-w=<workspace_name>. If no workspace is provided default workspace is used.

s=<state_key name>. If no key is provided terraform is used.

Path created in S3 backend: /<teamid>/<prjid>/<workspace>/<state-key>.tfstate

For more information refer to Terraform documentation

Setup environment variables

Workspace list file location `TF_WORKSPACE_FILE_LOCATION`

export TF_WORKSPACE_FILE_LOCATION=<workspace yml file location>

Reference file: link

AWS

❗️ Important - s3 bucket for remote state should reside in us-west-2

Set these env variables:

export TF_AWS_BUCKET=<your_remote_state_bucket_name>
export TF_AWS_BUCKET_REGION=us-west-2

One of below environment variable is required:

export TF_AWS_PROFILE=<aws profile to use>

export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>

Azure

To create storage for remote state there is handy script.

Run scripts/remote_state.sh (fill in the required information)

Set below env variables:

export TF_AZURE_STORAGE_ACCOUNT=<remote state storage account name>
export TF_AZURE_CONTAINER=<remote state container>
export ARM_ACCESS_KEY=<storage account access key>

GCP(gcloud)

https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform

Set below env variables:

export TF_GCLOUD_BUCKET=<remote state storage bucket name>
export TF_GCLOUD_CREDENTIALS=json credentials file path>

Usage

For GCP(gcloud):

tf plan -c=gcloud -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Google Storage Bucket:

alt text

For AWS:

tf plan -c=aws -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in AWS S3:

alt text

If you need to specify state_key in S3, specify -s=tryme-key

For Azure:

tf plan -c=azure -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace

The structure in Azure Storage:

alt text

For more available options:

tf --help
usage: tf [-h] [-var] [-var-file] [-c] [-w] [-wp] [-s] [-no-color] [-json] [-out] [-f] [-nf] [-v]

Terraform remote state wrapper package
--------------------------------------
Usage: Set below env variables to begin (more information: https://github.com/tomarv2/tfremote):
TF_WORKSPACE_FILE_LOCATION
aws: TF_AWS_BUCKET, TF_AWS_BUCKET_REGION=us-west-2, TF_AWS_PROFILE or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
azure: TF_AZURE_STORAGE_ACCOUNT, TF_AZURE_CONTAINER, ARM_ACCESS_KEY
gcloud: TF_GCLOUD_BUCKET, TF_GCLOUD_CREDENTIALS

optional arguments:
  -h, --help  show this help message and exit
  -var        Set Terraform configuration variable. This flag can be set multiple times
  -var-file   Set Terraform configuration variables from a file. This flag can be set multiple times
  -c          Specify cloud provider (default: 'aws'). Supported values: gcloud, aws, or azure
  -w          Specify existing workspace name(default: 'default')
  -wp         Overwrite workspace directory path structure
  -s          File name in remote state (default: 'terraform.tfstate')
  -no-color   Disables terminal formatting sequences in the output
  -json       Enables the machine readable JSON UI output
  -out        Writes the generated plan to the given filename in an opaque file format
  -f          Enable FIPS endpoints (default: True)
  -nf         Disable FIPS endpoints
  -v          show program's version number and exit

Metadata

Release files for tfremote 0.0.36

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tfremote 0.0.36
File	Size	Uploaded	Provenance
tfremote-0.0.36.tar.gz	18.5 kB	Sep 9, 2021	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for tfremote 0.0.36
File	Interpreter	ABI	Platform	Reset
tfremote-0.0.36-py3-none-any.whl 18.8 kB Sep 9, 2021 Without provenance	Python 3	none	any	Details

Total release size: 37.3 kB

Release files / tfremote-0.0.36.tar.gz

Download URL	tfremote-0.0.36.tar.gz
Size	18.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`2d4711786065e48422878217a6089f966f437b9015ebf5a70b27473d6cedb8e6`
BLAKE2b-256 checksum How to use checksums	`69ee916a3cf38bfe45efc770857679663c03013cc0a2065cf43f9d6b3fa12572`
Upload date	Sep 9, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.9.5`

Release files / tfremote-0.0.36-py3-none-any.whl

Download URL	tfremote-0.0.36-py3-none-any.whl
Size	18.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`ae6fbe845459ac7dc8b43a8113a73acf55b7f8a0fbdff907fbf5001c63e3abae`
BLAKE2b-256 checksum How to use checksums	`6ab8ca50ede15760b29a55d6fc5194255138b133e68e7901a5cc92dbc6bf81a8`
Upload date	Sep 9, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.9.5`

Security for tfremote 0.0.36

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.