Skip to main content

Github Terraform Import

Python library to allow simple programmatic control of importing a Github organization as terraform resources.

The library exposes four key components:

  • Dataclass definitions of all terraform resources in the Github Provider (github_terraform_import.formatter.github_types.*)
  • Formatter to easily convert dataclass types into valid terraform configuration syntax (github_terraform_import.formatter.Formatter)
  • A metaclass required to implement a visitor pattern for all resources (github_terraform_import.visitor.VisitMethodInjector)
  • An interface with the Github REST API that will dynamically load Github resources based on methods defined in the given visitor class github_terraform_import.provider.GithubProvider

Usage

Create a file for each repository which contains the following repository specific resources github_repository, github_repository_collaborator, github_team_repository.

import os
import sys

from github_terraform_import.visitor import VisitMethodInjector
from github_terraform_import.github_types import (
    GithubRepository,
    GithubRepositoryCollaborator,
    GithubTeamRepository,
)
from github_terraform_import.provider import GithubProvider
from github_terraform_import.formatter import Formatter


class RepositoryConfigurationGenerator(metaclass=VisitMethodInjector):
    def __init__(self, path):
        self._root = path

    def visit_github_repository(self, repository: GithubRepository):
        file_ = os.path.join(self._root, f"{repository.name}.tf")
        with open(file_, "a+") as f:
            Formatter.format(repository.name, repository, out=f)

    def visit_github_repository_collaborator(
        self, collaborator: GithubRepositoryCollaborator
    ):
        file_ = os.path.join(self._root, f"{collaborator.repository}.tf")
        with open(file_, "a+") as f:
            Formatter.format(collaborator.username, collaborator, out=f)

    def visit_github_team_repository(self, team: GithubTeamRepository):
        file_ = os.path.join(self._root, f"{team.repository}.tf")
        with open(file_, "a+") as f:
            Formatter.format(team.team_id, team, out=f)


def main():
    if len(sys.argv) < 4:
        print("Usage: example {token} {organization} {output_directory}")
        exit()

    directory = sys.argv[3]
    if not os.path.exists(directory):
        os.makedirs(directory)

    generator = RepositoryConfigurationGenerator(directory)
    provider = GithubProvider(sys.argv[1], sys.argv[2])
    provider.visit(generator)


if __name__ == "__main__":
    main()

Features

  • Wrap around the Github API to provide terraform resources
  • Typed resources for each of the terraform resources
  • Dynamic resource fetching based on declared visitor methods
  • Automatic caching of resources on multiple runs
  • Formatting of resource classes to terraform configuration
  • Detectable missing visitor methods

Supported Resources

Resource Supported Notes
github_actions_secret ✔️ plaintext_value value cannot be imported from Github API, this must be added manually. Defaults to 'WARNING: Secrets cannot be imported via Github API'
github_branch
github_branch_protection ✔️
github_issue_label ✔️
github_membership ✔️
github_organization_block ✔️
github_organization_project ✔️
github_organization_webhook ✔️
github_project_column ✔️
github_repository ✔️ delete_branch_on_merge, auto_init, gitignore_template, license_template are not provided by the API
github_repository_collaborator ✔️
github_repository_deploy_key ✔️
github_repository_file
github_repository_project ✔️
github_repository_webhook ✔️
github_team ✔️ ldap_dn not provided by the API
github_team_membership ✔️
github_team_repository ✔️
github_user_gpg_key
github_user_invitation_accepter
github_user_ssh_key

VisitMethodInjector

Use the VisitMethodInjector metaclass in your customised visitor class to provide instructions for handling various fetched resources.

Standard Use

The standard use case for your visitor class is to define a method for each of the resources you wish to import. For example if you wish to import all the repository webhooks, a class like FetchRepositoryWebhooks would suffice.

class FetchRepositoryWebhooks(metaclass=VisitMethodInjector):
    def visit_github_repository_webhook(self, webhook: GithubRepositoryWebhook):
        pass

Visit methods must be named, as visit_{resource_name}.

Default Method

If there is some method that should be run for every resource without an implemented visitor method, you can use the default method for this purpose.

class FetchWithDefault(metaclass=VisitMethodInjector):
    def visit_github_repository_webhook(self, webhook: GithubRepositoryWebhook):
        pass

    def default(self, resource: Resource):
        print(resource)

FetchWithDefault will call default for all of the resources except github_repository_webhook.

Validation

If you want to ensure that every resource has an implemented visitor method, then setting the class variable ignore_missing to False will cause a TypeError to be thrown if there are any missing visitor methods.

class FetchWithValidation(metaclass=VisitMethodInjector):
    ignore_missing = False

FetchWithValidation will throw a type exception when constructed listing all of the missing visit methods.

Formatter

The Formatter class provides a way of converting the supplied dataclasses for each resource into an appropriately formatted Terraform resource.

This is accomplished by calling the Formatter.format method. The method can also be used for any custom dataclasses so long as they inherit from github_terraform_import.github_types.Resource.

Any member variables starting with an underscore, conventionally private, are not generated in the resource.

@dataclass
class NestedDetails(Block):
    description: str
    topics: List[str]

@dataclass
class MyTerraformResource(Resource):
    name: str
    id: int
    valid: bool

    friends: List[str]

    details: NestedDetails

The above MyTerraformResource is an example dataclass definition which showcases the various supported types of the formatter.

Executing the following will write the resource to out.txt

details = NestedDetails(
    description="Things I like",
    topics=["verification", "optimisation"],
)

resource = MyTerraformResource(
    name="a special resource",
    id="special",
    valid=True,
    friends=[],
    details=details,
)


with open("out.txt", "w") as file:
    Formatter.format("my_resource_name", resource, out=file)

The contents of out should then be:

resource "my_terraform_resource" "my_resource_name" {
    name = "a special resource"
    id = special
    valid = true
    friends = []
    details {
        description = "Things I like"
        topics = [
            "verification"
            "optimisation"
        ]
    }
}

If the out keyword argument is omitted from the Formatter.format method then the resource will be printed to standard out.

Metadata

Release files for github-terraform-import 0.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for github-terraform-import 0.0.1
File Size Uploaded Provenance Actions
github_terraform_import-0.0.1.tar.gz 15.4 kB Without provenance Details for github_terraform_import-0.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for github-terraform-import 0.0.1
File Interpreter ABI Platform Actions
github_terraform_import-0.0.1-py3-none-any.whl Python 3 none any Details for github_terraform_import-0.0.1-py3-none-any.whl

Total release size: 37.7 kB

Release files / github_terraform_import-0.0.1.tar.gz

Download URL github_terraform_import-0.0.1.tar.gz
Size 15.4 kB
Tags Source
SHA-256 checksum
How to use checksums
e1e00060be1bf21420fd308f11e179cc32d9fa32bc004e50ea46452751b97d31
BLAKE2b-256 checksum
How to use checksums
cae38a94403201cd18b248dc33a06eacaa2af4844ce0f1cb13c475e8f3b484e1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.0.0 requests-toolbelt/0.8.0 tqdm/4.46.0 CPython/3.7.7

Release files / github_terraform_import-0.0.1-py3-none-any.whl

Download URL github_terraform_import-0.0.1-py3-none-any.whl
Size 22.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1cad689d84e81c3884068bbaf774a9e263c8ca7e7d17c8330f9b2368cdedab61
BLAKE2b-256 checksum
How to use checksums
5c0f419fffa4ab4cb70cc5eeb930bd56e4a9836dad6e8aeda01adf68a18797f5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.0.0 requests-toolbelt/0.8.0 tqdm/4.46.0 CPython/3.7.7

Security for github-terraform-import 0.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.0.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page