Import or sync existing Github infrastructure into terraform
Project description
Github Terraform Import
Python library to allow simple programmatic control of importing a Github organization as terraform resources.
The library exposes four key components:
- Dataclass definitions of all terraform resources in the Github Provider (
github_terraform_import.formatter.github_types.*
) - Formatter to easily convert dataclass types into valid terraform configuration syntax (
github_terraform_import.formatter.Formatter
) - A metaclass required to implement a visitor pattern for all resources (
github_terraform_import.visitor.VisitMethodInjector
) - An interface with the Github REST API that will dynamically load Github resources based on methods defined in the given visitor class
github_terraform_import.provider.GithubProvider
Usage
Create a file for each repository which contains the following repository specific resources github_repository
, github_repository_collaborator
, github_team_repository
.
import os
import sys
from github_terraform_import.visitor import VisitMethodInjector
from github_terraform_import.github_types import (
GithubRepository,
GithubRepositoryCollaborator,
GithubTeamRepository,
)
from github_terraform_import.provider import GithubProvider
from github_terraform_import.formatter import Formatter
class RepositoryConfigurationGenerator(metaclass=VisitMethodInjector):
def __init__(self, path):
self._root = path
def visit_github_repository(self, repository: GithubRepository):
file_ = os.path.join(self._root, f"{repository.name}.tf")
with open(file_, "a+") as f:
Formatter.format(repository.name, repository, out=f)
def visit_github_repository_collaborator(
self, collaborator: GithubRepositoryCollaborator
):
file_ = os.path.join(self._root, f"{collaborator.repository}.tf")
with open(file_, "a+") as f:
Formatter.format(collaborator.username, collaborator, out=f)
def visit_github_team_repository(self, team: GithubTeamRepository):
file_ = os.path.join(self._root, f"{team.repository}.tf")
with open(file_, "a+") as f:
Formatter.format(team.team_id, team, out=f)
def main():
if len(sys.argv) < 4:
print("Usage: example {token} {organization} {output_directory}")
exit()
directory = sys.argv[3]
if not os.path.exists(directory):
os.makedirs(directory)
generator = RepositoryConfigurationGenerator(directory)
provider = GithubProvider(sys.argv[1], sys.argv[2])
provider.visit(generator)
if __name__ == "__main__":
main()
Features
- Wrap around the Github API to provide terraform resources
- Typed resources for each of the terraform resources
- Dynamic resource fetching based on declared visitor methods
- Automatic caching of resources on multiple runs
- Formatting of resource classes to terraform configuration
- Detectable missing visitor methods
Supported Resources
Resource | Supported | Notes |
---|---|---|
github_actions_secret | :heavy_check_mark: | plaintext_value value cannot be imported from Github API, this must be added manually. Defaults to 'WARNING: Secrets cannot be imported via Github API' |
github_branch | ||
github_branch_protection | :heavy_check_mark: | |
github_issue_label | :heavy_check_mark: | |
github_membership | :heavy_check_mark: | |
github_organization_block | :heavy_check_mark: | |
github_organization_project | :heavy_check_mark: | |
github_organization_webhook | :heavy_check_mark: | |
github_project_column | :heavy_check_mark: | |
github_repository | :heavy_check_mark: | delete_branch_on_merge , auto_init , gitignore_template , license_template are not provided by the API |
github_repository_collaborator | :heavy_check_mark: | |
github_repository_deploy_key | :heavy_check_mark: | |
github_repository_file | ||
github_repository_project | :heavy_check_mark: | |
github_repository_webhook | :heavy_check_mark: | |
github_team | :heavy_check_mark: | ldap_dn not provided by the API |
github_team_membership | :heavy_check_mark: | |
github_team_repository | :heavy_check_mark: | |
github_user_gpg_key | ||
github_user_invitation_accepter | ||
github_user_ssh_key |
VisitMethodInjector
Use the VisitMethodInjector
metaclass in your customised visitor class to provide instructions for handling various fetched resources.
Standard Use
The standard use case for your visitor class is to define a method for each of the resources you wish to import. For example if you wish to import all the repository webhooks, a class like FetchRepositoryWebhooks
would suffice.
class FetchRepositoryWebhooks(metaclass=VisitMethodInjector):
def visit_github_repository_webhook(self, webhook: GithubRepositoryWebhook):
pass
Visit methods must be named, as visit_{resource_name}
.
Default Method
If there is some method that should be run for every resource without an implemented visitor method, you can use the default method for this purpose.
class FetchWithDefault(metaclass=VisitMethodInjector):
def visit_github_repository_webhook(self, webhook: GithubRepositoryWebhook):
pass
def default(self, resource: Resource):
print(resource)
FetchWithDefault
will call default for all of the resources except github_repository_webhook
.
Validation
If you want to ensure that every resource has an implemented visitor method, then setting the class variable ignore_missing
to False
will cause a TypeError
to be thrown if there are any missing visitor methods.
class FetchWithValidation(metaclass=VisitMethodInjector):
ignore_missing = False
FetchWithValidation
will throw a type exception when constructed listing all of the missing visit methods.
Formatter
The Formatter
class provides a way of converting the supplied dataclasses for each resource into an appropriately formatted Terraform resource.
This is accomplished by calling the Formatter.format
method. The method can also be used for any custom dataclasses so long as they inherit from github_terraform_import.github_types.Resource
.
Any member variables starting with an underscore, conventionally private, are not generated in the resource.
@dataclass
class NestedDetails(Block):
description: str
topics: List[str]
@dataclass
class MyTerraformResource(Resource):
name: str
id: int
valid: bool
friends: List[str]
details: NestedDetails
The above MyTerraformResource
is an example dataclass definition which showcases the various supported types of the formatter.
Executing the following will write the resource to out.txt
details = NestedDetails(
description="Things I like",
topics=["verification", "optimisation"],
)
resource = MyTerraformResource(
name="a special resource",
id="special",
valid=True,
friends=[],
details=details,
)
with open("out.txt", "w") as file:
Formatter.format("my_resource_name", resource, out=file)
The contents of out should then be:
resource "my_terraform_resource" "my_resource_name" {
name = "a special resource"
id = special
valid = true
friends = []
details {
description = "Things I like"
topics = [
"verification"
"optimisation"
]
}
}
If the out
keyword argument is omitted from the Formatter.format
method then the resource will be printed to standard out.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for github_terraform_import-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1e00060be1bf21420fd308f11e179cc32d9fa32bc004e50ea46452751b97d31 |
|
MD5 | 841c9e7e9b5d43e5f6437157d90b201b |
|
BLAKE2b-256 | cae38a94403201cd18b248dc33a06eacaa2af4844ce0f1cb13c475e8f3b484e1 |
Hashes for github_terraform_import-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1cad689d84e81c3884068bbaf774a9e263c8ca7e7d17c8330f9b2368cdedab61 |
|
MD5 | dc475274a3e3f62d0ff9a97bf5ba795b |
|
BLAKE2b-256 | 5c0f419fffa4ab4cb70cc5eeb930bd56e4a9836dad6e8aeda01adf68a18797f5 |