Git2S3

Backup GitHub projects to AWS S3

Platform Supported

Deployments

Kick off

Recommendations

• Install python 3.10 or 3.11
• Use a dedicated virtual environment

Install Git2S3

python -m pip install git2s3

Initiate - IDE

import git2s3


if __name__ == '__main__':
    git = git2s3.Git2S3()
    git.start()

Initiate - CLI

git2s3 start

Use git2s3 --help for usage instructions.

Environment Variables

• GIT_API_URL - GitHub API endpoint. Defaults to https://api.github.com/
• GIT_OWNER - GitHub profile owner or organization name.
• GIT_TOKEN - GitHub token to get ALL repos (including private).
• GIT_IGNORE - List of repositories/gists to ignore. Defaults to []
• MAX_PER_PAGE - Max number of repos/gists to pull from a single page. Defaults to 100
• SOURCE - List of source options to back up. Defaults to [repo, gist, wiki]
• LOG - Log options to log to a file or stdout. Does not apply when custom logger is used
• DEBUG - Boolean flag to enable debug level logging. Does not apply when custom logger is used
• DRY_RUN - Boolean flag to skip upload to S3. Defaults to False
• LOCAL_STORE - Boolean flag to store the backup locally. Defaults to False
• INCOMPLETE_UPLOAD - Boolean flag to upload incomplete cloning. Defaults to False
• AWS_PROFILE_NAME - AWS profile name. Uses the CLI config value AWS_DEFAULT_PROFILE by default.
• AWS_ACCESS_KEY_ID - AWS access key ID. Uses the CLI config value AWS_ACCESS_KEY_ID by default.
• AWS_SECRET_ACCESS_KEY - AWS secret key. Uses the CLI config value AWS_SECRET_ACCESS_KEY by default.
• AWS_REGION_NAME - S3 bucket's region. Uses the CLI config value AWS_DEFAULT_REGION by default.
• AWS_BUCKET_NAME - AWS bucket name to store the backups.
• AWS_S3_PREFIX - S3 prefix (folder like) for the backup. Defaults to github
• BOTO3_RETRY_ATTEMPTS - Number of retries for Boto3 client config. Defaults to 10
• BOTO3_RETRY_MODE - Boto3 retry configuration for S3 client. Defaults to standard
• CUT_OFF_DAYS - Cut off threshold to back up only the repos/gists that were "updated"/"pushed to"

Coding Standards

Docstring format: Google
Styling conventions: PEP 8 and isort

Release Notes

Requirement

python -m pip install gitverse

Usage

gitverse-release reverse -f release_notes.rst -t 'Release Notes'

Linting

pre-commit will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL markdown files (including Wiki pages)

Requirement

python -m pip install sphinx==5.1.1 pre-commit recommonmark

Usage

pre-commit run --all-files

Pypi Package

https://pypi.org/project/git2s3/

Runbook

https://thevickypedia.github.io/git2s3/

License & copyright

© Vignesh Rao

Licensed under the MIT License