Skip to main content

Git LFS for large files that need cloud storage.

Project description

Git Cloud Large File Storage

Git LFS for large files that need cloud storage.

Setup

AWS S3

  1. Install AWS CLI.
  2. Create an S3 bucket and AWS access keys.
  3. Put the access keys in ~/.aws/credentials as described here. Note, the profile used by gclfs is gclfs. This can be overridden in gclfs.config. See example below.
  4. Add bucket name to gclfs.config.

Example credentials file:

# ~/.aws/credentials
[default]
aws_access_key_id=AKIAASSFODNN7EXAMPLE
aws_secret_access_key=wJalrP0dnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[gclfs]
aws_access_key_id=BTIAIOSFODNN7EXAMPLE
aws_secret_access_key=xyalrXUghdEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Example config file with overridden profile name:

# gclfs.config
[s3]
bucket=my-awesome-bucket
profile=default

Usage

Tracking

To track files with a specific extension that will be uploaded to cloud storage, run gcl track *.<file_extension>

For example, to track wav files:

gcl track "*.wav"

This adds the extension to .gitattributes. When pushing to remote repo, this will sync all wav files to S3.

Important: To keep the large files from getting committed to the git repo, you still have to add them to .gitignore.

Commands

Your normal git commands work here and a remote git repo should be setup as usual (with git init and git remote add, etc.).

Example

gcl init (initialize repo)

gcl track "*.wav" (track wav files)

gcl add . (add files to commit)

gcl status (get repo status)

gcl commit -m "init commit" (commit changes)

gcl remote add origin git@github.com:me/my-awesome-project (add remote repo)

gcl push (push git repo and sync files with cloud)

gcl fetch (fetch changes)

gcl pull (pull remote changes and sync files with cloud)

Limitations

Currently, only AWS S3 is supported.

License

See the LICENSE file for license rights and limitations (MIT).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gclfs-0.1.1.tar.gz (5.7 kB view details)

Uploaded Source

Built Distribution

gclfs-0.1.1-py3-none-any.whl (7.2 kB view details)

Uploaded Python 3

File details

Details for the file gclfs-0.1.1.tar.gz.

File metadata

  • Download URL: gclfs-0.1.1.tar.gz
  • Upload date:
  • Size: 5.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.6

File hashes

Hashes for gclfs-0.1.1.tar.gz
Algorithm Hash digest
SHA256 c3eb8f0657e1a3b656fc35df3378b4e37528675e90d938162fb66ffd147cf403
MD5 d975b6f8be0e68aededbd4c824046ebf
BLAKE2b-256 60a99546e6cbce5f3e8665a8580dad8a7b5f026b54238a078ee34733b445b189

See more details on using hashes here.

File details

Details for the file gclfs-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: gclfs-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 7.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.6

File hashes

Hashes for gclfs-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 20e0b5ecc24483fdbb31de427fe703207c43df1279ef0af4a5fe8a6f2984ba10
MD5 270be043cf48685c9443ff2467a03fda
BLAKE2b-256 5e3582252c0806ac30398dfd6b1b446d907180d4f79cce48366488d80d2ba515

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page