Skip to main content

Git LFS for large files that need cloud storage.

Project description

Git Cloud Large File Storage

Git LFS for large files that need cloud storage.

Setup

AWS S3

  1. Install AWS CLI.
  2. Create an S3 bucket and AWS access keys.
  3. Put the access keys in ~/.aws/credentials as described here. Note, the profile used by gclfs is gclfs. This can be overridden in gclfs.config. See example below.
  4. Add bucket name to gclfs.config.

Example credentials file:

# ~/.aws/credentials
[default]
aws_access_key_id=AKIAASSFODNN7EXAMPLE
aws_secret_access_key=wJalrP0dnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[gclfs]
aws_access_key_id=BTIAIOSFODNN7EXAMPLE
aws_secret_access_key=xyalrXUghdEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Example config file with overridden profile name:

# gclfs.config
[s3]
bucket=my-awesome-bucket
profile=default

Usage

Tracking

To track files with a specific extension that will be uploaded to cloud storage, run gcl track *.<file_extension>

For example, to track wav files:

gcl track "*.wav"

This adds the extension to .gitattributes. When pushing to remote repo, this will sync all wav files to S3.

Important: To keep the large files from getting committed to the git repo, you still have to add them to .gitignore.

Commands

Your normal git commands work here and a remote git repo should be setup as usual (with git init and git remote add, etc.).

Example

gcl init (initialize repo)

gcl track "*.wav" (track wav files)

gcl add . (add files to commit)

gcl status (get repo status)

gcl commit -m "init commit" (commit changes)

gcl remote add origin git@github.com:me/my-awesome-project (add remote repo)

gcl push (push git repo and sync files with cloud)

gcl fetch (fetch changes)

gcl pull (pull remote changes and sync files with cloud)

Limitations

Currently, only AWS S3 is supported.

License

See the LICENSE file for license rights and limitations (MIT).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gclfs-0.1.1.tar.gz (5.7 kB view hashes)

Uploaded Source

Built Distribution

gclfs-0.1.1-py3-none-any.whl (7.2 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page