Skip to main content

replicate-github sets up and maintains mirrors of GitHub organizations and individual repos. It can serve webhook endpoints and keep the mirrors updated continuously, or it can be run ad hoc from the command line.

$ replicate-github --verbose mirror puppetlabs/puppet 'github/*'
mirror.Collection: Fetching puppetlabs/puppet
mirror.Collection: Fetching github/garethr-docker
mirror.Collection: Fetching github/github-ldap
^C
$ replicate-github --verbose serve --update-org github
WebhookServer: Webhook server listening on localhost:8080
127.0.0.1 - - [10/Aug/2016 01:18:35] "POST / HTTP/1.1" 202 -
mirror.Collection: Fetching puppetlabs/puppet
^C

Webhook

The webhook server accepts POST to any URL (the path is ignored). A shared secret should be set so that events can’t be sent from non-GitHub sources.

By default the webhook server ensures that mirrors are updated at least once a day if no events are received. See replicate-github serve --help for more information.

GitHub webhook configuration

Create the webhook with any URL under the one served by the application. Set the secret to a random string, and put it in replicate-github.yaml like so:

serve:
  secret: "secret configured for webhook in GitHub"

Select the “Repository” and “Push” events to send.

Security

If a secret is configured then any event not containing the correct secret will be rejected.

Only two parts of the payload are used: the full name of the repo, and the deleted flag. The full name of the repo is validated to ensure it contains no special characters (like / or .).

This will create or update a mirror for any repo name as long as the secret is correct. That means if you configure the webhook for a repo that hasn’t already been mirrored it will start mirroring the repo as soon as an event comes in.

Configuration file

Configuration is loaded from the value of --config-file, which defaults to /etc/replicate-github.yaml. The file requires three settings:

mirror_path: "/srv/replicate-github"
github_user: "GitHub username"
github_token: "GitHub API token"

You can generate a GitHub API token under Settings > Personal access tokens.

There is an additional top level option, workers, that sets the number of git subprocesses that can be run at once. It defaults to 1.

Optionally, defaults for subcommands (e.g. serve) may be set:

serve:
  secret: "secret configured for webhook in GitHub"
  port: 8000
  update_org:
    - github
    - puppetlabs

Note that parameter names in the configuration file use “_” instead of “-”. For example, the parameter --update-org is called update_org in the configuration file.

Metadata

Release files for replicate-github 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for replicate-github 1.0.0
File Size Uploaded
replicate-github-1.0.0.tar.gz 10.7 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for replicate-github 1.0.0
File Interpreter ABI Platform
replicate_github-1.0.0-py3-none-any.whl Python 3 none any Details

Total release size: 24.2 kB

Release files / replicate-github-1.0.0.tar.gz

Download URL replicate-github-1.0.0.tar.gz
Size 10.7 kB
Tags Source
SHA-256 checksum
How to use checksums
c0c71251f3735acbfb21926fdaad9398479f6e05e3ef1cf727e55d259dd0579b
BLAKE2b-256 checksum
How to use checksums
a747ad21078431ba58672ab2b36f4801b4ef2d9a141200694440b5e380f87baa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / replicate_github-1.0.0-py3-none-any.whl

Download URL replicate_github-1.0.0-py3-none-any.whl
Size 13.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7a783463d2f575d40ec137a60f18d6cba5fec5c0edd9a143d180095e5125fab6
BLAKE2b-256 checksum
How to use checksums
c1d7ab6a0d5a372c46adf17238f57c33d6856a9640937af12f4bbebe2fe5eb4d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files

0.6.2

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.1

2 release files

0.5.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page