Skip to main content

A tool to clone efficiently all the repos in an organization

Project description

cloner

A tool to clone efficiently all the repos in an organization

PyPI GitHub release (latest by date) PyPI - Python Version GitHub last commit CI PyPi downloads

Install

When installing from PyPi, use wr-cloner, cloner was already taken :sad:

pip install wr-cloner

Usage

Note: If using cloner after cloning the project and not a PyPi package, add "python" at the start. There's an example of that in the examples section.

Usage: cloner [OPTIONS] GITHUB_ORGANIZATION

  A tool to clone efficiently all the repos in an organization.

Options:
  --version                       Show the version and exit.
  --token TEXT                    GitHub token to read private repos. This
                                  parameter is needed when cloning from a
                                  GitHub Enterprise server.
  --ghe TEXT                      GitHub Enterprise URL. It needs the
                                  GITHUB_ORGANIZATION parameter to clone repos
                                  from there and the TOKEN option as well.
  --threads INTEGER               Number of threads and processes to use. For
                                  maximum threads and processes on the system,
                                  use '--max-threads'  [default: 4]
  --logging [ERROR|WARNING|INFO|DEBUG]
                                  Logging level  [default: INFO]
  --path TEXT                     Sets a path where to clone the repositories
                                  (eg: ./another/path/)
  --git-options TEXT              Add options to the clone command (eg: --git-
                                  options "--depth 1"). By default, clones
                                  quietly (--quiet).
  --max-threads                   If declared, uses the maximum available
                                  threads and processes in the system. As per
                                  physical cores on the system cpu.
  --ignore-archived               If declared, will ignore archived repos when
                                  cloning.
  --help                          Show this message and exit.

Examples

# For github.com with 8 threads
cloner --threads 8 GITHUB_ORGANIZATION

# For github.com with the maximum threads on the system running
cloner --max-threads GITHUB_ORGANIZATION

# For GHE, default threads
cloner --ghe GHE_URL --token SUPER_SECURE_TOKEN GITHUB_ORGANIZATION

# Cloning with options
cloner --git-options "--depth 1" GITHUB_ORGANIZATION

Examples if cloning the repo

# Install dependencies with
make install
# Adjust PYTHONPATH
export PYTHONPATH=$PYTHONPATH:.
# Use it
python cloner GITHUB_ORGANIZATION

Contributing

Check the CONTRIBUTING.md file.

Security

Follow the instructions in the SECURITY.md file.

License

MIT

Other & Troubleshooting

Multithreading doesn't work to clone repos, since the os.system call is 1 for each PID. The splitting is done with multithreading, the cloning with multiprocessing. Same amount of threads and processes.

Windows usage and support

Star History

Star History Chart

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wr-cloner-1.9.4.tar.gz (16.0 kB view details)

Uploaded Source

Built Distribution

wr_cloner-1.9.4-py3-none-any.whl (12.4 kB view details)

Uploaded Python 3

File details

Details for the file wr-cloner-1.9.4.tar.gz.

File metadata

  • Download URL: wr-cloner-1.9.4.tar.gz
  • Upload date:
  • Size: 16.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.19

File hashes

Hashes for wr-cloner-1.9.4.tar.gz
Algorithm Hash digest
SHA256 f1757981e38f48202c75616f06230afca32bedb572a70129bc8367a9ca891ce7
MD5 5ce05fd0bc89717c5ba48f8d4b84e3a4
BLAKE2b-256 e78911b9802b2f17dce47e90e87ac1b7240cfb8cd0d222888b88a2c7785668d0

See more details on using hashes here.

File details

Details for the file wr_cloner-1.9.4-py3-none-any.whl.

File metadata

  • Download URL: wr_cloner-1.9.4-py3-none-any.whl
  • Upload date:
  • Size: 12.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.19

File hashes

Hashes for wr_cloner-1.9.4-py3-none-any.whl
Algorithm Hash digest
SHA256 531c1561e90d66f534603801306850ad41e6f6615b0aac4662eda4c5e6c28494
MD5 dd8532dd2a3ce454a4698113c05598bc
BLAKE2b-256 920907cd173d497d4c9ca51d0f9af0ba4ed706e0b094e43cac1acd49c4c0b2bb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page