Skip to main content

swh-loader-git

The Software Heritage Git Loader is a tool and a library to walk a local Git repository and inject into the SWH dataset all contained files that weren’t known before.

The main entry points are:

  • swh.loader.git.loader.GitLoader for the main loader which can ingest either local or remote git repository’s contents. This is the main implementation deployed in production.

  • swh.loader.git.loader.GitLoaderFromArchive which ingests a git repository from an archive.

  • swh.loader.git.directory.GitCheckoutLoader which ingests a git tree at a specific commit, branch or tag.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

See top-level LICENSE file for the full text of the GNU General Public License along with this program.

Dependencies

### Runtime

  • python3

  • python3-dulwich

  • python3-swh.core

  • python3-swh.model

  • python3-swh.storage

  • python3-swh.scheduler

### Test

  • python3-nose

Requirements

  • implementation language, Python3

  • coding guidelines: conform to PEP8

  • Git access: via dulwich

CLI Run

You can run the loader from a remote origin (loader) or from an origin on disk (from_disk) directly by calling:

swh loader -C <config-file> run git <git-repository-url>

or “git_disk”.

## Configuration sample

/tmp/git.yml:

storage:
  cls: remote
  args:
    url: http://localhost:5002/

Metadata

Release files for swh.loader.git 3.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for swh.loader.git 3.4.0
File Size Uploaded Provenance Actions
swh_loader_git-3.4.0.tar.gz 103.5 kB Without provenance Details for swh_loader_git-3.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for swh.loader.git 3.4.0
File Interpreter ABI Platform Actions
swh_loader_git-3.4.0-py3-none-any.whl Python 3 none any Details for swh_loader_git-3.4.0-py3-none-any.whl

Total release size: 209.4 kB

Release files / swh_loader_git-3.4.0.tar.gz

Download URL swh_loader_git-3.4.0.tar.gz
Size 103.5 kB
Tags Source
SHA-256 checksum
How to use checksums
816f954a705f3f13c8eec2f5c9ee1e4b787ac2ed496a9254cdf5e2a6de46546d
BLAKE2b-256 checksum
How to use checksums
6c05980b9c3e9adcc538ba0c90a6bbd6f6fa6233efd6c2d7009f1024e772f8f3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.15

Release files / swh_loader_git-3.4.0-py3-none-any.whl

Download URL swh_loader_git-3.4.0-py3-none-any.whl
Size 105.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8c59f99a046030fafe6216863c0b80690344f31bc719f750e9ec6fbb1953ce00
BLAKE2b-256 checksum
How to use checksums
3a32e49d53798b79730011ecdebd65b27f58fd695a239a3831858f307d5b24ea
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.15

Security for swh.loader.git 3.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

3.4.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page