cal-gitlab-packages

Download and upload package files for GitLab projects and groups.

Part of the data-diode mirroring system — the send side downloads packages from a source GitLab, and the recv side uploads them to a destination GitLab.

Installation

pip install cal-gitlab-packages

Usage

# Download package files from a group
cal-gitlab-packages download -u https://gitlab.com -p myorg/public -o ./packages

# Download with manifest for incremental runs
cal-gitlab-packages download -u https://gitlab.com -p myorg -o ./packages -m manifest.json

# Upload package files to a target GitLab
cal-gitlab-packages upload -u https://gitlab.example.com -p target-group -i ./packages -t TOKEN

Commands

download

Download package files from GitLab package registries. Supports generic and PyPI package types, version filtering, manifest-based incremental downloads, size limits, and parallel threads.

Uses the group-level packages API for efficient fetching and stores a last_seen_at timestamp in the manifest for fast no-change detection.

Saves .package-meta.json alongside downloaded files so the upload command knows which registry API to use (generic vs PyPI).

upload

Upload package files to GitLab. Routes uploads by package type:

• generic — Uploaded via PUT to the generic package registry
• pypi — Uploaded via multipart POST to the PyPI package registry (wheels and sdists)

Checks existing files by SHA256 before uploading and handles 409/403 responses as idempotent success.

Configuration

Settings can be provided via CLI args, environment variables, or a config file at ~/.config/cal-gitlab-packages/config.json:

{
    "gitlab_url": "https://gitlab.example.com",
    "token": "glpat-xxxxxxxxxxxx",
    "base_group": "myorg",
    "output": "./downloads",
    "version_filter": "release",
    "max_file_size": "100M",
    "max_total_size": "5G",
    "parallel": 32,
    "project_exclude": ["some/project"]
}

Licence

MIT License — Copyright (c) 2026 Cyber Assessment Labs