cloudrepo-migrate

Moves a Nexus 2 or Nexus 3 server's repositories into CloudRepo and verifies every file: cloudrepo-migrate nexus2 or cloudrepo-migrate nexus3, with the same flags, report and exit codes.

• Maven: every file of each hosted Maven repository, POMs, maven-metadata.xml and checksum files included.
• npm: every version of each package in each hosted npm repository, scoped packages included, oldest first, then each dist-tag the source has (latest, next and every other) that CloudRepo lacks. A dist-tag CloudRepo already holds is never moved or removed: each one that names another version than the source's, or that the source lacks, is listed as kept in the report, which is not a failure. A deprecated version moves with its deprecation message; a version the source has unpublished is not in its packument, so it does not move. From version 0.4.0; versions before it moved only the version tagged latest, which --latest-only still does.
• PyPI (Nexus 3): every wheel and sdist of each hosted PyPI repository, uploaded through CloudRepo's PyPI upload form under the project name the file's own metadata gives (other files beside them, such as signatures, are counted in the report, not moved). --login does not cover PyPI yet: move it with CLOUDREPO_TOKEN.
• Skipped with a reason: proxy repositories (create a CloudRepo proxy instead), group repositories (create a CloudRepo group of the moved ones), virtual repositories, other formats, and files Nexus generates itself (.index, .meta, the root archetype-catalog.xml).

Before upload, each file's bytes are compared with the checksum Nexus recorded for it (Nexus 2's sha1; every checksum a Nexus 3 asset lists, a wheel's or sdist's sha256 among them; an npm package's dist.shasum and dist.integrity), and a file that differs is not uploaded. After upload, every file is read back from CloudRepo and its sha1 and sha256 compared with the source's bytes; a wheel or sdist must also be listed in its project's index, where pip looks for it. A checksum file the source carries (X.sha1, X.sha256, X.sha512, X.md5) must name the digest of X. Any mismatch fails the run and names the file. Nexus is only read, never written.

Install

pipx install cloudrepo-migrate

That installs it from pypi.org and puts cloudrepo-migrate on the PATH (pip install cloudrepo-migrate in a virtual environment works too).

It is one file, Python 3.8 or later, with no dependencies, so copying cloudrepo_migrate.py to a machine that can reach the Nexus server and running python3 cloudrepo_migrate.py --help works too.

Run

Create each target repository in CloudRepo first (same name as the Nexus repository, or map it with --repo SOURCE=TARGET) at https://admin.cloudrepo.io/repositories/create, and a repository token that can write them.

export NEXUS_USERNAME=deployer NEXUS_PASSWORD=...   # leave both unset for anonymous read
export CLOUDREPO_TOKEN=...
cloudrepo-migrate nexus2 --nexus-url https://nexus.example.com/nexus --org acme --dry-run
cloudrepo-migrate nexus2 --nexus-url https://nexus.example.com/nexus --org acme --report-json report.json
# Nexus 3: the same, with its base URL (no /nexus on a default install)
cloudrepo-migrate nexus3 --nexus-url https://nexus.example.com --org acme --dry-run
cloudrepo-migrate nexus3 --nexus-url https://nexus.example.com --org acme --report-json report.json

With the environment set, nothing prompts. On a terminal with a credential missing it asks for it.

Resume: run the same command again. A file CloudRepo already holds with the same bytes is counted as already present and not uploaded again; a 409 on a release CloudRepo already holds counts as moved once its bytes match.

Exit codes: 0 everything moved and verified; 1 at least one file failed (every failure is listed with its path; re-run to resume); 2 a usage or preflight problem, such as a missing target repository or a refused token (nothing moved).

Credentials

• Read only from the environment or a terminal prompt: never from the command line, never printed, never written to the report.
• The Nexus login is sent only to --nexus-url, the CloudRepo token only to CloudRepo.
• Redirects are never followed, and no URL is taken from a server's answer (a Nexus 3 asset's downloadUrl is never requested; a packument's dist.tarball contributes only its file name), so neither credential can be steered to another host.
• CloudRepo must be https (plain http only to localhost). A plain-http Nexus gets a warning.

Browser login (not live yet)

CloudRepo's server side for --login is not live yet; until it is, use CLOUDREPO_TOKEN as above. With it, nexus2 --org ORG --login (or nexus3) needs no token: the run builds its plan, prints a code and the page where your organization's owner approves exactly those repositories, and waits. The token it receives lives in memory for that run only, never on disk. --open-browser also opens the page on this machine. A set CLOUDREPO_TOKEN still wins. Exit code 4: the approval was denied, expired or refused (nothing moved).