Metadata-Version: 2.4
Name: cloudrepo-migrate
Version: 0.5.0
Summary: Move a Nexus 2, Nexus 3 or Artifactory server's Maven and npm repositories, and Nexus 3's PyPI repositories, into CloudRepo, verifying every file.
Project-URL: Homepage, https://www.cloudrepo.io
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# cloudrepo-migrate

Moves a Nexus 2, Nexus 3 or Artifactory server's repositories into CloudRepo and verifies every file: `cloudrepo-migrate nexus2`, `cloudrepo-migrate nexus3` or `cloudrepo-migrate artifactory`, with the same flags, report and exit codes.

- **Maven**: every file of each hosted (Nexus) or local (Artifactory) Maven repository, POMs, `maven-metadata.xml` and checksum files included.
- **npm**: every version of each package in each hosted or local npm repository, scoped packages included, oldest first, then each dist-tag the source has (`latest`, `next` and every other) that CloudRepo lacks. A dist-tag CloudRepo already holds is never moved or removed: each one that names another version than the source's, or that the source lacks, is listed as `kept` in the report, which is not a failure. A deprecated version moves with its deprecation message; a version the source has unpublished is not in its packument, so it does not move. This is the default from version 0.4.0 for Nexus, and from 0.5.0, the first version with `artifactory`, for Artifactory. Before 0.4.0, `nexus2` and `nexus3` moved only the version tagged `latest`, which `--latest-only` still does. For Artifactory, npm is tested against a stand-in for Artifactory's npm API, not yet against a real Artifactory server.
- **PyPI** (Nexus 3): every wheel and sdist of each hosted PyPI repository, uploaded through CloudRepo's PyPI upload form under the project name the file's own metadata gives (other files beside them, such as signatures, are counted in the report, not moved). `--login` does not cover PyPI yet: move it with `CLOUDREPO_TOKEN`.
- **Skipped with a reason**: proxy and remote repositories (create a CloudRepo proxy instead), group repositories (create a CloudRepo group of the moved ones), virtual and federated repositories, other formats, and files the server generates itself (Nexus's `.index`, `.meta` and root `archetype-catalog.xml`; Artifactory's `.index` and `.npm`). An Artifactory repository's root `archetype-catalog.xml` is not moved either (this version moves no archetype catalog), and the report names it on that repository's line.

Before upload, each file's bytes are compared with the checksum the source recorded for it (Nexus 2's sha1; every checksum a Nexus 3 asset lists, a wheel's or sdist's sha256 among them; the sha1 and sha256 Artifactory's AQL search lists, which Artifactory OSS answers as Pro does; an npm package's `dist.shasum` and `dist.integrity`), and a file that differs is not uploaded. After upload, every file is read back from CloudRepo and its sha1 and sha256 compared with the source's bytes; a wheel or sdist must also be listed in its project's index, where pip looks for it. A checksum file the source carries (`X.sha1`, `X.sha256`, `X.sha512`, `X.md5`) must name the digest of `X`. Artifactory serves `X.sha1` and `X.md5` without storing them, so for Artifactory the run writes both from `X`'s bytes as downloaded and checked, never from Artifactory's numbers alone, and Maven finds them in CloudRepo as it did in Artifactory. Any mismatch fails the run and names the file. The source is only read, never written.

## Install

```sh
pipx install cloudrepo-migrate
```

That installs it from pypi.org and puts `cloudrepo-migrate` on the PATH (`pip install cloudrepo-migrate` in a virtual environment works too).

It is one file, Python 3.8 or later, with no dependencies, so copying `cloudrepo_migrate.py` to a machine that can reach the Nexus or Artifactory server and running `python3 cloudrepo_migrate.py --help` works too.

## Run

Create each target repository in CloudRepo first (same name as the source repository, or map it with `--repo SOURCE=TARGET`) at https://admin.cloudrepo.io/repositories/create, and a repository token that can write them.

```sh
export NEXUS_USERNAME=deployer NEXUS_PASSWORD=...   # leave both unset for anonymous read
export CLOUDREPO_TOKEN=...
cloudrepo-migrate nexus2 --nexus-url https://nexus.example.com/nexus --org acme --dry-run
cloudrepo-migrate nexus2 --nexus-url https://nexus.example.com/nexus --org acme --report-json report.json
# Nexus 3: the same, with its base URL (no /nexus on a default install)
cloudrepo-migrate nexus3 --nexus-url https://nexus.example.com --org acme --dry-run
cloudrepo-migrate nexus3 --nexus-url https://nexus.example.com --org acme --report-json report.json
```

Artifactory (self-hosted or JFrog Cloud), with an access token, or a legacy API key in `ARTIFACTORY_API_KEY`:

```sh
export ARTIFACTORY_ACCESS_TOKEN=...   # read access to the local repositories you move
export CLOUDREPO_TOKEN=...
cloudrepo-migrate artifactory --artifactory-url https://artifactory.example.com/artifactory --org acme --dry-run
cloudrepo-migrate artifactory --artifactory-url https://artifactory.example.com/artifactory --org acme --report-json report.json
```

| Flag                                 | Meaning                                                                                                                                       |
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `--nexus-url URL`                    | The Nexus base URL: ending in `/nexus` on a default Nexus 2 install, the bare host on Nexus 3.                                                |
| `--artifactory-url URL`              | For `artifactory`: the Artifactory base URL, ending in `/artifactory`.                                                                        |
| `--org ORG` or `--cloudrepo-url URL` | Your CloudRepo organization, or its full base URL (https).                                                                                    |
| `--repo SOURCE[=TARGET]`             | Move only these repositories (repeatable). Default: every hosted or local Maven and npm repository, and every Nexus 3 hosted PyPI repository. |
| `--dry-run`                          | List what would move. Uploads nothing. Checks the token and targets when `CLOUDREPO_TOKEN` is set.                                            |
| `--latest-only`                      | For npm, move only the version each package's `latest` dist-tag names (default: every version, with its dist-tags).                           |
| `--report-json PATH`                 | Also write the report as JSON.                                                                                                                |
| `--jobs N`                           | Parallel Maven and PyPI uploads (default 4).                                                                                                  |
| `--work-dir DIR`                     | Where each file is spooled while it moves (default `TMPDIR`).                                                                                 |
| `--nexus-ca-file PEM`                | CA bundle for a Nexus with a private certificate (`--artifactory-ca-file` for Artifactory).                                                   |

With the environment set, nothing prompts. On a terminal with a credential missing it asks for it.

**Resume**: run the same command again. A file CloudRepo already holds with the same bytes is counted as already present and not uploaded again; a `409` on a release CloudRepo already holds counts as moved once its bytes match.

**Exit codes**: `0` everything moved and verified; `1` at least one file failed (every failure is listed with its path; re-run to resume); `2` a usage or preflight problem, such as a missing target repository or a refused token (nothing moved).

## Credentials

- Read only from the environment or a terminal prompt: never from the command line, never printed, never written to the report.
- Artifactory: `ARTIFACTORY_ACCESS_TOKEN` is sent as `Authorization: Bearer`; without it, `ARTIFACTORY_API_KEY` is sent as `X-JFrog-Art-Api` (JFrog has deprecated API keys, so prefer a token). Neither is ever prompted for.
- The source's login is sent only to `--nexus-url` or `--artifactory-url`, the CloudRepo token only to CloudRepo.
- Redirects are never followed, and no URL is taken from a server's answer (a Nexus 3 asset's `downloadUrl` is never requested; a packument's `dist.tarball` contributes only its file name), so neither credential can be steered to another host.
- CloudRepo must be https (plain http only to localhost). A plain-http Nexus or Artifactory gets a warning.

## Browser login (not live yet)

CloudRepo's server side for `--login` is not live yet; until it is, use `CLOUDREPO_TOKEN` as above. With it, `nexus2 --org ORG --login` (or `nexus3`, or `artifactory`) needs no token: the run builds its plan, prints a code and the page where your organization's owner approves exactly those repositories, and waits. The token it receives lives in memory for that run only, never on disk. `--open-browser` also opens the page on this machine. A set `CLOUDREPO_TOKEN` still wins. Exit code `4`: the approval was denied, expired or refused (nothing moved).
