Metadata-Version: 2.5
Name: repoclone-cli
Version: 0.1.1
Summary: Bulk clone, update and report on every repository in a GitLab group or GitHub organisation
Project-URL: Homepage, https://github.com/devops-monk/repoclone-cli
Project-URL: Issues, https://github.com/devops-monk/repoclone-cli/issues
License: MIT
License-File: LICENSE
Keywords: cli,clone,devtools,git,github,gitlab
Requires-Python: >=3.11
Requires-Dist: click>=8.1
Requires-Dist: pygithub>=2.3
Requires-Dist: python-gitlab>=4.4
Requires-Dist: rich>=13.7
Description-Content-Type: text/markdown

# repoclone

Clone, update and report on **every repository in a GitLab group or GitHub organisation** — in parallel, safely, and repeatably.

Point it at a namespace and it mirrors the whole tree onto your laptop. Run it again next week and it fast-forwards what changed, leaves your local work alone, and tells you what happened.

```console
$ repoclone clone platform/tooling --token glpat-xxxx
gitlab platform/tooling → /Users/you/src/work
   cloned  platform/tooling/api
   cloned  platform/tooling/cli
  updated  platform/tooling/docs
  skipped  platform/tooling/legacy (local changes, left alone)

42 repositories: 30 cloned, 9 updated, 2 unchanged, 1 skipped
```

## Install

Requires Python 3.11+ and `git` on your PATH.

```bash
# recommended — isolated install, command on your PATH
uv tool install repoclone-cli

# or with pipx
pipx install repoclone-cli

# or plain pip
pip install repoclone-cli

# or straight from the repository, before a release is cut
uv tool install git+https://github.com/devops-monk/repoclone-cli
```

The package is published as **`repoclone-cli`** and installs a command called **`repoclone`** — the
name `repoclone` was already taken on PyPI by an unrelated project.

Check everything is wired up:

```bash
repoclone check --token glpat-xxxx
```

New to it? Follow [the step-by-step first test](docs/usage.md#step-by-step-first-test) — the first four steps cannot touch your disk.

## Quick start

```bash
# GitLab group (path or numeric id), token on the command line
repoclone clone platform/tooling --token glpat-xxxx --dest ~/src/work

# GitHub organisation
repoclone clone kubernetes --provider github --token ghp_xxxx --dest ~/src/oss

# see what would happen first
repoclone clone platform/tooling --token glpat-xxxx --dry-run
```

**Run the same command again tomorrow.** Existing clones are pulled up to date, anything new is cloned, and repositories with uncommitted work are left alone. That is the default; `--sync` says it explicitly if you prefer it spelled out in a script:

```bash
repoclone clone platform/tooling --token glpat-xxxx --sync
```

### Private / self-hosted instances

Pass your own domain with `--host` — this is the common case for company GitLab and GitHub Enterprise, and everything else works identically:

```bash
# self-hosted GitLab on your own domain
repoclone clone platform/tooling \
  --host https://git.tech.example.com \
  --token glpat-xxxx \
  --dest ~/src/work

# GitHub Enterprise Server
repoclone clone platform \
  --provider github \
  --host https://github.example.com \
  --token ghp_xxxx \
  --dest ~/src/work

# internal host with a self-signed certificate
repoclone clone platform/tooling --host https://git.internal.example.com --token $T --insecure
```

The API path is derived for you (`/api/v4` for GitLab, `/api/v3` for GitHub Enterprise) — give it the plain domain. Without `--host` the public gitlab.com / github.com is used. Put the host in a profile once and you never type it again.

Re-running is the normal case: repositories already on disk are fast-forwarded, new ones are cloned, and anything you have edited locally is left untouched.

## Commands

| Command | What it does |
|---|---|
| `repoclone clone [NAMESPACE]...` | Clone new repositories, update existing ones |
| `repoclone list [NAMESPACE]...` | List what would be cloned — no disk changes |
| `repoclone report` | Branch, last commit, dirty state of clones already on disk |
| `repoclone check` | Verify git, config and token before a large run |
| `repoclone config init` | Write a starter configuration file |
| `repoclone config show` | Show the settings that would apply, and from where |

Every command supports `-h/--help`, and `repoclone --version` prints the installed version. Passing an unknown option or a bad value prints the full help for that command, so you never have to guess.

`[NAMESPACE]...` is a GitLab group path (`platform/tooling`) or numeric id (`4231`), or a GitHub organisation or user (`kubernetes`). Pass several to do them in one run; pass none and the profile's `namespaces` are used.

## Options

Every flag is listed here. Nothing is required except a token and a namespace, and both of those can come from a profile instead.

### Connection — accepted by `clone`, `list`, `check`

| Option | Default | Notes |
|---|---|---|
| `--profile, -P NAME` | `default_profile` | Which profile to read defaults from |
| `--provider [gitlab\|github]` | `gitlab` | Which forge to talk to |
| `--host URL` | public forge | Self-hosted GitLab or GitHub Enterprise. Plain domain — the API path is added for you |
| `--token TEXT` | — | Read-only token. `read_api` (GitLab) or `repo:read` (GitHub) |
| `--archived / --no-archived` | `--no-archived` | Include archived repositories |
| `--forks / --no-forks` | `--no-forks` | Include forked repositories |
| `--insecure` | off | Skip TLS verification (self-signed internal forge) |
| `--timeout N` | `30` | Seconds allowed per operation. Raise it for large repositories |

### Selection — accepted by `clone`, `list`

| Option | Default | Notes |
|---|---|---|
| `--include REGEX` | — | Only repositories whose path matches. Repeatable — any match wins |
| `--exclude REGEX` | — | Skip repositories whose path matches. Repeatable — `--exclude` beats `--include` |

Patterns are Python regular expressions matched anywhere in the full repository path, not anchored globs. An invalid pattern is reported by name rather than crashing. Empty repositories are always skipped.

### `clone` only

| Option | Default | Notes |
|---|---|---|
| `--dest, -d PATH` | current directory | Where the clones go |
| `--protocol [ssh\|https]` | `ssh` | `https` uses the token for auth |
| `--update, -u [pull\|fetch\|skip]` | `pull` | What to do with repositories already on disk |
| `--sync` | on | Pull existing clones, clone new ones. Explicit form of the default; conflicts with `--update fetch\|skip` |
| `--jobs, -j N` | `8` | Parallel git operations |
| `--depth N` | `0` | Shallow clone depth. `0` keeps full history |
| `--blobless / --no-blobless` | `--no-blobless` | Partial clone: full history, file contents fetched on demand |
| `--flat / --nested` | `--nested` | Flatten `a/b/c` into one directory level instead of nesting it |
| `--dry-run` | off | Show the plan, change nothing |
| `--prune` | off | Also report local clones that no longer exist remotely |
| `--quiet, -q` | off | Only print the final summary — good for cron |

### `list` only

| Option | Default | Notes |
|---|---|---|
| `--output, -o [table\|json\|csv]` | `table` | How to print the results |

### `report` only

| Option | Default | Notes |
|---|---|---|
| `--profile, -P NAME` | `default_profile` | Which profile to read defaults from |
| `--dest, -d PATH` | current directory | Which directory of clones to inspect |
| `--output, -o [table\|json\|csv]` | `table` | How to print the results |
| `--stale-days N` | — | Only repositories whose last commit is older than this |

`report` reads the disk only — it needs no token and never contacts the forge.

### `config` only

| Option | Command | Notes |
|---|---|---|
| `--force` | `config init` | Overwrite an existing configuration file |
| `--profile, -P NAME` | `config show` | Which profile to resolve |
| `--output, -o [table\|json\|csv]` | `config show` | How to print the settings |

### Worked examples, one per flag

```bash
# --- picking what to clone -------------------------------------------------
# several namespaces in one run
repoclone clone platform/tooling platform/data --token $T

# a GitLab group by numeric id, and a GitHub user rather than an org
repoclone clone 4231 --token $T
repoclone clone torvalds --provider github --token $T

# only the services, skipping anything archived-looking
repoclone clone platform --token $T --include '/services/' --exclude '^platform/old-'

# include forks and archived repositories in a full audit
repoclone clone platform --token $T --forks --archived

# a profile that includes forks, overridden back off for one run
repoclone clone platform --token $T --no-forks --no-archived

# --- how much to fetch -----------------------------------------------------
# a fast, space-efficient mirror for code search
repoclone clone platform --token $T --blobless -j 16

# just the tip of each default branch
repoclone clone platform --token $T --depth 1

# full history and full contents, overriding a blobless profile
repoclone clone platform --token $T --no-blobless --depth 0

# --- where the files land --------------------------------------------------
# the namespace you asked for is stripped, so with a repository platform/tooling/api:
repoclone clone platform/tooling --token $T --dest ~/src   # → ~/src/api
repoclone clone platform --token $T --dest ~/src           # → ~/src/tooling/api
repoclone clone platform --token $T --dest ~/src --flat    # → ~/src/tooling__api
repoclone clone platform --token $T --dest ~/src --nested  # the default, spelled out

# clone over https instead of ssh — no ssh key needed
repoclone clone platform --token $T --protocol https

# --- updating what you already have ----------------------------------------
# daily refresh: pull everything, clone anything new
repoclone clone platform --token $T --sync

# refresh remote refs without ever touching working trees
repoclone clone platform --token $T --update fetch

# clone only the new repositories, leave existing clones completely alone
repoclone clone platform --token $T --update skip

# quiet daily run, and tell me about clones that are gone from the forge
repoclone clone platform --token $T --quiet --prune

# --- being careful ---------------------------------------------------------
repoclone clone platform --token $T --dry-run                # change nothing
repoclone clone platform --token $T --timeout 120            # fail fast on a stuck repository
repoclone clone platform --host https://git.internal.example.com --token $T --insecure

# --- looking without touching ----------------------------------------------
repoclone list platform --token $T                            # table
repoclone list platform --token $T --output json | jq -r '.[].repository'
repoclone list platform --token $T --forks --output csv > forks.csv

repoclone report                                              # clones in the current directory
repoclone report --dest ~/src/work --output table
repoclone report --dest ~/src/work --stale-days 180 --output csv > stale.csv

repoclone check --token $T                                    # git, config, token, whoami
repoclone check --profile work
repoclone check --provider github --host https://github.example.com --token $T

# --- configuration ---------------------------------------------------------
repoclone config init                    # write a starter config file
repoclone config init --force            # overwrite the existing one
repoclone config show                    # what would apply right now
repoclone config show --profile oss --output json
repoclone clone --profile oss            # namespaces, host and dest all from the profile
```

## Tokens

Pass the token on the command line with `--token`, which is what most people do:

```bash
repoclone clone platform/tooling --token glpat-xxxx
```

If you would rather not have it in shell history, any of these also work — in this order of precedence:

1. `--token`
2. `REPOCLONE_TOKEN`
3. `token_env` in your profile (points at a variable of your choosing)
4. `GITLAB_TOKEN` / `GITHUB_TOKEN`
5. `token_cmd` in your profile, e.g. `op read op://Private/GitLab/token`

A read-only scope is enough. Over `https`, the token is used for the clone and then **stripped from the saved remote**, so it never lands in `.git/config`. Tokens are redacted from error output.

## Configuration profiles

Typing the same flags every day gets old. `repoclone config init` writes `~/.config/repoclone/config.toml`:

```toml
default_profile = "work"

[profiles.work]
provider = "gitlab"
host = "https://gitlab.example.com"
namespaces = ["platform/tooling"]
dest = "~/src/work"
protocol = "ssh"
concurrency = 8
token_env = "GITLAB_TOKEN"

[profiles.oss]
provider = "github"
namespaces = ["kubernetes"]
dest = "~/src/oss"
protocol = "https"
```

Then:

```bash
repoclone clone                 # uses the default profile
repoclone clone --profile oss   # uses the oss profile
repoclone config show           # what would apply right now
```

### Every profile key

| Key | Type | Default | Same as |
|---|---|---|---|
| `provider` | string | `gitlab` | `--provider` |
| `host` | string | public forge | `--host` |
| `namespaces` | list | — | the `NAMESPACE` arguments |
| `dest` | path | current directory | `--dest` |
| `protocol` | `ssh`/`https` | `ssh` | `--protocol` |
| `update` | `pull`/`fetch`/`skip` | `pull` | `--update` |
| `concurrency` | integer | `8` | `--jobs` |
| `depth` | integer | `0` | `--depth` |
| `blobless` | boolean | `false` | `--blobless` |
| `flat` | boolean | `false` | `--flat` |
| `include` | list | — | `--include` |
| `exclude` | list | — | `--exclude` |
| `include_archived` | boolean | `false` | `--archived` |
| `include_forks` | boolean | `false` | `--forks` |
| `insecure` | boolean | `false` | `--insecure` |
| `timeout` | integer | `30` | `--timeout` |
| `strip_prefix` | boolean | `true` | drop the namespace prefix from the local path |
| `token` | string | — | `--token` (prefer `token_env`/`token_cmd`) |
| `token_env` | string | — | name of the variable holding the token |
| `token_cmd` | string | — | command that prints the token, e.g. `op read op://Private/GitLab/token` |

A few keys accept friendlier aliases: `group`/`groups`/`org`/`orgs` for `namespaces`, and `strip_namespace_prefix` for `strip_prefix`. An unrecognised key is an error rather than a silent no-op, so a typo tells you immediately.

The config file lives at `~/.config/repoclone/config.toml`, or `$XDG_CONFIG_HOME/repoclone/config.toml` if that is set. `REPOCLONE_CONFIG=/path/to/config.toml` overrides both — handy for a repository-local config checked in with your team.

## Environment variables

Useful when you cannot pass flags — CI, cron, a shared shell profile. Every one of these is overridden by the matching command-line flag.

| Variable | Sets | Also accepted |
|---|---|---|
| `REPOCLONE_TOKEN` | the token | `GITLAB_TOKEN` / `GITHUB_TOKEN`, by provider |
| `REPOCLONE_PROVIDER` | `--provider` | — |
| `REPOCLONE_HOST` | `--host` | `GITLAB_HOST`, `GITHUB_HOST` |
| `REPOCLONE_NAMESPACE` | the namespaces, comma-separated | `GITLAB_GROUP`, `GITHUB_ORG` |
| `REPOCLONE_DEST` | `--dest` | `REPOSITORIES_ROOT_DIR` |
| `REPOCLONE_CONCURRENCY` | `--jobs` | — |
| `REPOCLONE_TIMEOUT` | `--timeout` | — |
| `REPOCLONE_CONFIG` | path to the config file | `XDG_CONFIG_HOME` for the directory |

```bash
export REPOCLONE_HOST=https://git.tech.example.com
export REPOCLONE_NAMESPACE=platform/tooling,platform/data
export REPOCLONE_DEST=~/src/work
export GITLAB_TOKEN=glpat-xxxx

repoclone clone            # no flags needed at all
```

**Precedence:** command-line flags → environment variables → profile → built-in defaults. A flag you do not pass never overwrites your profile.

## How it decides what to do

| Situation | What happens |
|---|---|
| Directory missing | Clone it |
| Clone exists, clean, behind | Fast-forward |
| Clone exists, clean, up to date | Nothing (`unchanged`) |
| Clone exists, **uncommitted changes** | Fetch only, working tree untouched |
| Clone exists, **diverged** | Fetch only, reported as needing a manual merge |
| Directory exists but is not a repository | Skipped, never overwritten |
| Repository is empty | Skipped |

`repoclone` never runs `git merge` unless it can fast-forward, never force-updates, and never deletes anything. `--prune` only *reports* orphans — removing them stays your decision.

## Output formats

`list` and `report` accept `--output table|json|csv`, so results pipe into other tools:

```bash
repoclone list platform --token $T --output json | jq -r '.[].repository'
repoclone report --output csv > inventory.csv
```

## Exit codes

| Code | Meaning |
|---|---|
| `0` | Success |
| `1` | At least one repository failed, or configuration was invalid |
| `2` | Bad command-line usage (help is printed) |

## Documentation

- [docs/usage.md](docs/usage.md) — task-by-task guide with worked examples
  - [Step-by-step first test](docs/usage.md#step-by-step-first-test) — start here if you have just installed it
- [docs/architecture.md](docs/architecture.md) — how the pieces fit together
- [CONTRIBUTING.md](CONTRIBUTING.md) — how to add a command, a provider or an output format
- [docs/releasing.md](docs/releasing.md) — publishing to PyPI, versioning, trusted publishing

## Licence

MIT.
