Metadata-Version: 2.4
Name: devpi-gitea-sync
Version: 0.1.0
Summary: Sync Python packages from the Gitea package registry into Devpi indices.
License-File: LICENSE
Author: ramos
Author-email: crqdev@gmail.com
Requires-Python: >=3.13
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: click (>=8.1.0,<9.0.0)
Requires-Dist: devpi-api-client (>=1.0.0,<2.0.0)
Requires-Dist: flask (>=3.0.0,<4.0.0)
Requires-Dist: packaging (>=24.0,<25.0)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Project-URL: Documentation, https://veloslab.github.io/python-devpi-gitea-sync/
Project-URL: Repository, https://github.com/veloslab/python-devpi-gitea-sync
Description-Content-Type: text/markdown

# devpi-gitea-sync

Synchronize Python packages from the Gitea package registry into one or more Devpi
indices. The tool queries the Gitea PyPI package registry, downloads wheel and source
distribution files, and uploads them to the configured Devpi index.

The intended deployment is **daemon mode** — the tool runs continuously, polling Gitea
on a configurable interval and automatically picking up newly published packages.
One-off manual syncs are also supported for testing or recovery scenarios.

**[Full documentation](https://veloslab.github.io/python-devpi-gitea-sync/)**

## Getting started

1. Install the package (for example with `poetry install` or `pip install -e .`).
   The tool requires Python 3.13 and the dependencies declared in `pyproject.toml`.
2. Create a configuration file named `devpi-gitea-sync.conf` in your working directory.
3. On macOS/Linux, lock down the file permissions so only you can read and write it:

   ```bash
   chmod 600 devpi-gitea-sync.conf
   ```

   The CLI enforces this setting and refuses to run when the file is world or group readable.

### Minimal configuration

The simplest possible setup — one Gitea organization, one Devpi index, one token:

```ini
[gitea]
url = https://gitea.example.com
token_env = GITEA_TOKEN

[devpi]
url = https://devpi.example.com
username = devpi-user
password_env = DEVPI_PASSWORD

[mapping:my-org]
organization = my-org
index = user/dev
```

Start the server (recommended):

```bash
devpi-gitea-sync --server -v
```

Open `http://localhost:8080/` for the live dashboard. Or run a single sync to verify your
setup before committing to server mode:

```bash
devpi-gitea-sync --dry-run -v
```

See [More configuration examples](#more-configuration-examples) for multi-org, multi-server,
and other advanced setups.

### Understanding `[gitea]` vs `[gitea:<name>]` and `[devpi]` vs `[devpi:<name>]`

The bare `[gitea]` and `[devpi]` sections define the **defaults**. Named variants add
extra entries that can be referenced by name from a mapping.

**Gitea — token overrides per organization:**

| Section | Purpose |
| --- | --- |
| `[gitea]` | Base URL and the fallback token used for any org without an override |
| `[gitea:my-org]` | Replaces the token only when syncing `my-org` |

The URL is always taken from `[gitea]`; only the token is overridden per org.
See [Token resolution](#token-resolution) for the full precedence rules.

**Devpi — additional named servers:**

| Section | Purpose |
| --- | --- |
| `[devpi]` | The default server, used by any mapping that does not specify `devpi = ...` |
| `[devpi:prod]` | A second server reachable under the name `prod` |

A mapping opts into a named server with `devpi = prod`. Omit the option and it uses
`[devpi]` automatically.

Each credential is stored directly in the configuration file, so the strict `chmod 600`
requirement is essential. If you prefer to keep secrets in environment variables you can
swap any `token`/`password` option for the corresponding `token_env`/`password_env`.

If you prefer a custom filename, point the CLI at it with `--config path/to/file.conf`.

---

## How it works

### Sync flow

For each configured mapping the tool runs the following pipeline:

```
Gitea package registry (org, type=pypi)
        │
        │  list all packages + files (paginated)
        ▼
Is file a .whl, .tar.gz, or .zip?
        │ no  ──────────────────────────────────►  skip (not a Python distribution)
        │ yes
        ▼
Is repository in the allowlist?          (only checked when repositories = ... is set)
        │ no  ──────────────────────────────────►  skip
        │ yes (or no allowlist configured)
        ▼
Is package name in the package filter?   (only checked when --package-filters is set)
        │ no  ──────────────────────────────────►  skip
        │ yes (or no filter active)
        ▼
Group files by (canonical package name, version)
        │
        ▼
Does this version already exist in Devpi?
        │ yes, and --force not set  ────────────►  skip
        │ no, or --force set
        ▼
Download file to staging dir
        │
        ▼
Upload to Devpi index
        │
        ▼
Delete staged file
```

### Mapping concept

A mapping links one Gitea organization to one Devpi index. You can define as many
mappings as you need — the same organization can appear in multiple mappings to fan
packages out to several indices, or different organizations can target the same index.

```
Gitea                          Devpi
─────────────────────────────────────────────────────
org: team-alpha  ──────────►  server: default  index: alpha/packages
org: team-beta   ──────────►  server: default  index: beta/packages
                 ┌─────────►  server: default  index: user/staging
org: my-org  ───┤
                 └─────────►  server: prod      index: user/stable
```

### Token resolution

For each mapping, the Gitea token is resolved in order of precedence — the first match wins:

```
[mapping:<name>] token / token_env     (most specific)
        │  not set?
        ▼
[gitea:<organization>] token / token_env
        │  not set?
        ▼
[gitea] token / token_env              (global default)
        │  not set?
        ▼
    error — mapping is skipped
```

This lets you set a single root token for most organisations while selectively overriding
it for organisations that require their own credentials.

---

## More configuration examples

The examples below build from simple to complex. Start with the one closest to your setup.

### Secrets via environment variables

Keep credentials out of the config file by referencing environment variables instead:

```ini
[gitea]
url = https://gitea.example.com
token_env = GITEA_TOKEN

[devpi]
url = https://devpi.example.com
username = devpi-user
password_env = DEVPI_PASSWORD

[mapping:my-org]
organization = my-org
index = user/dev
```

### Single Gitea instance, root token, multiple orgs into one Devpi index

The simplest multi-team setup: one root token covers all organizations, and every
mapping targets the same Devpi server and index.

```
Gitea (single instance, root token)
├── org: team-alpha  ─────┐
├── org: team-beta   ─────┼──►  Devpi: devpi.example.com  index: shared/packages
└── org: team-gamma  ─────┘
```

```ini
[gitea]
url = https://gitea.example.com
token_env = GITEA_ROOT_TOKEN       # one token with read access to all orgs

[devpi]
url = https://devpi.example.com
username = devpi-user
password_env = DEVPI_PASSWORD

[mapping:team-alpha]
organization = team-alpha
index = shared/packages

[mapping:team-beta]
organization = team-beta
index = shared/packages

[mapping:team-gamma]
organization = team-gamma
index = shared/packages
```

Because all mappings share the same `[gitea]` root token you don't need any
`[gitea:<org>]` sections. To run continuously with the web dashboard:

```bash
devpi-gitea-sync --server -vv
```

To do a one-off dry run for a single org before committing:

```bash
devpi-gitea-sync --org team-alpha --dry-run -vv
```

### Multiple organizations, each with its own token

```ini
[gitea]
url = https://gitea.example.com

[gitea:team-alpha]
token_env = GITEA_TOKEN_ALPHA

[gitea:team-beta]
token_env = GITEA_TOKEN_BETA

[devpi]
url = https://devpi.example.com
username = devpi-user
password_env = DEVPI_PASSWORD

[mapping:team-alpha]
organization = team-alpha
index = alpha/packages

[mapping:team-beta]
organization = team-beta
index = beta/packages
repositories = core-lib, data-utils
```

Sync only one team at a time:

```bash
devpi-gitea-sync --org team-alpha
```

### One organization mirrored into multiple Devpi indices

Two mappings point at the same Gitea org but different Devpi indices — useful when
you want packages available on both a staging and a production server simultaneously:

```ini
[gitea]
url = https://gitea.example.com
token_env = GITEA_TOKEN

[devpi]
url = https://staging.devpi.example.com
username = devpi-user
password_env = DEVPI_STAGING_PASSWORD

[devpi:prod]
url = https://prod.devpi.example.com
username = devpi-user
password_env = DEVPI_PROD_PASSWORD

[mapping:my-org-staging]
organization = my-org
index = user/staging

[mapping:my-org-prod]
organization = my-org
devpi = prod
index = user/stable
```

Run only the staging mapping during development:

```bash
devpi-gitea-sync --mapping my-org-staging
```

### Repository allowlist

Restrict a mapping to specific repositories within the organization, ignoring everything else:

```ini
[mapping:selected-repos]
organization = my-org
index = user/dev
repositories = repo-a, repo-b, repo-c
```

---

## Configuration reference

### `[gitea]`

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `url` | string | required | Base URL of the Gitea instance |
| `token` | string | — | Personal access token (fallback for all orgs) |
| `token_env` | string | — | Environment variable containing the token |
| `verify_ssl` | bool or path | `true` | `true`, `false`, or a path to a CA bundle file |
| `timeout` | float | `10` | HTTP request timeout in seconds |

### `[gitea:<org>]`

Overrides the token for a specific organization. The URL is always taken from `[gitea]`.

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `token` | string | — | Token for this org (overrides `[gitea]` token) |
| `token_env` | string | — | Environment variable containing the token |

### `[devpi]` and `[devpi:<name>]`

`[devpi]` is the default server. Additional servers use `[devpi:<name>]` with the same options.

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `url` | string | required | Base URL of the Devpi server |
| `username` | string | required | Devpi username |
| `password` | string | — | Password (mutually exclusive with `token`) |
| `password_env` | string | — | Environment variable containing the password |
| `token` | string | — | Auth token (mutually exclusive with `password`) |
| `token_env` | string | — | Environment variable containing the token |
| `verify_ssl` | bool or path | `true` | `true`, `false`, or a path to a CA bundle file |
| `timeout` | float | `30` | HTTP request timeout in seconds |

### `[mapping:<name>]`

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `organization` | string | section name | Gitea organization to sync from |
| `index` | string | required | Devpi index to sync into (`user/index` format) |
| `devpi` | string | `default` | Name of the Devpi server to target |
| `token` | string | — | Per-mapping Gitea token (overrides org and global tokens) |
| `token_env` | string | — | Environment variable containing the token |
| `repositories` | string | — | Comma-separated allowlist of repositories |
| `include_archived` | bool | `false` | Whether to include archived repositories |

### `[runtime]`

Optional section for operational overrides.

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `poll_interval_seconds` | int | `300` | Seconds between sync polls in server mode (minimum: `60`) |
| `download_dir` | path | system temp | Directory used to stage downloads before uploading |
| `server_host` | string | `0.0.0.0` | Host address the web server binds to |
| `server_port` | int | `8080` | Port the web server listens on |

---

## Usage

### Server mode (recommended)

The primary way to run `devpi-gitea-sync` is as a long-running server that polls Gitea
continuously, uploads new releases, and exposes a web dashboard:

```bash
devpi-gitea-sync --server -v
```

The server binds to `0.0.0.0:8080` by default. Open `http://localhost:8080/` in your
browser to see the package dashboard. A JSON health endpoint is available at `/health`.

The poll interval defaults to 5 minutes. Override the server address and interval in the
`[runtime]` section:

```ini
[runtime]
poll_interval_seconds = 120
server_host = 127.0.0.1
server_port = 9090
```

Command-line flags take precedence over the config file values:

```bash
devpi-gitea-sync --server --host 127.0.0.1 --port 9090
```

Scope the server to specific organizations or mappings if needed:

```bash
devpi-gitea-sync --server --org my-org
devpi-gitea-sync --server --mapping my-mapping
```

Use `--dry-run` with `--server` to display the current state without uploading anything —
useful for verifying your configuration before enabling writes.

### One-off sync

Run without `--server` to perform a single sync pass and exit. Useful for testing your
configuration or triggering a manual sync:

```bash
# Dry run — discover assets without uploading anything
devpi-gitea-sync --dry-run -v

# Real sync, single pass
devpi-gitea-sync -v
```

### All options

| Option | Description |
| --- | --- |
| `-c / --config PATH` | Path to the config file. Defaults to `devpi-gitea-sync.conf`. |
| `-o / --org ORG` | Only sync the specified organization. Repeat to include more. |
| `-m / --mapping NAME` | Only sync the specified mapping. Repeat to include more. |
| `--dry-run` | List discovered assets without downloading or uploading them. |
| `--force` | Re-upload packages even when the version already exists in Devpi. |
| `--server` | Start the web dashboard and continuously sync in the background. |
| `--host HOST` | Web server bind address (overrides `[runtime] server_host`). |
| `--port PORT` | Web server port (overrides `[runtime] server_port`). |
| `-v / --verbose` | Increase log verbosity. Use `-vv` for debug level. |

### Forcing a re-upload

Add `--force` when you want to push an existing version again (for example, to recover a
corrupt upload or to republish a freshly signed artifact). The flag applies to every mapping
processed in the run, so combine it with `--org` or `--mapping` if you need to target a
specific organization or mapping.

---

## Development

Install all dependencies (including dev tools) with:

```bash
poetry install
```

The `dev` group (pytest, pytest-cov) is included by default. To install production
dependencies only — for example in a deployment — use:

```bash
poetry install --only main
```

Run the test suite with coverage:

```bash
poetry run pytest
```

Format and lint using your preferred tools.

### Publishing a release

Publishing is automated via `.github/workflows/publish.yml` and triggered by creating a
GitHub Release. The workflow runs tests and a docs build first; if everything passes it
publishes to PyPI using [Trusted Publishing](https://docs.pypi.org/trusted-publishers/)
(no API token stored in the repository).

```bash
# 1. Bump version
poetry version minor   # or patch / major

# 2. Commit, open PR, merge
git checkout -b release/v0.2.0
git add pyproject.toml && git commit -m "Release v0.2.0"
gh pr create --title "Release v0.2.0"

# 3. After merge, create the GitHub Release (triggers publish)
gh release create v0.2.0 --title "v0.2.0" --target main
```

See the [Release Guide](https://veloslab.github.io/python-devpi-gitea-sync/release-guide/)
for the one-time PyPI setup and the full checklist.

### Contract testing

The project includes contract tests that validate `GiteaClient`'s API calls against
the live OpenAPI spec of real Gitea instances. They run automatically in CI against a
matrix of Gitea versions (currently 1.23 and `latest`) on every push to `main`,
every pull request, and on a weekly schedule.

**What they check:**

- The endpoints the client calls (`/packages/{owner}`, `/packages/{owner}/{type}/{name}/{version}/files`, etc.) exist in the spec for each Gitea version
- The query parameters the client passes (`type`, `limit`, `page`) are accepted
- The response schemas contain the fields the client reads (`name`, `type`, `size`)

**Why this matters:**

Gitea ships breaking API changes across minor versions. For example, Gitea 1.24
renamed the `Size` field to `size` in the package file response
([PR #34173](https://github.com/go-gitea/gitea/pull/34173)). The contract tests
caught this automatically and the client already handled both casings defensively.

The weekly schedule means a breaking Gitea release will surface within a week rather
than when a user reports a runtime failure.

To run contract tests locally against a running Gitea instance (defaults to
`http://localhost:3000`):

```bash
poetry run pytest tests/contracts/ -v --no-cov

# Or point at a different host
GITEA_URL=http://gitea.example.com poetry run pytest tests/contracts/ -v --no-cov
```
