Metadata-Version: 2.4
Name: gitlab-proxmox-runner
Version: 0.0.1
Summary: GitLab Runner Custom Executor that provisions a Proxmox VM and runs the job script inside it
Author-email: "V. Vorobev" <vorobev.github@pm.me>
License: MIT
Project-URL: Homepage, https://gitlab.com/vorobjovvictor/gitlab-runner-pve
Project-URL: Repository, https://gitlab.com/vorobjovvictor/gitlab-runner-pve
Project-URL: Issues, https://gitlab.com/vorobjovvictor/gitlab-runner-pve/-/issues
Keywords: gitlab,gitlab-runner,proxmox,ci,custom-executor
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: System :: Systems Administration
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: proxmoxer
Requires-Dist: requests
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov>=4; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Dynamic: license-file

# gitlab-proxmox-runner

A GitLab Runner [Custom Executor](https://docs.gitlab.com/runner/executors/custom.html)
driver that provisions a fresh Proxmox VM for each CI job and runs the job's
script **inside** that VM. CI variables (`CI_JOB_ID`, `CI_COMMIT_SHA`, …) are
visible to user scripts the same way the docker executor exposes them.

The package ships four console scripts: `vm_config_exec`, `vm_prepare_exec`,
`vm_run_exec`, `vm_cleanup_exec`. They map 1:1 to the four hooks of the
gitlab-runner custom executor protocol.

## How it works

Per CI job, gitlab-runner invokes the four entry points this package installs:

| Entry point       | Stage         | Responsibility                                                                |
|-------------------|---------------|-------------------------------------------------------------------------------|
| `vm_config_exec`  | once, first   | Emit driver/build dirs/job_env JSON; resolves the requested template ID.      |
| `vm_prepare_exec` | once          | Clone the Proxmox template, boot the VM, wait for the QEMU guest agent, save `{vmid, ip}` state. |
| `vm_run_exec`     | per stage     | `scp` the runner-generated stage script to the VM, then `ssh <user>@vm sudo /bin/bash …`. Runs `prepare_script`, `get_sources`, `build_script`, `upload_artifacts_*`, etc., all in the VM. |
| `vm_cleanup_exec` | once, last    | Best-effort stop + destroy of the VM, remove state file.                       |

Per-job state is persisted to `/tmp/gitlab-proxmox-runner-${CI_JOB_ID}.json`
so each stage can find the VM that was provisioned in `vm_prepare_exec`.

## Templates

Pick one via `PROXMOX_TEMPLATE` (job-level variable):

| Template name    | Proxmox VMID |
|------------------|--------------|
| `ubuntu-focal`   | 60900        |
| `ubuntu-bionic`  | 60910        |
| `debian-11`      | 60920        |
| `debian-12`      | 60930        |

> **Prerequisite:** these template VMs **must already exist on the Proxmox
> server** at the matching VMIDs before any CI job runs. The driver only
> *clones* an existing template — it does not create one. The VMID column
> above is the ID the driver looks up in `pve.<node>.qemu/<vmid>`. If you use
> different VMIDs in your environment, fork this repo and edit
> `VM_Templates` in [`gitlab_proxmox_runner/__init__.py`](gitlab_proxmox_runner/__init__.py).

Each template VM must:

- have the QEMU guest agent installed and enabled,
- contain the SSH user named in `VM_SSH_USER` (default `ci`) with passwordless
  `sudo` and the runner's SSH public key in `~/.ssh/authorized_keys`,
- be marked as a template in Proxmox (`qm template <vmid>`).

## Required environment on the runner host

Set in `[[runners]].environment` (config.toml). gitlab-runner automatically
prefixes build variables with `CUSTOM_ENV_` when handing them to custom
executor scripts, so write the names below **without** the prefix in
config.toml — the executor reads them as `CUSTOM_ENV_<NAME>`.

| Variable in config.toml | Read by executor as              | Purpose                                          |
|-------------------------|----------------------------------|--------------------------------------------------|
| `PROXMOX_HOST`          | `CUSTOM_ENV_PROXMOX_HOST`        | Proxmox API host                                 |
| `PROXMOX_USER`          | `CUSTOM_ENV_PROXMOX_USER`        | Proxmox user (e.g. `root@pam`)                   |
| `PROXMOX_TOKEN_NAME`    | `CUSTOM_ENV_PROXMOX_TOKEN_NAME`  | Proxmox API token name                           |
| `PROXMOX_TOKEN_VALUE`   | `CUSTOM_ENV_PROXMOX_TOKEN_VALUE` | Proxmox API token value                          |
| `PROXMOX_NODE`          | `CUSTOM_ENV_PROXMOX_NODE`        | Proxmox node name to clone on                    |
| `VM_SSH_USER`           | `CUSTOM_ENV_VM_SSH_USER`         | SSH user inside the VM (default: `ci`)           |

`VM_SSH_PRIVATE_KEY` carries the private key for `ssh`/`scp` into the VM. It
is typically a **GitLab CI variable** (instance- or project-level, File or
Variable type). gitlab-runner exposes it to the executor as
`CUSTOM_ENV_VM_SSH_PRIVATE_KEY`. The driver also accepts `VM_SSH_PRIVATE_KEY`
(no prefix) for setups where the value is exported via the runner's systemd
unit. The variable may contain either:

- the actual private-key text (the driver materialises it to a `0600` temp
  file on the runner host), or
- a filesystem path to an existing key file.

## Installation on the runner host

```sh
sudo /usr/bin/pip install -U --break-system-packages gitlab-proxmox-runner
```

`--break-system-packages` is required on Debian 12 / Python 3.12+ where the
system pip enforces [PEP 668](https://peps.python.org/pep-0668/).

The four entry points install to
`/usr/local/bin/{vm_config_exec,vm_prepare_exec,vm_run_exec,vm_cleanup_exec}`.

## Runner config

A worked `config.toml` example lives at [examples/config.toml](examples/config.toml).
Copy the relevant `[[runners]]` block onto your runner host and
`systemctl restart gitlab-runner`.

## Using it from a job

The job script is plain shell — no manual SSH/SCP plumbing:

```yaml
my-job:
  tags: [proxmox]
  variables:
    PROXMOX_TEMPLATE: ubuntu-focal
  script:
    - apt-get update && apt-get install -y build-essential
    - make
  artifacts:
    paths: [out/]
```

`get_sources`, the user `script:`, and `upload_artifacts_*` all execute inside
the VM. CI variables are exported automatically. Artifacts are uploaded by
gitlab-runner's normal flow.

## Development

```sh
python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
ruff check .
pytest -q --cov=gitlab_proxmox_runner --cov-report=term-missing
python -m build      # produces sdist + wheel under dist/
```

The unit-test suite does not require a live Proxmox instance — Proxmox API
calls and SSH/SCP subprocess calls are mocked out. End-to-end testing must be
done on a host with network access to a Proxmox cluster.

## Releasing to PyPI

CI publishes on tag pushes matching `vX.Y.Z` via
[PyPI Trusted Publishing (OIDC)](https://docs.pypi.org/trusted-publishers/).
One-time PyPI setup: at <https://pypi.org/manage/account/publishing/> add a
publisher with:

- Project name: `gitlab-proxmox-runner`
- Owner: your gitlab.com namespace
- Repository name: this repo's slug
- Workflow filename: `.gitlab-ci.yml`

Then `git tag v0.1.0 && git push --tags` triggers the `publish` job.

## License

MIT — see [LICENSE](LICENSE).
