Metadata-Version: 2.5
Name: waldur-site-agent-rancher-kc-crd
Version: 1.0.8rc10
Summary: Rancher + Keycloak CRD-driven plugin for Waldur Site Agent
Author-email: OpenNode Team <info@opennodecloud.com>
Requires-Python: <4,>=3.9.2
Requires-Dist: kubernetes>=29.0.0
Requires-Dist: waldur-site-agent>=1.0.8rc10
Description-Content-Type: text/markdown

# waldur-site-agent-rancher-kc-crd

CRD-driven Rancher + Keycloak membership-sync plugin for Waldur Site
Agent. Translates Waldur `Resource` + `ResourceProject` + `UserRole`
state into `ManagedRancherProject` Custom Resources; the
`rancher-keycloak-operator` running inside the target Kubernetes
cluster owns the actual Rancher project + Keycloak group lifecycle.

## At a glance

| Entry point | Group | Role |
|---|---|---|
| `rancher-kc-crd` | `waldur_site_agent.backends` | membership sync only |

**Modes:** `membership_sync`, `event_process`.

| Operation | Behaviour |
|---|---|
| Create / terminate resource | Not handled — clusters pre-exist; use another backend for orders |
| Add / remove members | Writes one `ManagedRancherProject` CR per ResourceProject; the operator binds Keycloak groups |
| Pause / downscale / restore | **No-op** — returns `True` without changing anything |
| Usage reporting | **No-op** — reports nothing |

## Expected operator

This plugin only writes CRs — it relies on
[`waldur/rancher-keycloak-operator`][operator-github] to be running
in the target Kubernetes cluster. That operator owns the
`ManagedRancherProject` and `RancherProjectInventory` CRD definitions
and is the only thing that talks to Rancher and Keycloak APIs.

Minimum operator version compatible with the current plugin: **`0.3.0`**.
**Recommended: `0.3.1`+** — earlier 0.3.x bug-fixes a metadata-sync gap
where description / organization / projectSlug changes never reached
Rancher after the initial create. The plugin no longer emits
`spec.namespace`, which `0.2.x` operators relied on for namespace
creation; earlier versions also miss audit fields and the
stale-project-ID cleanup fallback that the plugin assumes.

Attribute lookup (`keycloak_user_lookup: attribute`, see
[Matching users across Keycloaks by civil code](#matching-users-across-keycloaks-by-civil-code))
needs operator **`0.5.0`+**: older CRDs reject the `lookupAttribute`
member field, so the plugin emits it only when attribute lookup is
configured.

The operator's helm chart lives in its own repo under
`helm/rancher-keycloak-operator/`; install instructions are in the
[Setup](#setup) section below.

[operator-github]: https://github.com/waldur/rancher-keycloak-operator

## Scope

**`membership_sync_backend` only.** This plugin handles user-to-project
role bindings. It does not provision or terminate Rancher clusters,
does not process orders, and does not report usage. Order processing
and reporting are out of scope; if the offering needs them they have
to be wired through other backends.

**Cluster pre-exists.** Each Waldur Resource's `backend_id` is its
Rancher cluster ID; this plugin does not stand up Rancher clusters.

**Opt-in per offering.** Set
`membership_sync_backend: rancher-kc-crd` on the offering's
site-agent config to route its membership sync through this plugin.

**Membership is per-ResourceProject, not a flat per-Resource team.**
Unlike SLURM-style backends, there's no single "team" for the whole
Resource — each ResourceProject has its own `UserRole` list, and
that's what drives Keycloak group membership (see
[User identity matching](#user-identity-matching) for exactly which
field). The site agent's generic team-vs-backend membership diff
(used by flat-team backends to detect new/stale users) doesn't apply
here and is skipped for this backend — see `skip_resource_team_diff`
in `waldur_site_agent_rancher_kc_crd/backend.py`.

---

## Architecture

```mermaid
flowchart LR
    subgraph Waldur ["Waldur Mastermind"]
        WR[Resource]
        WRP[ResourceProject]
        WUR[UserRole]
        WR --- WRP --- WUR
    end

    subgraph Agent ["Site Agent / this plugin"]
        PR[pull_resource]
        TR[translator.build_cr_spec]
        CC[CrdClient.apply / list / delete]
        SR[status_reader]
    end

    subgraph K8s ["Kubernetes cluster"]
        CR[(ManagedRancherProject CR)]
        OP[rancher-keycloak-operator]
        CR -. watched by .-> OP
    end

    subgraph External ["External APIs"]
        RA[Rancher Mgmt API]
        KC[Keycloak Admin API]
    end

    Waldur -- "SDK GET<br/>(provider-resource-projects,<br/>list_users)" --> PR
    PR --> TR --> CC -- "kube apply" --> CR
    CR --> SR
    SR -- "status.* + drift" --> Waldur

    OP -- "v3 REST" --> RA
    OP -- "Admin REST" --> KC
```

The plugin's responsibilities stop at writing CRs and reading their
status. The operator owns:

- creating / adopting Rancher projects, namespaces, and resource quotas
- creating Keycloak parent + child groups and binding them to the
  Rancher project via `ProjectRoleTemplateBinding` (PRTB) with
  `groupPrincipalId: keycloakoidc_group://<group-name>`
- adding/removing users to/from Keycloak groups based on
  `spec.keycloak.roleBindings[].members[]`
- cascading cleanup on CR delete (`@kopf.on.delete`)

---

## Sequence: end-to-end membership sync

This is what happens on a single `pull_resource` call (i.e. one
membership-sync cycle per Resource the offering owns).

```mermaid
sequenceDiagram
    autonumber
    participant SA as Site Agent
    participant PG as plugin
    participant W as Waldur API
    participant K as Kubernetes API
    participant OP as Operator
    participant R as Rancher
    participant KC as Keycloak

    SA->>PG: pull_resource(WaldurResource)
    PG->>W: GET /api/marketplace-provider-resource-projects/?resource_uuid=…
    W-->>PG: [RP1, RP2, …]
    loop per ResourceProject
        PG->>W: GET /api/marketplace-provider-resource-projects/<uuid>/list_users/
        W-->>PG: [UserRole, …]
        PG->>PG: build_cr_spec(resource, RP, users)
        PG->>K: kube apply ManagedRancherProject (server-side)
        K-->>OP: watch event (create/update)
        OP->>R: ensure project (create or adopt by name)
        OP->>R: ensure namespace + ResourceQuota
        OP->>KC: ensure parent group, child groups
        OP->>R: ensure PRTB(s) bound to KC groups
        OP->>KC: GET /users/{id} or /users?username=X<br/>(per member)
        OP->>KC: add/remove user from KC group
        OP->>K: patch status.conditions + status.keycloakRoleBindings.syncedMembers
        PG->>K: kube get ManagedRancherProject
        K-->>PG: status.* (synced members per binding)
    end
    PG->>K: list ManagedRancherProject by label waldur.io/resource-uuid
    K-->>PG: [all CRs for this resource]
    PG->>K: kube delete (CRs whose RP no longer in Waldur)
    K-->>OP: watch event (delete)
    OP->>R: delete PRTBs + project (with stale-ID find-by-name fallback)
    OP->>KC: delete child groups, parent group if empty
    PG-->>SA: BackendResourceInfo(users=union of synced members)
```

**Key invariants:**

- **Idempotent at every step.** Re-applying the same CR yields the
  same Rancher project and Keycloak groups. Re-syncing with the same
  user set is a no-op on the Rancher/Keycloak side.
- **One CR per ResourceProject**, named
  `<resource.slug>-<rp.uuid[:8]>`. Stable across renames.
- **Orphan pruning by label.** CRs are stamped with
  `metadata.labels.waldur.io/resource-uuid`; on each sync the plugin
  computes the set of expected CR names from the *current* Waldur RP
  list, then deletes any label-matching CR outside that set.
- **Lookup-only for users.** The operator never creates Keycloak
  users; it only binds existing ones. Users absent from Keycloak get
  a `WARNING User <id> not found in Keycloak` and are skipped (the
  PRTB and the group are still created — they're just empty for that
  user).

---

## Sequence: orphan-CR pruning

```mermaid
sequenceDiagram
    autonumber
    participant SA as Site Agent
    participant W as Waldur
    participant K as Kubernetes
    participant OP as Operator
    participant R as Rancher
    participant KC as Keycloak

    Note over SA: pull_resource loop completes — 2 CRs applied for RP1, RP2
    SA->>W: GET resource-projects?resource_uuid=…
    W-->>SA: [RP1] (RP2 was deleted in Waldur)
    SA->>K: list mrp -l waldur.io/resource-uuid=…
    K-->>SA: [CR_RP1, CR_RP2]
    Note over SA: expected={CR_RP1}<br>found={CR_RP1, CR_RP2}<br>orphans={CR_RP2}
    SA->>K: kube delete CR_RP2
    K-->>OP: watch event (delete + finalizer hold)
    OP->>R: delete PRTBs for RP2's project
    OP->>KC: remove members from KC groups
    OP->>KC: delete KC child groups
    OP->>KC: delete KC parent group (if empty)
    OP->>R: delete Rancher project
    alt stored project ID is stale (e.g. project recreated externally with new ID)
        OP->>R: DELETE /v3/projects/<stored_id> -> 404
        OP->>R: GET /v3/projects?clusterId=…&name=<projectName>
        R-->>OP: {id: <current_id>}
        OP->>R: DELETE /v3/projects/<current_id> -> 200
    end
    OP->>K: remove finalizer (CR fully gone)
```

**Why pruning lives in the plugin, not the operator:** the operator
doesn't know about Waldur — the Waldur RP list is the source of
truth, and only the plugin sees both sides. The label selector keeps
pruning safe: CRs without the label (manually-created, or from a
different source) are never touched.

---

## Sequence: cleanup on stale `rancherProjectId`

When a Rancher project is externally deleted and recreated between
two operator reconciles, the CR's `status.rancherProjectId` points
at a non-existent ID. The cleanup falls back to finding the live
project by `clusterId+projectName`:

```mermaid
sequenceDiagram
    autonumber
    participant U as User / Test
    participant K as Kubernetes
    participant OP as Operator
    participant R as Rancher

    Note over U: status.rancherProjectId = p-OLD<br>(actual Rancher project: p-NEW, same name)
    U->>K: kubectl delete mrp <name>
    K-->>OP: on_delete handler fires
    OP->>R: DELETE /v3/projects/p-OLD
    R-->>OP: 404 Not Found
    Note over OP: delete_project returned False<br>-> stored ID is stale
    OP->>R: GET /v3/projects?clusterId=…&name=<projectName>
    R-->>OP: [{id: p-NEW, …}]
    OP->>OP: log WARNING — stored projectId p-OLD was stale, deleting current p-NEW found by name
    OP->>R: DELETE /v3/projects/p-NEW
    R-->>OP: 200 OK
    OP->>K: cleanup complete — release finalizer
```

This was the failure mode that left orphan Rancher projects after
external recreation; fix landed in operator `0.2.2`.

---

## Setup

### 1. Operator: install in target cluster

The [`waldur/rancher-keycloak-operator`][operator-github] must be
running in the cluster you point this plugin at. One operator
instance handles **all** `ManagedRancherProject` CRs in its
namespace; one operator can manage Rancher projects across multiple
downstream Rancher clusters (each CR specifies its own `clusterId`).

#### 1a. CRDs

```bash
# Clone the operator repo (separate from waldur-site-agent):
git clone https://github.com/waldur/rancher-keycloak-operator.git
cd rancher-keycloak-operator

kubectl apply -f helm/rancher-keycloak-operator/templates/crds/
kubectl get crds | grep waldur.io
# expect:
#   managedrancherprojects.waldur.io
#   rancherprojectinventories.waldur.io
```

#### 1b. Helm install (published image)

The published image is `opennode/rancher-keycloak-operator:<version>`
on Docker Hub. Pin a specific version (don't use `:latest`):

```bash
kubectl create namespace waldur-system

helm upgrade --install rko \
  ./helm/rancher-keycloak-operator \
  --namespace waldur-system \
  --set image.repository=opennode/rancher-keycloak-operator \
  --set image.tag=0.3.1 \
  --set image.pullPolicy=IfNotPresent \
  --set "config.rancher.url=https://rancher.example.com" \
  --set "config.rancher.bearerToken=<rancher-bearer-token>" \
  --set "config.rancher.verifySsl=true" \
  --set "config.keycloak.url=https://keycloak.example.com" \
  --set "config.keycloak.realm=<realm>" \
  --set "config.keycloak.userRealm=master" \
  --set "config.keycloak.username=<kc-admin-user>" \
  --set "config.keycloak.password=<kc-admin-password>" \
  --set "config.keycloak.verifySsl=true"

kubectl rollout status deploy/rko-rancher-keycloak-operator -n waldur-system
```

#### 1c. Smoke-test the operator (optional but recommended)

The [operator repo][operator-github] ships a Tier-1 runbook at
`docs/tier-1-runbook.md` that walks through applying a hand-crafted
CR end-to-end against the configured Rancher and Keycloak before
wiring in the site-agent. Run it once per cluster to catch
credential / connectivity issues early.

#### 1d. Required Rancher + Keycloak permissions

| System | Role / scope |
|---|---|
| Rancher token | unscoped admin OR cluster-owner across all clusters this operator instance will manage |
| Keycloak admin user | realm-admin on the target realm (group create/delete, group member add/remove, user lookup) |

---

### 2. Plugin: install on the site-agent host

The plugin is a workspace member of the
`waldur-site-agent` repo and is installed automatically when you run
`uv sync --all-packages` at the repo root. To verify it's discovered:

```bash
uv run python -c "from waldur_site_agent_rancher_kc_crd import backend; print(backend.RancherKcCrdBackend)"
```

#### 2a. Site-agent host needs

- Network access to: Waldur Mastermind API, the Kubernetes API of the
  cluster running the operator.
- A kubeconfig file (or in-cluster service-account credentials if
  you run the agent inside the operator's cluster).
- The Kubernetes API user must be allowed to
  `get/list/create/update/delete/patch` `managedrancherprojects.waldur.io`
  in the chosen namespace.

#### 2b. Configure the offering

Add a stanza to `waldur-site-agent-config.yaml`. Full reference at
[examples/rancher-kc-crd-config.yaml](examples/rancher-kc-crd-config.yaml).

```yaml
offerings:
  - name: "my-rancher-offering"

    waldur_api_url: "https://waldur.example.com/api/"
    waldur_api_token: "${WALDUR_API_TOKEN}"
    waldur_offering_uuid: "<offering-uuid>"

    backend_type: "rancher-kc-crd"
    membership_sync_backend: "rancher-kc-crd"

    backend_settings:
      # Repeats the top-level waldur_api_url/waldur_api_token on purpose.
      # This plugin fetches ResourceProjects and UserRoles directly from
      # Waldur inside pull_resource, but backends aren't handed the
      # framework's own Waldur client -- that's deliberately scoped to
      # the processor, not backends in general. So the plugin builds its
      # own AuthenticatedClient from these settings instead. Most other
      # backends don't need this because they don't talk to the Waldur
      # API themselves.
      waldur_api_url: "https://waldur.example.com/api/"
      waldur_api_token: "${WALDUR_API_TOKEN}"
      waldur_verify_ssl: true

      # Where the operator listens.
      kubeconfig_path: "~/.kube/config"   # or omit for in-cluster
      context: "my-cluster"
      namespace: "waldur-system"

      # Cluster ID comes from each Resource's `backend_id` (1:1 with
      # a Rancher downstream cluster). There is no offering-level
      # cluster_id setting -- if a resource's backend_id is empty,
      # CR build raises a clear error rather than emitting an
      # invalid CR.

      # Keycloak group naming -- see note A below for the full variable
      # list, including ${customer_slug}, ${project_slug}, ${resource_slug},
      # and ${rp_uuid_short}. The default keeps the names compact and
      # immutable; override `group_name_template` for human-readable
      # group names.
      parent_group_name: "c_${cluster_id}"
      group_name_template: "c_${cluster_id}_${rp_uuid}_${role_name}"  # default

      # Map Waldur offering role names -> Rancher role template IDs.
      # Roles absent from this map are skipped (operator can't bind them).
      role_map:
        project_member: "project-member"
        project_admin: "project-owner"
        create_ns: "create-ns"

      # User-identity link to Keycloak. See "User identity matching" below.
      keycloak_user_identity_source: username  # default
      keycloak_user_lookup: username           # default
```

#### 2c. Run

`rancher-kc-crd` is a `membership_sync_backend` only. Order
processing and reporting still need other backends (or none).

```bash
uv run waldur_site_agent --mode membership_sync \
    --config-file waldur-site-agent-config.yaml
```

Observability:

```bash
# Watch CRs being created/updated/deleted
kubectl get mrp -n waldur-system -L waldur.io/resource-uuid -w

# Inspect any CR's reconciliation state
kubectl describe mrp <name> -n waldur-system

# Operator logs (cleanup, member sync, drift)
kubectl logs deploy/rko-rancher-keycloak-operator -n waldur-system --tail=100 -f
```

---

## Importing access from existing clusters

The sync is one-way: Waldur is the source of truth, and the agent never reads the bindings a
cluster already has. Pointed at a cluster that is already in use, it adopts each Rancher project
whose name matches a ResourceProject and grants only what Waldur knows. Existing project and
cluster role bindings keep working, because the operator only touches bindings for its own
groups, but Waldur doesn't show them. Import them **before the agent's first sync** with
`waldur-rancher-kc-crd-import`.

This assumes the offering's Resources already exist in Waldur with `backend_id` set to the
Rancher cluster ID. The command reads the agent config for the offering, `role_map`,
`cluster_role_map` and the identity settings, and takes the Rancher and Keycloak credentials from
the environment variables the operator uses:

```bash
export RANCHER_URL=https://rancher.example.com RANCHER_BEARER_TOKEN=token-...
export KEYCLOAK_URL=https://keycloak.example.com KEYCLOAK_REALM=rancher \
       KEYCLOAK_USERNAME=... KEYCLOAK_PASSWORD=...   # also KEYCLOAK_USER_REALM, *_VERIFY_SSL
export WALDUR_API_TOKEN=...   # a staff token; see below

# 1. Read every cluster of the offering: projects, PRTBs, CRTBs, and the Keycloak
#    users behind them (group principals expand to the group's direct members).
uv run waldur-rancher-kc-crd-import inventory -c waldur-site-agent-config.yaml \
    -o inventory.json

# 2. Review the plan (dry run), then apply it.
uv run waldur-rancher-kc-crd-import apply -c waldur-site-agent-config.yaml -i inventory.json
uv run waldur-rancher-kc-crd-import apply -c waldur-site-agent-config.yaml -i inventory.json \
    --execute --report import-report.json
```

To try the import on part of the offering first, narrow both commands with the same filters
(each can be repeated):

| Filter | Matches |
|--------|---------|
| `--cluster c-m-abc12345` | Rancher cluster ID (`Resource.backend_id`) |
| `--resource <uuid-or-name>` | Waldur Resource |
| `--project <name-or-id>` | Rancher project, by name, `c-m-…:p-…` ID or bare `p-…` ID |

`--project` imports project-scope bindings only. Cluster role bindings don't belong to any one
project, so they are left out.

```bash
uv run waldur-rancher-kc-crd-import inventory -c waldur-site-agent-config.yaml \
    --cluster c-m-abc12345 -o inventory-c-m-abc12345.json
uv run waldur-rancher-kc-crd-import apply -c waldur-site-agent-config.yaml \
    -i inventory-c-m-abc12345.json --project analytics
```

`apply` creates a ResourceProject for each Rancher project, named exactly like the project so
that the operator adopts it, and records the Rancher project ID as its `backend_id`. It then
grants each user the Waldur role that maps back to their Rancher role template: `role_map` for
project bindings, `cluster_role_map` for cluster bindings. It can be re-run safely: existing
ResourceProjects and roles are left as they are, and a missing `backend_id` is filled in.

The Waldur token must be a **staff token**. The import lists the offering's resources as their
provider, records backend IDs through the provider API, and looks users up across organizations;
neither the agent's own token nor an organization owner's token can do all three.

**Quotas.** The operator replaces a project's Rancher quota with the ResourceProject's `limits`
on every sync, so a ResourceProject without limits clears the quota. `apply` therefore copies
each project's `resourceQuota.limit` into the new ResourceProject: `limitsCpu` becomes `cpu`
(millicores), `limitsMemory` becomes `memory`, and `requestsStorage` becomes `storage`. Other quota
keys (pods, `requests.cpu`, and so on) have no Waldur equivalent. The plan flags them with
`WARN`, because the first sync removes them. It also flags an existing ResourceProject that has
no limits while its Rancher project has a quota. The offering must accept these limits, or the
creation fails and is reported.

The plan lists everything it skips, with the reason:

- the `Default` and `System` projects (`--include-system` imports them);
- a Rancher project already linked, by `backend_id`, to a ResourceProject with a different
  name: the operator adopts by name, so a second ResourceProject would put two CRs on one
  project;
- several projects on one cluster sharing a name, because the operator would adopt an arbitrary
  one of them;
- role templates that no Waldur role maps to, and Waldur roles the offering doesn't define;
- principals that aren't Keycloak OIDC users or groups (local Rancher users, other auth
  providers);
- Keycloak users with no matching active Waldur user under the configured identity settings,
  for example a user who has never logged in to Waldur.

Rancher makes whoever creates a project its `project-owner`. Projects created by one admin
therefore all carry that admin's binding, and the import grants the matching Waldur role on every
one of them if the admin has a Waldur account. Drop those entries from the inventory if that
isn't wanted.

The inventory is a plain JSON file, so you can drop entries from it before running `apply`. It
is written with mode `0600`, because it names users and, in civil-code mode, contains civil
codes.

After the agent's first sync, confirm the CRs are `Ready` and that users have access through the
new groups. Then delete the old bindings in Rancher; the operator never removes them. Keep in
mind that an adopted project is treated like any other: **removing its ResourceProject from
Waldur deletes the Rancher project**.

---

## User identity matching

Each CR member is `{userIdentifier, lookupByID[, lookupAttribute]}`.
Two settings decide what goes in it:

- `keycloak_user_identity_source` — the Waldur field that supplies
  `userIdentifier`: `username` (default), `uuid`, or `civil_number`.
- `keycloak_user_lookup` — how the operator finds that value in the
  Rancher Keycloak: `username` (default), `id` (Keycloak user ID), or
  `attribute` (a user attribute named by `keycloak_lookup_attribute`).

`keycloak_use_user_id: true` is the deprecated spelling of
`source: uuid` + `lookup: id`; it still works (with a warning in the
agent log) and produces the same CRs as before. Combining it with
conflicting new settings is rejected.

**Username (default) — `source: username`, `lookup: username`**
: Plugin sends `UserRole.user_username`. Operator does
  `GET /admin/realms/<realm>/users?username=X&exact=true` and uses
  the resulting `user.id` for group membership operations. Works in
  both OIDC and self-hosted Waldur as long as Waldur usernames
  align with Keycloak usernames — the typical OIDC mapping does this
  via the `preferred_username` claim.

**UUID — `source: uuid`, `lookup: id`**
: Plugin sends `UserRole.user_uuid`. Operator does
  `GET /admin/realms/<realm>/users/{uuid}` (matches the Keycloak
  internal `user.id`). Use this only when Waldur was OIDC-provisioned
  AND its user UUIDs were seeded from the Keycloak `sub` claim, so
  that `Waldur.user.uuid == Keycloak.user.id`. The username path is
  preferred because it tolerates UUID divergence and works in more
  topologies.

**This is not `OfferingUser.username`.** SLURM-style backends key
membership off `OfferingUser.username` — a per-offering, often
auto-generated identifier (e.g. a POSIX account name like
`test-rancher-00`) meant for the *target system*, not Keycloak. This
plugin never looks at that field. Its membership unit is each
ResourceProject's `UserRole`, and the identity it hands to the
operator is `UserRole.user_username` (or `user_uuid`) — the person's
actual Waldur account identity. If a user was added to Keycloak under
a name that doesn't match their real Waldur username, the operator
logs `WARNING User <id> not found in Keycloak` and silently skips
them (see Troubleshooting) even though everything else — CR, group,
PRTB — looks fully synced.

**The operator never creates users.** A user that doesn't exist in
Keycloak under the chosen identifier gets logged as
`WARNING User <id> not found in Keycloak` and is skipped — the
PRTB and the group are still created and bound, the user just isn't
a member yet. They become a member on the next reconcile after the
user appears in Keycloak (e.g. their first OIDC login).

**Civil code — `source: civil_number`**
: For a Rancher Keycloak that is not the one Waldur logs in with; see
  the next section.

The CR identifier and the identity reported back to Waldur are kept
apart: the membership sync report is keyed by `username` in username
mode and by `user_uuid` in the uuid and civil-number modes.

---

## Matching users across Keycloaks by civil code

When Waldur and Rancher authenticate against **different** Keycloaks,
Waldur usernames and UUIDs mean nothing on the Rancher side. What both
sides share is the person's civil code (personal identification code),
which Waldur stores as `civil_number`. Set
`keycloak_user_identity_source: civil_number` and pick the lookup that
matches how the Rancher Keycloak stores the code.

| Key | Values (default first) | Meaning |
|---|---|---|
| `keycloak_user_identity_source` | `username`, `uuid`, `civil_number` | Waldur field that becomes `userIdentifier` |
| `keycloak_user_lookup` | `username`, `id`, `attribute` | How the operator looks it up; `id` needs source `uuid` |
| `keycloak_lookup_attribute` | string | Attribute to match; required with (only with) lookup `attribute` |
| `keycloak_user_identity_template` | `${value}`, e.g. `EE${value}` | Rendered around the source value |
| `keycloak_user_identity_lowercase` | `true` for lookup `username`, else `false` | Lowercase the result |

**Layout 1 — the username is the civil code.** E.g. a Keycloak brokering
TARA with the `sub` claim (`EE38001010000`) mapped to the username.
Use `keycloak_user_lookup: username` (the default) and a template for
any country prefix:

```yaml
keycloak_user_identity_source: civil_number
keycloak_user_identity_template: "EE${value}"
# keycloak_user_lookup: username   (default)
```

Keycloak stores usernames lowercased and the operator compares the spec
string with the stored username, so the identifier is lowercased by
default (`ee38001010000`). A mixed-case value would be added and
removed again on every reconcile.

**Layout 2 — the civil code is a user attribute.** Use attribute lookup
and name the attribute. Requires operator **`0.5.0`+**. The attribute
must be declared in the realm's user profile with **edit permission
`admin` only**, or filled by an identity-provider or LDAP mapper.
Attribute lookup gives the grant to whoever holds the value, so a
user-editable attribute would let anyone copy someone else's civil code
into their own profile. The operator refuses attribute lookups in that
case and logs an error:

```yaml
keycloak_user_identity_source: civil_number
keycloak_user_lookup: attribute
keycloak_lookup_attribute: personalCode
# keycloak_user_identity_template: "EE${value}"   if the stored value is prefixed
```

The value is matched verbatim (no lowercasing unless
`keycloak_user_identity_lowercase: true`), and `syncedMembers` in the CR
status echoes it.

**The offering must expose the civil code.** Waldur includes
`user_civil_number` in the `list_users` responses the plugin reads only
when the offering's user attribute config has `expose_civil_number`
enabled. Without it no member can be matched: the plugin logs a warning
each cycle naming the setting, and every grant is reported as
`missing_in_idp` with "Civil code not available for this user". A user
whose civil code is empty is skipped the same way, individually.

**Privacy.** Civil codes never leave the agent towards Waldur — sync
reports are keyed by `user_uuid` and the plugin maps confirmed members
back to UUIDs — and agent logs show them masked (`3800…00`). They are,
however, stored **in plaintext** in each CR's `spec` and `status`
(`userIdentifier`, `syncedMembers`). Keep `get`/`list`/`watch` on
`managedrancherprojects` in the CR namespace restricted to the agent,
the operator and cluster administrators.

---

## Configuration reference

`waldur_api_url`/`waldur_api_token` below intentionally repeat the
offering's top-level fields of the same name — see the setup example
above for why.

| Key | Type | Required | Description |
|---|---|---|---|
| `waldur_api_url` | string | no | Mastermind API root with `/api/`; see **note B**. |
| `waldur_api_token` | string | no | Long-lived token from `/api/users/<uuid>/keys/`. Don't use a session token. |
| `waldur_verify_ssl` | bool | no (default `true`) | TLS verify for Waldur calls. |
| `kubeconfig_path` | string | no | Path to a kubeconfig file. Omit to use in-cluster credentials. |
| `context` | string | no | kubeconfig context to use when `kubeconfig_path` is set. |
| `namespace` | string | no (default `waldur-system`) | Namespace for `ManagedRancherProject` CRs. |
| `parent_group_name` | string | no | Top-level KC group; var `${cluster_id}`. Default `c_${cluster_id}`. |
| `group_name_template` | string | no | Per-role child KC group, vars in **note A**. |
| `role_map` | dict | no (default `{}`) | Waldur role → Rancher project role template ID; see **note C**. |
| `cluster_role_map` | dict | no | Resource-level role → Rancher cluster role template ID; see **note C**. |
| `cluster_group_name_template` | string | no | KC group per cluster role. |
| `keycloak_user_identity_source` | string | no (default `username`) | `username`, `uuid` or `civil_number`. |
| `keycloak_user_lookup` | string | no (default `username`) | `username`, `id` or `attribute`. |
| `keycloak_lookup_attribute` | string | with lookup `attribute` | User attribute to match (operator `0.5.0`+). |
| `keycloak_user_identity_template` | string | no (default `${value}`) | E.g. `EE${value}`. |
| `keycloak_user_identity_lowercase` | bool | no | Default `true` for lookup `username`, `false` otherwise. |
| `keycloak_use_user_id` | bool | no | **Deprecated**: `true` = source `uuid` + lookup `id`. |

Defaults: `group_name_template` is `c_${cluster_id}_${rp_uuid}_${role_name}`,
`cluster_group_name_template` is `c_${cluster_id}_cluster_${role_name}`.

**Note B — Waldur API credentials.** `waldur_api_url` / `waldur_api_token` are optional to
the loader, but without them `pull_resource` does nothing and membership is never synced.
The plugin strips a trailing `/api` for the SDK; use a long-lived token, not a session token.

**Note C — role maps.** Roles outside `role_map` are skipped, so an empty map binds nobody.
When `cluster_role_map` is set, the Resource-level user roles are written to every CR's
`spec.cluster` block for cluster-wide bindings; when it is unset there are no cluster bindings.

The settings are validated by
`waldur_site_agent_rancher_kc_crd.schemas.RancherKcCrdBackendSettingsSchema`; a misspelt
key is logged as a warning when the agent loads its configuration. An invalid member-identity
combination (e.g. source `civil_number` with lookup `id`) is logged the same way and also stops
the backend from starting, since it would grant access to nobody or to the wrong users.

`spec.clusterId` is resolved from each Resource's `backend_id` (1:1
with a Rancher cluster) — there is no offering-level `cluster_id`
setting. An empty `backend_id` raises a clear `KeyError` rather than
silently emitting an invalid CR.

**Note A — `group_name_template` variables.** Available substitutions:

| Variable | Source | Notes |
|---|---|---|
| `${cluster_id}` | `Resource.backend_id` | Rancher cluster ID, opaque |
| `${role_name}` | UserRole.role_name | Pre-mapping (before `role_map`) |
| `${rp_uuid}` | `ResourceProject.uuid` | Full 32-char hex |
| `${rp_uuid_short}` | first 8 chars of `${rp_uuid}` | ~4B combos, collision-free; same as `cr_name` |
| `${customer_slug}` | `Resource.customer_slug` | Waldur Customer (organization) slug |
| `${project_slug}` | `Resource.project_slug` | Waldur Project slug (parent Project; RPs have no slug) |
| `${resource_slug}` | `Resource.slug` | Waldur Resource slug (1:1 with cluster) |
| `${project_name}` | `ResourceProject.name` | Human-readable; may contain spaces |

Default is `c_${cluster_id}_${rp_uuid}_${role_name}` — one Keycloak group
per (cluster × project × role), matching Rancher's per-project-PRTB
access model. The default uses `${rp_uuid}` (immutable) for stability;
override only if you have a strong reason.

**Recommended human-readable opt-in template:**

```text
c_${cluster_id}_${customer_slug}_${project_slug}_${rp_uuid_short}_${role_name}
```

Renders e.g. `c_c-m-glwxdksp_hpc-demo-org_genomics-2026_8706dd1a_project_member`.
Stays unique per RP via `${rp_uuid_short}` while the slugs make the
group name self-explaining in the Keycloak admin UI.

**Custom-template constraints.**

1. MUST include a per-project discriminator (`${rp_uuid}`,
   `${rp_uuid_short}`, or `${project_name}`); without it, multiple
   projects share one group and a user added to project A also gains
   access to B, C, … via the shared group's PRTBs.
2. Slugs (`customer_slug`, `project_slug`, `resource_slug`,
   `project_name`) are **mutable** -- renaming the entity in Waldur
   creates a new Keycloak group on the next reconcile and orphans the
   old one (the operator adopts groups by name and never renames adopted
   groups). Memberships in the old group become stale.
3. **Switching the template after deployment** has the same effect as a
   bulk rename: every existing CR re-renders, the operator creates fresh
   groups, the old groups linger with their stale members. Plan a
   one-time manual migration if you change the template against an
   existing deployment.
4. Keycloak's `GROUP.NAME` column is `varchar(255)`. The plugin guards
   this at render time and raises a `ValueError` (with a hint about
   `${rp_uuid_short}`) if the rendered name would exceed 255 chars, so
   the operator never tries to apply a CR that Keycloak would reject
   with HTTP 500. Stay well under by preferring `${rp_uuid_short}`
   over `${rp_uuid}` in long templates and keeping Waldur slugs
   reasonably short (say ≤ 50 chars each).

---

## Troubleshooting

**Plugin logs `HTTP/1.1 401 Unauthorized` from Waldur on every iteration**
: `waldur_api_token` is a session token from `/api-auth/password/` (rotates on each call). Use a long-lived
  API token from `/api/users/<uuid>/keys/`.

**`pull_resource` succeeds but no CRs are created**
: The resource has an empty `backend_id`. The agent's resource fetcher
  (`waldur_site_agent/common/processors.py:_get_waldur_resources`) drops resources without one. Fix:
  `POST /api/marketplace-provider-resources/<uuid>/set_backend_id/`.

**Plugin logs `IndexError: list index out of range` in processor `__init__`**
: Customer is not registered as a service provider. Fix: `POST /api/marketplace-service-providers/`
  with the `customer` URL.

**Plugin logs `GET .../api/api/marketplace-provider-resource-projects/...` (doubled `/api/`)**
: You're on a build older than the URL fix bundled with the orphan-pruning commit. Pull latest plugin code,
  or as a workaround drop the trailing `/api/` from `waldur_api_url`.

**Operator logs `WARNING User X not found in Keycloak` for every user**
: Identity mismatch — the chosen identifier (username by default; UUID or civil code per
  `keycloak_user_identity_source`) isn't resolvable in Keycloak. With the default username path: ensure Waldur
  usernames map to existing Keycloak usernames. With the UUID path: align Waldur user UUIDs with the Keycloak
  OIDC `sub`. With civil codes: check the template (country prefix) and, for attribute lookup, that the
  attribute is declared in the realm user profile with edit permission `admin` only (the operator
  logs `users can edit their own attribute` and refuses lookups otherwise).

**`kubectl delete mrp` succeeds in 0s but the Rancher project remains**
: `status.rancherProjectId` is stale; operator versions before `0.2.2` treated 404 on delete as success.
  Upgrade the operator to `0.2.2`+ — cleanup now falls back to find-by-name.

Operator `status.conditions` to check for any CR (`kubectl describe mrp <name>`):

| Condition | What `status=False` means |
|---|---|
| `RancherProjectReady` | Rancher create/adopt failed. Check operator log for `httpx.HTTPStatusError`. |
| `ResourceQuotaReady` | `spec.resourceQuotas` apply failed (only emitted by operator 0.3.0+). |
| `KeycloakGroupsReady` | Couldn't create/find parent or child KC groups. Check KC admin credentials. |
| `RancherBindingsReady` | PRTB creation failed — usually invalid `rancherRole` in `role_map`. |
| `MembershipSynced` | Per-user add/remove failed — see `User X not found in Keycloak` warnings. |

---

## Development

```bash
# Workspace install
cd <repo-root>
uv sync --all-packages

# Run unit tests for this plugin only
cd plugins/rancher-kc-crd && uv run pytest tests/

# Integration tests (requires a real K8s cluster with operator + CRDs)
K8S_CRD_TEST=1 \
RANCHER_CLUSTER_ID=c-m-abc12345 \
KUBE_CONTEXT=docker-desktop \
  uv run pytest tests/test_backend_integration.py -v

# Lint + format
uvx prek run --all-files
```

`tests/` holds pure unit tests (translator, status reader, sync-report derivation,
unmapped-role warnings, settings schema) and `test_backend_integration.py`, which only
runs with `K8S_CRD_TEST=1` against a real cluster with the operator and CRDs installed.

---

## Companion components

| Component | Role |
|---|---|
| [`rancher-keycloak-operator`][operator-github] (separate repo) | Reconciles `ManagedRancherProject` CRs. |
| `ManagedRancherProject` CRD (in operator helm chart) | API surface the plugin writes to. |
