# rootcoz

> AI-powered CI failure analysis for Jenkins, Prow, and JUnit XML

---

Source: quickstart.md

# Quickstart

You want RootCoz running fast so you can move from failing CI output to a report you can review and act on. The quickest path is to start the bundled Docker stack, create an operator account, and submit one Jenkins, Prow, or JUnit XML analysis from the web UI.

## Prerequisites

- Docker with Docker Compose
- One AI credential for `claude`, `gemini`, or `cursor`
- For Jenkins: a job name, build number, and Jenkins access
- For Prow: a job name, build ID, and access to the relevant Prow/GCS location
- For JUnit XML: a `.xml` file or raw JUnit XML text

## Quick Example

```bash
    cp .env.example .env
```

```dotenv
    AI_PROVIDER=claude
    AI_MODEL=your-model-name
    ANTHROPIC_API_KEY=your-anthropic-api-key
    DEFAULT_USER_ROLE=operator
    REQUIRE_APPROVAL=false
```

```bash
    docker compose up -d
    curl http://localhost:8000/health
```

1. Open `http://localhost:8000`.
2. Click `Register`, enter a username, save the API key RootCoz shows once, then click `I've saved my key — Continue`.
3. Open `New Analysis`.
4. Choose `Upload File` or `Paste XML` for the fastest first run, or switch to `Jenkins Job` or `Prow Job`.
5. Click `Submit Analysis`.

XML submissions open the result directly. Jenkins and Prow submissions open a live status page first, then redirect to the report when analysis completes.

> **Note:** The Docker image already includes the supported AI CLIs. For a first run, you only need one working provider credential plus `AI_PROVIDER` and `AI_MODEL`.


> **Warning:** `DEFAULT_USER_ROLE=operator` and `REQUIRE_APPROVAL=false` are convenient for a local trial. For a shared deployment, keep approval enabled and manage roles deliberately. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

## Step-by-Step

1. Prepare `.env`.

   Start from `.env.example`. It already sets `AI_PROVIDER`, `AI_MODEL`, and an `ANTHROPIC_API_KEY` placeholder — replace the model and key with your own, or switch to `GEMINI_API_KEY` or `CURSOR_API_KEY`. Then add the two quickstart lines shown above.

   | If your first analysis is... | Add before startup | You can also enter it in `New Analysis` |
   | --- | --- | --- |
   | JUnit XML | nothing else | n/a |
   | Jenkins | `JENKINS_URL`, `JENKINS_USER`, `JENKINS_PASSWORD` | yes |
   | Prow | `PROW_URL`, `GCS_BUCKET` | yes |

   | AI provider | Credential variable |
   | --- | --- |
   | Claude | `ANTHROPIC_API_KEY` |
   | Gemini | `GEMINI_API_KEY` |
   | Cursor | `CURSOR_API_KEY` |

   > **Warning:** The bundled Compose file uses example Jenkins values when you do not override them. Replace those placeholders before you submit a Jenkins run, or type the correct values into the form for that specific submission.

2. Start RootCoz.

   ```bash
   docker compose up -d
   curl http://localhost:8000/health
   ```

   The web UI and API are both served from `http://localhost:8000`. If the health check succeeds, the service is up and ready.

3. Sign in with a user that can submit analyses.

   With the quickstart config above, click `Register`, choose a username, save the API key, and continue into the app. RootCoz creates an active `operator` session immediately after registration.

   > **Tip:** If you already have an operator or admin API key, use `Log in` instead of `Register`.


   > **Tip:** Leave `GitHub Token`, `Jira Email`, and `Jira Token` blank for your first run. Add them later when you are ready to create issues or link external trackers. See [Managing Your Account and Notifications](manage-account-and-notifications.html) for details.

4. Open `New Analysis` and choose the input mode that matches what you already have.

   | Choose in the UI | Required fields | Optional first-run fields | What opens after submit |
   | --- | --- | --- | --- |
   | `Upload File` | a `.xml` file | `AI Provider`, `AI Model`, tags | Result page |
   | `Paste XML` | JUnit XML content | `AI Provider`, `AI Model`, tags | Result page |
   | `Jenkins Job` | `Job Name`, `Build Number` | `Jenkins URL`, `Jenkins User`, `Jenkins Password / Token`, `Wait for build completion` | Status page, then result |
   | `Prow Job` | `Job Name`, numeric `Build ID` | `Prow URL`, `GCS Bucket`, `GCS Prefix` | Status page, then result |

   Leave `GCS Prefix` empty unless you want to override auto-detection. For Jenkins, `Wait for build completion` is enabled by default.

   > **Tip:** `Upload File` and `Paste XML` are the fastest first-run options because they do not depend on Jenkins or Prow connectivity.

5. Submit and review the first result.

   Click `Submit Analysis`. RootCoz keeps the job on the dashboard so you can reopen it later without resubmitting.

   See [Submitting Analyses](submit-analyses.html) for details. See [Reviewing and Classifying Failures](review-and-classify-failures.html) for the next workflow. See [Tracking Analysis Progress](track-analysis-progress.html) for the live status view.

## Advanced Usage

- If you want admin login on day one, set `ADMIN_KEY` before startup, then log in as `admin` with that key.
- If you want to keep approval enabled, set `REQUIRE_APPROVAL=true`, log in as `admin`, and approve or create an `operator` before submitting jobs. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.
- If you want self-registered users to submit analyses without admin role changes, keep `DEFAULT_USER_ROLE=operator` before you start the stack.
- If you want richer analysis on the first run, add a tests repository, extra repositories, peer review models, or Jira search settings from the form. See [Configuring Analysis Context](configure-analysis-context.html) for details.
- If you prefer terminal automation once the server is running, see [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) and [CLI Command Reference](cli-reference.html).
- If you need a production-style installation instead of the local Docker stack, see [Deploying RootCoz](deploy-rootcoz.html) for details.

## Troubleshooting

- If `docker compose up -d` fails with `AI_PROVIDER is required` or `AI_MODEL is required`, add both values to `.env` and start again.
- If you can log in but do not see `New Analysis`, your account is not `operator` or `admin`. Set `DEFAULT_USER_ROLE=operator` for a local trial or ask an admin to change your role.
- If registration succeeds but RootCoz says your account is awaiting approval, either disable approval for the local quickstart with `REQUIRE_APPROVAL=false` or have an admin approve the account.
- If a Jenkins submission fails immediately, verify `JENKINS_URL`, `JENKINS_USER`, and `JENKINS_PASSWORD`, or enter them directly on the form for that run.
- If a Prow submission fails immediately, provide `Prow URL` and `GCS Bucket` in the form or set `PROW_URL` and `GCS_BUCKET` in `.env` before restarting.
- If an XML submission ends as `failed` with a parse error, you pasted or uploaded something that is not JUnit XML — console output or an HTML error page will not parse. If instead it completes with nothing to analyze, the XML was valid but contained no failing tests. RootCoz only analyzes failures and errors from the XML.

## Related Pages

- [Deploying RootCoz](deploy-rootcoz.html)
- [Submitting Analyses](submit-analyses.html)
- [Configuring Analysis Context](configure-analysis-context.html)
- [Tracking Analysis Progress](track-analysis-progress.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)


---

Source: deploy-rootcoz.md

# Deploying RootCoz

> **Note:** The Helm chart bootstraps RootCoz itself. Configure Jenkins, Jira, GitHub, Report Portal, and other runtime settings in the UI after first login. See [Configuration Reference](configuration-reference.html) and [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.


> **Warning:** The Helm chart is single-replica and expects one persistent volume. Enable either `route` or `ingress`, not both.

## Prerequisites

- Docker with Docker Compose for the local recipe, or Helm 3 plus `kubectl` or `oc` for the cluster recipes
- A cluster with a `ReadWriteOnce` PersistentVolume provisioner for the SQLite data volume
- One AI provider credential: `GEMINI_API_KEY` or `ANTHROPIC_API_KEY` (or a Vertex service account key). `CURSOR_API_KEY` alone is not enough — the bundled Compose file leaves `CURSOR_API_KEY`, `ACPX_AGENTS`, and `CLI_AGENTS` commented out, so a Cursor deployment needs those uncommented and `ACPX_AGENTS` (or `CLI_AGENTS`) set before Cursor models are discovered.
- An admin API key of at least 16 characters if you want to set `admin.key` yourself; otherwise the interactive setup script requires one and plain `helm install` auto-generates it
- `openssl`, only for the self-signed certificate in the Ingress recipe

## Start locally with Docker Compose

Bring up a local RootCoz on `http://localhost:8000` with persistent data in `./data`.

```bash
    cat > .env <<'EOF'
    JENKINS_URL=https://jenkins.example.com
    JENKINS_USER=ci-reader
    JENKINS_PASSWORD=jenkins-api-token
    JENKINS_SSL_VERIFY=true
    AI_PROVIDER=gemini
    AI_MODEL=gemini-2.5-pro
    GEMINI_API_KEY=replace-with-real-gemini-key
    LOG_LEVEL=INFO
    DEBUG=false
    EOF

    docker compose up -d
    curl http://localhost:8000/health
```

This uses the repo’s `docker-compose.yaml`, builds the local image, and keeps the SQLite database in `./data`. Use it for laptops, demos, and single-user environments where `localhost` access is enough.

- After changing `.env`, reload with `docker compose up -d --force-recreate rootcoz`.
- Continue with [Quickstart](quickstart.html) once the health check returns `{"status":"healthy"}`.

> **Note:** This `.env` leaves the shipped defaults for access control in place: `REQUIRE_APPROVAL` is `true` and `DEFAULT_USER_ROLE` is `reviewer`. Add `REQUIRE_APPROVAL=false` and `DEFAULT_USER_ROLE=operator` if you want to register and submit without an admin in the loop.

> **Note:** To register custom pi providers for the sidecar (pi-sidecar >=4.8.5), mount an agent dir with `models.json` / `auth.json` / `settings.json` into the container and add `PI_SIDECAR_AGENT_DIR=/path/to/dir` plus any `"$VAR"` API key variables to `.env` — the sidecar reads the dir once at start, so recreate the container after changing it. On Helm, use `sidecar.agentDir.*` values instead; see the [Configuration Reference](configuration-reference.html#sidecar-custom-providers-agent-dir).

## Bootstrap a shared cluster interactively

Use the setup script to generate safe Helm values files outside the repo and install the chart in one pass.

```bash
    mkdir -p "$HOME/.config/rootcoz/helm"

    uv run python scripts/helm-setup.py \
      --release rootcoz \
      --namespace rootcoz \
      --output-dir "$HOME/.config/rootcoz/helm"
```

The script prompts for cluster type, hostname, AI provider, credentials, and the bootstrap admin key, then writes `values.generated.yaml` and `values.secrets.yaml` before running `helm upgrade --install`. Use this when you want the fastest first-time shared deployment without hand-editing values files.

- Add `--skip-helm` to write files only.
- Add `--dry-run` to pass `--dry-run` through to Helm.

## Install on OpenShift with a Route

Publish RootCoz on OpenShift with a stable route and keep sensitive values outside the git checkout.

```bash
    NAMESPACE=rootcoz
    VALUES_DIR="$HOME/.config/rootcoz/helm"
    mkdir -p "$VALUES_DIR"

    cat > "$VALUES_DIR/values.generated.yaml" <<'EOF'
    route:
      enabled: true
      host: rootcoz.apps.example.com
    ingress:
      enabled: false
    ai:
      provider: gemini
      model: gemini-2.5-pro
    EOF

    cat > "$VALUES_DIR/values.secrets.yaml" <<'EOF'
    ai:
      geminiApiKey: "replace-with-real-gemini-key"
    admin:
      key: "rootcoz-admin-2026-demo-key-please-change"
    encryptionKey: "7d7f4cef5a224778a3a1a5d8af4c12b62"
    EOF

    helm upgrade --install rootcoz ./chart \
      --namespace "$NAMESPACE" --create-namespace \
      -f "$VALUES_DIR/values.generated.yaml" \
      -f "$VALUES_DIR/values.secrets.yaml"

    oc get route -n "$NAMESPACE"
```

This uses the chart’s default OpenShift-friendly path: a Route on top of the `rootcoz` service, persistent storage, and a bootstrap admin key you control from day one. Use it when you want a shared internal deployment with the smallest amount of cluster-specific tuning.

- Omit `route.host` or set it to `""` if you want OpenShift to generate the hostname.
- After the route exists, sign in as `admin` and continue with [Quickstart](quickstart.html).

## Install on Kubernetes with TLS Ingress

Run RootCoz behind a standard Kubernetes Ingress and a TLS secret so browser sessions stay on HTTPS.

```bash
    NAMESPACE=rootcoz
    VALUES_DIR="$HOME/.config/rootcoz/helm"
    mkdir -p "$VALUES_DIR" /tmp/rootcoz-tls

    kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -

    openssl req -x509 -nodes -newkey rsa:2048 \
      -keyout /tmp/rootcoz-tls/tls.key \
      -out /tmp/rootcoz-tls/tls.crt \
      -days 365 \
      -subj "/CN=rootcoz.example.com"

    kubectl create secret tls rootcoz-tls \
      --cert=/tmp/rootcoz-tls/tls.crt \
      --key=/tmp/rootcoz-tls/tls.key \
      -n "$NAMESPACE" \
      --dry-run=client -o yaml | kubectl apply -f -

    cat > "$VALUES_DIR/values.generated.yaml" <<'EOF'
    route:
      enabled: false
    ingress:
      enabled: true
      host: rootcoz.example.com
      className: nginx
      tls:
        enabled: true
        secretName: rootcoz-tls
    ai:
      provider: gemini
      model: gemini-2.5-pro
    EOF

    cat > "$VALUES_DIR/values.secrets.yaml" <<'EOF'
    ai:
      geminiApiKey: "replace-with-real-gemini-key"
    admin:
      key: "rootcoz-admin-2026-demo-key-please-change"
    encryptionKey: "7d7f4cef5a224778a3a1a5d8af4c12b62"
    EOF

    helm upgrade --install rootcoz ./chart \
      --namespace "$NAMESPACE" --create-namespace \
      -f "$VALUES_DIR/values.generated.yaml" \
      -f "$VALUES_DIR/values.secrets.yaml"
```

This recipe is for vanilla Kubernetes clusters where you want shared browser access and secure cookies from the start. The self-signed certificate keeps the recipe copy-pasteable; swap it for your normal cluster TLS secret or cert-manager output before exposing the service broadly.

- Replace `className: nginx` with your actual ingress class if needed.
- For production certificates, keep the same `secretName` and remove the `openssl` step.

## Run a private ClusterIP-only release and port-forward it

Use this when you want a shared in-cluster deployment without exposing RootCoz through a Route or Ingress yet.

```bash
    NAMESPACE=rootcoz
    VALUES_DIR="$HOME/.config/rootcoz/helm"
    mkdir -p "$VALUES_DIR"

    cat > "$VALUES_DIR/values.generated.yaml" <<'EOF'
    route:
      enabled: false
    ingress:
      enabled: false
    ai:
      provider: gemini
      model: gemini-2.5-pro
    EOF

    cat > "$VALUES_DIR/values.secrets.yaml" <<'EOF'
    ai:
      geminiApiKey: "replace-with-real-gemini-key"
    admin:
      key: "rootcoz-admin-2026-demo-key-please-change"
    encryptionKey: "7d7f4cef5a224778a3a1a5d8af4c12b62"
    EOF

    helm upgrade --install rootcoz ./chart \
      --namespace "$NAMESPACE" --create-namespace \
      -f "$VALUES_DIR/values.generated.yaml" \
      -f "$VALUES_DIR/values.secrets.yaml"

    kubectl port-forward svc/rootcoz 8000:8000 -n "$NAMESPACE"
```

This keeps the service internal to the cluster and gives you temporary browser and API access on `http://localhost:8000` through `kubectl port-forward`. Use it for admin-only testing, locked-down evaluation clusters, or the period before your ingress or route is approved.

- With no Route or TLS Ingress, the chart automatically falls back to non-secure cookies for this HTTP-only access pattern.
- When you are ready to publish it, switch to the Route or Ingress recipe instead of editing the Service directly.

## Upgrade and smoke-test a Helm release

Apply new values, wait for the rollout, and run the chart’s built-in health test after any deployment change.

```bash
    VALUES_DIR="$HOME/.config/rootcoz/helm"

    helm upgrade rootcoz ./chart -n rootcoz \
      -f "$VALUES_DIR/values.generated.yaml" \
      -f "$VALUES_DIR/values.secrets.yaml"

    kubectl rollout status deployment/rootcoz -n rootcoz
    helm test rootcoz -n rootcoz
```

This is the shortest safe path for normal Helm updates once your release is already running. The `helm test` pod curls `/health`, so you get a quick verification that the app is listening after the rollout finishes.

- If you changed external secrets out of band, follow with `kubectl rollout restart deployment/rootcoz -n rootcoz`.
- For first-login and first-analysis steps after the rollout, see [Quickstart](quickstart.html).

## Troubleshooting

- `route.enabled and ingress.enabled are mutually exclusive`
  The chart refuses to render with both on. Set the one you are not using to `false`.

- `ingress.host is required when ingress.enabled is true`
  Kubernetes Ingress has no auto-generated hostname. Add `ingress.host` or switch to the OpenShift Route recipe.

- `ai.provider and ai.model are required for install`
  These two keys are enforced on first install only. Add them to `values.generated.yaml`.

- `ai.geminiApiKey is required when ai.provider is gemini`
  Add the matching credential key for your provider: `ai.geminiApiKey`, `ai.anthropicApiKey`, or `ai.cursor.apiKey`.

- `ADMIN_KEY not found in existing secret and admin.key not set in values`
  The credentials Secret was edited or deleted outside Helm. Restore the key by setting `admin.key` in values to match your original key, or reinstall from scratch.

- `helm template` renders empty secrets
  Expected. Auto-generated secrets use Helm `lookup`, which needs a live cluster. Run `helm install` or `helm upgrade` instead.

- `kubectl port-forward` reports the pod is not ready
  Wait for the rollout with `kubectl rollout status deployment/rootcoz -n rootcoz`, then run `helm test rootcoz -n rootcoz` to confirm `/health` responds.

- I cannot log in as `admin`.
  Retrieve the bootstrap key with `kubectl get secret rootcoz-secret -n rootcoz -o jsonpath='{.data.ADMIN_KEY}' | base64 -d; echo` and use it as the password for username `admin`.

## Related Pages

- [Quickstart](quickstart.html)
- [Configuration Reference](configuration-reference.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [API Endpoint Reference](api-reference.html)
- [Submitting Analyses](submit-analyses.html)


---

Source: configuration-reference.md

# Configuration Reference

RootCoz reads its configuration from environment variables at startup and lets admins override most of it later from the Server Settings page. This page lists every variable, its default, whether it is a secret, and whether a restart is needed — organised into the same categories the Server Settings UI uses.

## Prerequisites
- A running or planned RootCoz deployment. See [Deploying RootCoz](deploy-rootcoz.html) for the Kubernetes and OpenShift paths.
- Admin access if you plan to change settings after login. See [Managing Users and Server Settings](manage-users-and-server-settings.html).
- One AI credential for `claude`, `gemini`, or `cursor` if you want analysis to run at all.

## Quick Example

```dotenv
    AI_PROVIDER=claude
    AI_MODEL=<a model id from GET /api/ai-models>
    ANTHROPIC_API_KEY=your-anthropic-api-key
    DEFAULT_USER_ROLE=operator
    REQUIRE_APPROVAL=false
    SECURE_COOKIES=false
```

Those six lines are enough to run the local Docker stack at `http://localhost:8000` and submit your first analysis.

Only `AI_PROVIDER` and `AI_MODEL` are hard-required. `docker-compose.yaml` fails fast without them, and the Helm chart requires `ai.provider` and `ai.model`. `ADMIN_KEY` is optional: the chart auto-generates it on first install. Everything else has a working default.

> **Note:** Fill `AI_MODEL` in from your own deployment. There is no built-in default model list — the catalogue is discovered from the Pi-sidecar at runtime, so a model id copied from elsewhere may not resolve. Run `rootcoz ai-models` (or `GET /api/ai-models`) against a running server and pick an id that is actually listed.

`AI_PROVIDER` and `AI_MODEL` are resolved as a pair at submit time, and the resolution is strict:

- An **exact** `provider/model` pair that appears in the sidecar catalogue is used as-is.
- The friendly names `claude`, `gemini`, and `cursor` are **not** provider IDs. Each maps to a sidecar provider (`claude` to `google-vertex-claude` or `cli-claude`, `gemini` to `google` or `cli-gemini`, `cursor` to any `*-cursor` provider including `cli-cursor`), and that mapping is applied only when the requested model id appears under **exactly one** catalogue provider *and* that provider is the mapped one.
- Anything else is rejected with `Unknown Pi-sidecar provider/model pair`, including an id that appears under two providers.

The practical consequence is that `ANTHROPIC_API_KEY` alone does not authenticate `AI_PROVIDER=claude`: that key authenticates the `anthropic` provider, while the `claude` alias maps to `google-vertex-claude`, so a literal model id such as `claude-sonnet-4` is rejected unless your sidecar happens to expose it solely under `google-vertex-claude`. Check `rootcoz ai-models` before choosing an id, or set `AI_PROVIDER` to the exact provider ID that endpoint lists alongside the model.

## Step-by-Step

1. **Understand the two configuration layers.**

   Environment variables (and `.env`) provide the base values. Settings saved by an admin in the Server Settings UI are stored in the database, encrypted when sensitive, and merged over the environment values at request time. Emptying a field in the UI deletes the override and falls back to the environment value, then to the code default.

   | Layer | Set by | Scope |
   | --- | --- | --- |
   | Environment variable | Deployment (`.env`, ConfigMap, Helm values) | Bootstrap defaults for the whole server |
   | Server Settings | Admin, after login | Overrides one server-wide value, applied without restart for most fields |

   Two further override layers sit above both for analysis settings: the per-request body, and `.rootcoz/settings.json` in the cloned tests repository. Resolution order is request → `settings.json` → Server Settings → environment variable. See [Configuring Analysis Context](configure-analysis-context.html).

2. **Read the tables below as a matrix.**

   Columns:

   | Column | Meaning |
   | --- | --- |
   | Variable | Environment variable name, always the field name uppercased |
   | Default | Value used when nothing else sets it |
   | Secret | Encrypted at rest and masked in API responses and the UI |
   | Restart | Server restart required for a change to take effect |
   | After login | Whether an admin can change it in Server Settings without a restart |

   Fields marked `after login: no` are server-only. `PUT /api/admin/settings` rejects them with `400 Server-only settings (env var only)`, so they must come from the environment.

3. **Jenkins category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `JENKINS_URL` | empty | string | no | no | yes |
   | `JENKINS_USER` | empty | string | yes | no | yes |
   | `JENKINS_PASSWORD` | empty | string | yes | no | yes |
   | `JENKINS_SSL_VERIFY` | `true` | boolean | no | no | yes |
   | `JENKINS_TIMEOUT` | `30` | integer > 0 | no | no | yes |
   | `JENKINS_ARTIFACTS_MAX_SIZE_MB` | `500` | integer > 0 | no | no | yes |
   | `GET_JOB_ARTIFACTS` | `true` | boolean | no | no | yes |

   An empty `JENKINS_URL` means Jenkins is not configured, and each request can still carry its own credentials.

4. **AI category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `AI_PROVIDER` | empty | string | no | no | yes |
   | `AI_MODEL` | empty | string | no | no | yes |
   | `FORCE_SERVER_CREDENTIALS` | `false` | boolean | no | no | yes |
   | `ENABLE_FREE_MODELS` | `false` | boolean | no | no | **no** |
   | `AI_CALL_TIMEOUT` | `10` | integer > 0 (minutes) | no | no | yes |
   | `MAX_CONCURRENT_AI_CALLS` | `3` | integer > 0 | no | no | yes |
   | `PEER_AI_CONFIGS` | empty | string | no | no | yes |
   | `PEER_ANALYSIS_MAX_ROUNDS` | `3` | integer 1–10 | no | no | yes |

   `AI_PROVIDER` accepts any exact provider ID that pi-sidecar reports, provided the `provider/model` pair appears in its catalogue. `claude`, `gemini`, and `cursor` are convenience aliases that resolve through the mapping described above; they are not the only accepted values. The chart's `ai.provider` accepts the same freedom — any provider ID string, including a custom provider registered through the sidecar agent dir (the chart's install-time credential checks only apply to the three built-in aliases). Values are lowercased on load, and legacy `*-cli` names are normalised to their canonical form. To see what your sidecar actually exposes, read the provider list from `GET /api/ai-models`. `PEER_AI_CONFIGS` uses `provider:model,provider:model`; model names may contain commas inside square brackets, for example `cursor:gpt-5.4[context=272k,reasoning=medium]`.

   > **Note:** An unsupported `AI_PROVIDER` value fails at submit time, immediately, even when the model is not yet resolved. Blank provider strings mean "not configured" and defer the error to a clearer message later.

5. **Jira category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `JIRA_URL` | unset | string | no | no | yes |
   | `JIRA_EMAIL` | unset | string | yes | no | yes |
   | `JIRA_API_TOKEN` | unset | string | yes | no | yes |
   | `JIRA_PAT` | unset | string | yes | no | yes |
   | `JIRA_PROJECT_KEY` | unset | string | no | no | yes |
   | `JIRA_SSL_VERIFY` | `true` | boolean | no | no | yes |
   | `JIRA_MAX_RESULTS` | `5` | integer > 0 | no | no | yes |
   | `ENABLE_JIRA` | unset | boolean | no | no | yes |
   | `ENABLE_JIRA_ISSUES` | unset | boolean | no | no | yes |

   `JIRA_EMAIL` decides the auth mode. With an email, RootCoz uses Jira Cloud Basic auth and prefers `JIRA_API_TOKEN`, falling back to `JIRA_PAT`. Without an email it treats the server as Server/DC, prefers `JIRA_PAT`, and falls back to `JIRA_API_TOKEN`. Setting `ENABLE_JIRA=false` forces Jira off; setting it to `true` without a URL, credentials, or project key logs a warning and stays off.

6. **GitHub category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `GITHUB_TOKEN` | unset | string | yes | no | yes |
   | `TESTS_REPO_URL` | unset | string | no | no | yes |
   | `TESTS_REPO_TOKEN` | unset | string | yes | no | yes |
   | `ENABLE_GITHUB_ISSUES` | unset | boolean | no | no | **no** |

   With `ENABLE_GITHUB_ISSUES` unset, GitHub issue creation auto-enables when both `TESTS_REPO_URL` and `GITHUB_TOKEN` are configured. The user-facing feedback dialog stays available whenever the toggle is not explicitly `false`.

7. **Report Portal category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `REPORTPORTAL_URL` | unset | string | no | no | yes |
   | `REPORTPORTAL_API_TOKEN` | unset | string | yes | no | yes |
   | `REPORTPORTAL_PROJECT` | unset | string | no | no | yes |
   | `REPORTPORTAL_VERIFY_SSL` | `true` | boolean | no | no | yes |
   | `ENABLE_REPORTPORTAL` | unset | boolean | no | no | **no** |
   | `RP_PUSH_CLASSIFICATIONS` | `true` | boolean | no | no | **no** |
   | `RP_PUSH_ROOTCOZ_URL` | `true` | boolean | no | no | **no** |
   | `RP_PUSH_TRACKER_LINKS` | `true` | boolean | no | no | **no** |

   Auto-detection mirrors GitHub: with the toggle unset, Report Portal is on when URL, token, and project are all set. `RP_PUSH_CLASSIFICATIONS` maps RootCoz classifications to Report Portal defect types (`PRODUCT_BUG`, `AUTOMATION_BUG`, `SYSTEM_ISSUE`).

8. **Auth and Security category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `ADMIN_KEY` | empty | string | yes | no | yes |
   | `SECURE_COOKIES` | `true` | boolean | no | **yes** | **no** |
   | `TRUST_PROXY_HEADERS` | `false` | boolean | no | **yes** | **no** |
   | `REQUIRE_APPROVAL` | `true` | boolean | no | no | **no** |
   | `ADMIN_WAIT_APPROVE_MSG` | empty | string | no | no | **no** |
   | `ALLOWED_USERS` | empty | string | no | no | **no** |
   | `DEFAULT_USER_ROLE` | `reviewer` | string | no | **yes** | **no** |

   `ADMIN_KEY` bootstraps the `admin` superuser that lives outside the database; log in as `admin` with that key. `DEFAULT_USER_ROLE` accepts only `viewer`, `reviewer`, or `operator` — any other value fails validation at startup. `ALLOWED_USERS` is a comma-separated list of usernames and is empty (open) by default. When it is non-empty it is enforced at two points, not one: at registration, where a username that is not on the list is rejected with `403 Registration is restricted. Contact an admin.` before the account is created; and after authentication, on the protected routes that call the allow-list check, where admins always bypass it and any other off-list or unauthenticated caller gets `403 User not allowed. Contact an administrator to be added to the allow list.` A user who was registered before the list tightened can therefore still hold an account but gets 403 on those routes.

   > **Warning:** Set `SECURE_COOKIES=false` only for local HTTP development. Behind a TLS-terminating proxy, leave it `true` or sessions will not be stored by the browser.

   `TRUST_PROXY_HEADERS=true` makes `X-Forwarded-User` a valid identity for every route, which means the proxy becomes the authentication boundary. Only enable it behind a proxy you control.

9. **Prow category.**

   | Variable | Default | Type | Secret | Restart | After login |
   | --- | --- | --- | --- | --- | --- |
   | `PROW_URL` | empty | string | no | no | yes |
   | `GCS_BUCKET` | empty | string | no | no | yes |

   Both are validated on load rather than transformed. `PROW_URL` must start with `https://`, must contain a hostname, and must not embed credentials (`prow_validation.normalize_prow_url`) — surrounding whitespace is trimmed, but a trailing slash is left in place, so `https://prow.example.com/` and `https://prow.example.com` are both accepted and stay as written. `GCS_BUCKET` must match `[a-z0-9][a-z0-9._-]*` after trimming (`normalize_gcs_bucket`), so a `gs://` prefix is **rejected** rather than stripped; pass the bare bucket name. Both are also request-tunable, so a per-build submission can override the server default.

10. **Server category.**

    | Variable | Default | Type | Secret | Restart | After login |
    | --- | --- | --- | --- | --- | --- |
    | `PUBLIC_BASE_URL` | unset | string | no | no | **no** |
    | `ADDITIONAL_REPOS` | empty | string | no | no | yes |
    | `MAX_CONCURRENT_REPO_CLONES` | `10` | integer > 0 | no | no | yes |
    | `WAIT_FOR_COMPLETION` | `true` | boolean | no | no | yes |
    | `POLL_INTERVAL_MINUTES` | `2` | integer > 0 | no | no | yes |
    | `MAX_WAIT_MINUTES` | `0` | integer ≥ 0 | no | no | yes |
    | `METADATA_RULES_FILE` | empty | string | no | **yes** | **no** |
    | `ENABLE_AUTO_REVIEW` | `true` | boolean | no | no | **no** |
    | `AUTO_PUSH_EXPORTERS` | empty | string | no | no | **no** |

    `ADDITIONAL_REPOS` uses `name:url,name:url`, with an optional `:ref` and `@token` suffix per entry; names must be unique. `MAX_WAIT_MINUTES=0` means wait indefinitely. `PUBLIC_BASE_URL` is the only origin RootCoz will build absolute links from — request `Host` and `X-Forwarded-*` headers are never trusted, which is what blocks host-header injection in result and tracker links.

    `METADATA_RULES_FILE` points at a YAML or JSON file of name-based metadata rules. Rules load once and cache for the process lifetime, so edits need a restart. A broken rules file logs a warning and yields no rules rather than failing the server.

    `AUTO_PUSH_EXPORTERS` takes a comma-separated exporter list such as `reportportal`, and only applies when `ENABLE_AUTO_REVIEW` is also on.

   `ENABLE_FREE_MODELS` lets every user run the admin-declared free provider+model pairs (Users page → *Free AI models*) on server credentials; the free-providers grant is not needed for admins or users who already have `can_use_server_providers`.

11. **Web Push category.**

    | Variable | Default | Type | Secret | Restart | After login |
    | --- | --- | --- | --- | --- | --- |
    | `VAPID_PUBLIC_KEY` | empty | string | no | **yes** | **no** |
    | `VAPID_PRIVATE_KEY` | empty | string | yes | **yes** | yes |
    | `VAPID_CLAIM_EMAIL` | empty | string | no | **yes** | **no** |

    Leave both keys empty to auto-generate a pair on first use. Setting only one of them logs a partial-configuration warning but does **not** disable Web Push: the missing key is auto-generated and persisted, and the pair in effect may therefore not match what you set. Because notification signing resolves keys through `get_vapid_config()`, which reads environment variables or the generated key file and never the database, a `VAPID_PRIVATE_KEY` saved through Server Settings is not used for signing. Set the keys in the environment to take effect. Note also that the public key and claim email are environment-only, because changing them would invalidate subscriptions already registered with browsers.

### Variables that never reach the Server Settings UI

These are read straight from the process environment and have no `Settings` field, so they are bootstrap-only. Set them in `.env`, the Compose file, the Helm chart, or a ConfigMap.

| Variable | Default | Purpose |
| --- | --- | --- |
| `ROOTCOZ_ENCRYPTION_KEY` | auto-generated file key | Fernet key for at-rest encryption and the HMAC secret for API key hashes |
| `XDG_DATA_HOME` | `~/.local/share` | Where the auto-generated `.encryption_key` file lives |
| `XDG_CONFIG_HOME` | user default | Writable config directory the container points at `/tmp/config` |
| `LOG_LEVEL` | `INFO` | Log verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `DEV_MODE` / `DEBUG` | unset | Set to `true` to enable uvicorn auto-reload |
| `PORT` | `8000` | Port the ASGI server binds |
| `ANTHROPIC_API_KEY` | unset | Claude credential for the sidecar |
| `GEMINI_API_KEY` | unset | Gemini credential for the sidecar |
| `CURSOR_API_KEY` | unset | Cursor credential for the sidecar |
| `CLAUDE_CODE_USE_VERTEX` | unset | Set to `1` to authenticate Claude through Vertex AI |
| `CLOUD_ML_REGION` | unset | Vertex region, used with `CLAUDE_CODE_USE_VERTEX` |
| `ANTHROPIC_VERTEX_PROJECT_ID` | unset | Vertex project, used with `CLAUDE_CODE_USE_VERTEX` |
| `ACPX_AGENTS` | unset | Comma-separated providers exposed through the Cursor ACPX bridge |
| `CLI_AGENTS` | unset | Comma-separated providers exposed through the local CLI agents |
| `PI_SIDECAR_AGENT_DIR` | unset | Sidecar agent dir for custom providers and settings (see below) |

`gemini` resolves to the `google` provider, and `GEMINI_API_KEY` is the credential RootCoz documents and charts for it (`ai.provider=gemini` pairs with `ai.geminiApiKey`). `gemini auth login` is a host-side login for the standalone Gemini CLI; the sidecar exposes that agent under its own provider id — `cli-gemini`, the canonical form of the legacy `gemini-cli` name — not under `google`, so it is not a substitute for `GEMINI_API_KEY` on the `gemini` alias. For `claude`, see the resolution rules above: the alias maps to `google-vertex-claude`, so `ANTHROPIC_API_KEY` on its own is not the credential that provider uses — the Vertex path is. RootCoz passes the environment through to the sidecar and cannot say which credentials a provider accepts, so confirm the requirement in that provider's own documentation.

RootCoz does not read the Vertex trio itself — `config.py` documents them as variables the Claude CLI consumes, and the server only passes them through the process environment to the sidecar. What this repo guarantees is limited to that pass-through. The Helm chart is the more useful reference, because it treats Vertex as a unit: setting `ai.vertex.enabled` writes `CLAUDE_CODE_USE_VERTEX=1`, `CLOUD_ML_REGION`, and `ANTHROPIC_VERTEX_PROJECT_ID`, and separately mounts `ai.vertex.serviceAccountKey` and points `GOOGLE_APPLICATION_CREDENTIALS` at the mounted `application_default_credentials.json`. Install fails outright if `ai.vertex.enabled` is set without `ai.vertex.projectId` or `ai.vertex.serviceAccountKey`, so the chart's own definition of "Vertex enabled" includes a credential file, not just the three variables above.

Whether the provider ends up registered and usable, and what credentials it actually requires, is decided inside the Pi-sidecar and the Anthropic Vertex tooling, not here. For the authoritative credential requirements, follow the Anthropic Claude Code on Vertex AI documentation rather than treating the trio as self-sufficient. This matters for `AI_PROVIDER=claude` in particular, because that alias resolves to the `google-vertex-claude` provider — so Vertex is the path on which `claude` is expected to resolve at all.

### Sidecar custom providers (agent dir)

Pi-sidecar >=4.8.5 reads an **agent dir** at process start: `models.json` registers custom pi providers (listed by `GET /api/ai-models`, usable as `AI_PROVIDER`/`AI_MODEL`), `auth.json` supplies provider credentials, and `settings.json` seeds the sidecar's in-memory settings store. Point the sidecar at a persistent dir with `PI_SIDECAR_AGENT_DIR` (unset means an ephemeral scratch dir, i.e. no custom providers). A `models.json` `apiKey` may reference an environment variable — `"$MYGATEWAY_API_KEY"` resolves against the sidecar process environment, so the variable must be exported on the same container.

Two operational rules follow from "read at process start": changing any file requires a sidecar restart (the Helm chart handles this with a checksum-annotated rolling restart; in Docker Compose restart the container), and `rootcoz ai-models` refresh does not re-read the directory. There is no provider allowlist on the RootCoz side — any exact provider/model pair present in the sidecar catalogue is accepted, so a custom provider is selected the same way as a built-in one.

> **Warning:** `ROOTCOZ_ENCRYPTION_KEY` is the one variable worth getting right before you have real data. Rotating it invalidates every encrypted stored token *and* every stored API key hash, so all keys must be re-issued afterwards. Existing sessions use plain SHA-256 hashing and survive. If you do not set it, RootCoz generates a file under `$XDG_DATA_HOME/rootcoz/.encryption_key` and warns you in the health endpoint.

### Helm chart coverage

The Helm chart exists for the bootstrap-only variables, since those cannot be set after login:

| Chart value | Variable |
| --- | --- |
| `ai.provider`, `ai.model` | `AI_PROVIDER`, `AI_MODEL` |
| `ai.geminiApiKey` | `GEMINI_API_KEY` |
| `ai.anthropicApiKey` | `ANTHROPIC_API_KEY` |
| `ai.vertex.enabled`, `ai.vertex.projectId`, `ai.vertex.region`, `ai.vertex.serviceAccountKey` | `CLAUDE_CODE_USE_VERTEX`, `ANTHROPIC_VERTEX_PROJECT_ID`, `CLOUD_ML_REGION`, plus `GOOGLE_APPLICATION_CREDENTIALS` pointing at the mounted GCP key |
| `ai.cursor.apiKey`, `ai.cursor.authJson` | `CURSOR_API_KEY` and the mounted Cursor `auth.json` |
| `sidecar.agentDir.modelsJson`, `sidecar.agentDir.authJson`, `sidecar.agentDir.settingsJson` | `PI_SIDECAR_AGENT_DIR` pointing at a writable emptyDir at `/etc/pi-sidecar-agent`, populated by an initContainer from the rendered Secret (pi writes `models-store.json` next to `models.json`, so the dir must be writable); `sidecar.agentDir.env` adds Secret-backed env vars for `"$VAR"` API key references in `models.json` (omitting a key preserves its stored value on upgrade, `""` removes it; chart-managed variable names are rejected) |
| `admin.key` | `ADMIN_KEY` |
| `encryptionKey` | `ROOTCOZ_ENCRYPTION_KEY` |
| `env.xdgDataHome`, `env.xdgConfigHome` | `XDG_DATA_HOME`, `XDG_CONFIG_HOME` |
| `env.secureCookies`, `env.publicBaseUrl` | `SECURE_COOKIES`, `PUBLIC_BASE_URL`; both auto-derived when left empty, as described below |
| `env.metadataRulesFile` | `METADATA_RULES_FILE` |
| `tuning.logLevel` | `LOG_LEVEL` |
| `route.*`, `ingress.*` | Routing for the deployment, including the derived `PUBLIC_BASE_URL` origin |

`admin.key` and `encryptionKey` are auto-generated on first install when empty. Database-configurable fields deliberately have no chart values — the chart comment says so explicitly, and you set those from Server Settings after the first login.

When `env.secureCookies` and `env.publicBaseUrl` are left empty, the chart derives them in `_helpers.tpl`, and the two do not follow the same rule:

| Configuration | `SECURE_COOKIES` | `PUBLIC_BASE_URL` |
| --- | --- | --- |
| `route.enabled` with `route.host` set | `true` | `https://<route.host>` |
| `route.enabled` with `route.host` empty (the chart default on OpenShift) | `true` | unset (empty) |
| Ingress with `ingress.host` set and `ingress.tls.enabled` | `true` | `https://<ingress.host>` |
| Ingress with `ingress.host` set but **no** TLS | `false` | `http://<ingress.host>` |
| Neither Route nor Ingress enabled | `false` | unset (empty) |

The Ingress requires `ingress.host` — `helm install` fails without it — so the "Ingress enabled with no host" case cannot occur.

So a plain-HTTP Ingress still gets a derived base URL — as an `http://` origin — while `SECURE_COOKIES` stays `false`, because that value only flips to `true` for a Route or a TLS Ingress. Session cookies are consequently issued without the `Secure` attribute on a non-TLS Ingress. Set `env.secureCookies` and `env.publicBaseUrl` explicitly if you terminate TLS upstream of the Ingress or need the cookies hardened anyway.

## Advanced Usage

Change settings after login without touching a restart. Open Server Settings, edit the field, and save. The value is validated against the same Pydantic constraints as the environment variable, encrypted if sensitive, and applied to the running process immediately. Setting a field to an empty string deletes the override.

Sensitive values are returned masked as `••••••••`. `GET /api/admin/settings?reveal_key=ADMIN_KEY` unmasks one key, and `reveal_key=__all__` unmasks everything. Responses also report whether the value came from the database, the environment, or the code default, and flag when an environment variable of the same name also exists. See [API Endpoint Reference](api-reference.html).

Encryption uses Fernet (AES-128-CBC plus HMAC-SHA256). Sensitive fields are `JENKINS_USER`, `JENKINS_PASSWORD`, `JIRA_EMAIL`, `JIRA_API_TOKEN`, `JIRA_PAT`, `GITHUB_TOKEN`, `TESTS_REPO_TOKEN`, `REPORTPORTAL_API_TOKEN`, `ADMIN_KEY`, and `VAPID_PRIVATE_KEY`. They are encrypted before storage and never logged at any level. They are stripped from ordinary API responses, with one exception an administrator should know about: `GET /api/admin/settings` decrypts them and masks them by default, but returns them in full when called with `reveal_key`. Treat that response as secret-bearing.

Blank strings normalise consistently: optional strings become unset, `AI_PROVIDER` is lowercased and normalised, `DEFAULT_USER_ROLE` is validated against the three allowed roles, and secret strings lose surrounding whitespace — but only five of them: `GITHUB_TOKEN`, `TESTS_REPO_TOKEN`, `JIRA_API_TOKEN`, `JIRA_PAT`, and `REPORTPORTAL_API_TOKEN`. `ADMIN_KEY` and `VAPID_PRIVATE_KEY` are **not** trimmed, so a value pasted with surrounding whitespace keeps it, and a trailing newline — easy to pick up from a mounted Kubernetes Secret or `echo` — stays part of the value and will not match on comparison. Pass those two without surrounding whitespace.

## Troubleshooting
- `AI_PROVIDER is required` at startup.
  The entrypoint hard-fails without a provider and model. Set both in `.env` and restart the stack.

- `docker compose up` fails but the UI shows no error.
  The entrypoint exits before the server binds. Run `docker compose logs` and read the first lines.

- `400 Server-only settings (env var only)`.
  You tried to set an environment-only field from Server Settings. Change it in `.env`, the ConfigMap, or the Helm values and restart.

- A Server Settings change has no effect.
  Check the `restart_required` marker on the field. `SECURE_COOKIES`, `TRUST_PROXY_HEADERS`, `DEFAULT_USER_ROLE`, `METADATA_RULES_FILE`, and all three Web Push fields only apply after a restart.

- A secret looks like `••••••••` in the API response.
  That is intentional. Use `reveal_key` to unmask a specific key, and never log the full response.

- `ROOTCOZ_ENCRYPTION_KEY is not set` in `/api/health`.
  Expected outside production. Set the variable for real deployments; otherwise the key lives in a local file and any container rebuild discards it.

- `Peer config` errors mentioning an unmatched bracket or an empty entry.
  `PEER_AI_CONFIGS` is comma-separated and bracket-aware. Check for a trailing comma or a model name with an unclosed `[`.

- Metadata rules are ignored after editing the file.
  `METADATA_RULES_FILE` is cached for the process lifetime. Restart the server.

## Related Pages
- [Deploying RootCoz](deploy-rootcoz.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [Configuring Analysis Context](configure-analysis-context.html)
- [Quickstart](quickstart.html)
- [API Endpoint Reference](api-reference.html)
- [CLI Command Reference](cli-reference.html)

---

Source: submit-analyses.md

# Submitting Analyses

Use this page when you have CI output and want RootCoz to turn it into a report. It covers the four input modes, what each one needs, the difference between collecting CI results and running AI analysis over them, and the role you need before you can submit anything.

## Prerequisites
- A running RootCoz server. See [Quickstart](quickstart.html) or [Deploying Rootcoz](deploy-rootcoz.html).
- An account with the `operator` or `admin` role. `reviewer` and `viewer` can read results but cannot submit.
- At least one AI provider credential for `claude`, `gemini`, or `cursor`, unless you are only collecting CI results.
- Your input: a Jenkins job and build, a Prow job and build ID, a JUnit XML file, or raw JUnit XML text.

## Quick Example
```bash
    rootcoz analyze --source file --file junit-results.xml
```

This queues a JUnit XML analysis from the terminal and prints the job ID to poll. The same work from the browser is `New Analysis` → `Upload File` → `Submit Analysis`.

## Step-by-Step
1. **Open `New Analysis`.**

   The form loads your server defaults first, so most fields are already filled in. If the defaults request times out, the form still works and shows a warning that it fell back to local values.

   You are signed in as `operator` or `admin`; otherwise the `New Analysis` link is not available and the API returns a permission error.

2. **Pick the input mode that matches what you have.**

   The four mode buttons are `Jenkins Job`, `Prow Job`, `Paste XML`, and `Upload File`. `Upload File` and `Paste XML` are both JUnit XML input and send the same `file` request type; the difference is only how the XML gets into the form.

   | Mode | `type` sent | Required fields | What happens after submit |
   | --- | --- | --- | --- |
   | `Jenkins Job` | `jenkins` | `Job Name`, `Build Number` | Status page, then the report |
   | `Prow Job` | `prow` | `Job Name`, `Build ID` | Status page, then the report |
   | `Upload File` | `file` | a `.xml` file | Report opens directly |
   | `Paste XML` | `file` | `JUnit XML Content` | Report opens directly |

   `Paste XML` and `Upload File` only accept JUnit XML. Console output, log files, and HTML error pages are not parsed as an empty report — they fail the XML parse, and the job ends as `failed` with a parse error. A job reaches the empty-report path only when the XML is well-formed and contains no failing tests.

3. **Fill in only what the mode needs.**

   Everything outside the mode's own section is optional and falls back to your server settings. The form sends the additional-repository and peer-provider lists only when they are non-empty, so leaving those fields unfilled does not override a configured server default.

   | Field | Appears in | Notes |
   | --- | --- | --- |
   | `Job Name` | `Jenkins Job`, `Prow Job` | Jenkins uses the full path, for example `folder/job-name` |
   | `Build Number` | `Jenkins Job` | Positive integer |
   | `Build ID` | `Prow Job` | Numeric only; the field strips non-digits as you type |
   | `Jenkins URL`, `Jenkins User`, `Jenkins Password / Token` | `Jenkins Job` | Override the server default for this run only |
   | `Wait for build completion` | `Jenkins Job` | On by default. Off means analyze the build as it stands now |
   | `Poll Interval (min)`, `Max Wait (min, 0 = no limit)` | `Jenkins Job` | Only shown when `Wait for build completion` is on |
   | `Prow URL`, `GCS Bucket` | `Prow Job` | Override the server default for this run only |
   | `GCS Prefix` | `Prow Job` | Leave empty. RootCoz resolves it from `prowjob.json` or the PR directory pointer |
   | `JUnit XML Content` | `Paste XML` | Raw JUnit XML |
   | `AI Provider`, `AI Model` | always | Must be an available provider on the server, or a key on your own account |
   | `Tags` | always | Free-form, added to the job and merged with rule-assigned labels |
   | `Fetch build artifacts` / `Fetch GCS build artifacts` | `Jenkins Job`, `Prow Job` | On by default |

   > **Note:** Jenkins and Prow validate what you type before the request is sent. An invalid Prow job name or a non-numeric build ID is rejected client-side; `GCS Prefix` is also checked against the job name and build ID server-side when you fill it in.

4. **Click `Submit Analysis`.**

   The button stays disabled until the mode's required fields are valid and an AI provider and model are available — with one exception. If a Tests Repo URL is filled and both AI fields are left empty, the form defers the AI choice to the repository's own `.rootcoz/settings.json` and enables the button. `Paste XML` and `Upload File` go straight to the report page; `Jenkins Job` and `Prow Job` open the status page first because they have to reach your CI server and wait for the build.

   See [Tracking Analysis Progress](track-analysis-progress.html) for what the status page shows.

5. **Know what runs after the request is accepted.**

   `POST /analyze` returns `202` immediately with a `job_id` and does the real work in the background. In order:

   | Stage | What happens |
   | --- | --- |
   | Wait | With `Wait for build completion` on, the job sits in `waiting` and polls Jenkins until the build finishes |
   | Fetch | The source plugin collects failures, passed tests, and skipped tests from the CI job or the XML |
   | Preflight | RootCoz confirms the AI provider and model are actually reachable before starting expensive work |
   | Group | Failures are grouped by error signature: a SHA-256 hash of the normalized error message and stack trace, with timestamps, UUIDs, pod names, and build numbers stripped out first |
   | Clone | The tests repo and any additional repos are cloned into a workspace, and `.rootcoz/` agents, skills, and extensions are copied in |
   | Analyze | One AI call per unique error signature, not per failing test. The analysis is then applied to every test that shares that signature |
   | Cross-failure pass | A final pass looks for patterns spanning multiple failure groups |
   | Save | Results, all test outcomes, and token usage are persisted |

   Two consequences worth knowing before you submit. A run with fifty failing tests that share three distinct errors makes three AI calls, not fifty. And if every group fails, the job is marked `failed` rather than quietly reporting partial results.

   > **Note:** Analysis sessions run with a restricted tool set (`read`, `ls`, `find`, `grep`). The AI can read the cloned workspace but cannot run shell commands.

6. **Check the result.**

   The report groups failures by the same error signature used during analysis, so a card can cover several tests. See [Reviewing and Classifying Failures](review-and-classify-failures.html) for the review and classification workflow, and [Exploring History and Reports](explore-history-and-reports.html) to find an older run.

## Advanced Usage
- **Collect results now, analyze later.** `POST /submit` runs the exact same fetch and storage as `POST /analyze` and then stops before the clone and AI stages. The job is stored with `analysis_state=submitted` and the summary `CI results have been collected and stored. AI analysis has not run.` Later, `POST /results/{job_id}/analyze` runs the AI pipeline in place against the same job ID. Use this when you want a cheap, immediate record of a build and only need the expensive analysis for the ones that matter.

  | | `POST /analyze` | `POST /submit` |
  | --- | --- | --- |
  | CI fetch and storage | yes | yes |
  | Repository clone, workspace, AI calls | yes | no |
  | Resulting `analysis_state` | `analyzed` | `submitted` |
  | Needs AI configuration | yes | no |

  In the web UI, a submitted job shows `Submitted` in the dashboard Mode column and an `Analyze` button in the report header instead of `Re-Analyze`.

  > **Tip:** Jobs that predate this split are stored as `analyzed`, so you never have to guess what a missing `analysis_state` means.

- **Change the analysis context for one run.** `Tests Repo URL` and `Ref / Branch`, `Additional Repositories`, `Raw Prompt`, `Peer Analysis`, `Jira Integration`, and `AI Call Timeout` are all per-submission overrides that take precedence over server settings. See [Configuring Analysis Context](configure-analysis-context.html) for details.

- **Submit from the terminal.** `rootcoz analyze` and `rootcoz submit` take the same flags, and `--source` selects between `jenkins`, `file`, and `prow`. The CLI reads the same server for defaults, so anything you set in the form is set in your CLI profile. Use `submit` when you want the results stored without running AI.

  ```bash
  rootcoz analyze --source jenkins --job-name folder/job-name --build-number 123
  rootcoz submit --source jenkins --job-name folder/job-name --build-number 123
  ```

  See [CLI Command Reference](cli-reference.html) for the full flag list and [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) for scripted workflows.

- **Reuse a stored submission.** `Re-Analyze` in the report header replays the original request with your overrides applied and produces a new job ID. Sensitive fields such as the tests repo token are stored encrypted and restored automatically, so you do not re-enter credentials. The new report links back to its origin.

## Troubleshooting
- **I do not see `New Analysis`, or the API returns a permission error.**
  Your account is `viewer` or `reviewer`. Submitting requires `operator` or `admin`. Set `DEFAULT_USER_ROLE=operator` for a local trial, or ask an admin to change your role.

- **The form says `Select an available AI provider and model.`**
  No provider the form lists is usable for this account. Add your own API key in your account settings, ask an admin to configure server providers, or use `POST /submit` and run the AI later with `POST /results/{job_id}/analyze`.

- **`Submit Analysis` is greyed out.**
  The required fields for the selected mode are not filled in. Jenkins needs both `Job Name` and a positive `Build Number`; Prow needs a valid `Job Name` and a numeric `Build ID`; the XML modes need non-empty XML content.

- **A Jenkins or Prow submission fails immediately.**
  The server could not reach CI. Check `Jenkins URL`, `Jenkins User`, and `Jenkins Password / Token`, or `Prow URL` and `GCS Bucket`, either in the form or in the server environment variables.

- **The Prow job fails to find artifacts.**
  Leave `GCS Prefix` empty so RootCoz resolves it, and confirm the `GCS Bucket` actually holds the build. Source-level problems such as denied GCS access are shown under `Source Warnings` on the status page rather than failing the run outright.

- **The report has no failures but the XML was not empty.**
  RootCoz only analyzes `failure` and `error` test cases in JUnit XML. Verify you uploaded JUnit XML and not console output, and check the report header's passed and skipped counts.

- **The job is `completed` with `No test failures found in the provided input.`**
  The fetch succeeded and there was nothing to analyze, so RootCoz skipped the AI, the clone, and the workspace entirely. Test outcomes are still saved and the counts still appear on the dashboard.

## Related Pages
- [Quickstart](quickstart.html)
- [Tracking Analysis Progress](track-analysis-progress.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Configuring Analysis Context](configure-analysis-context.html)
- [CLI Command Reference](cli-reference.html)
- [API Reference](api-reference.html)


---

Source: track-analysis-progress.md

# Tracking Analysis Progress

Use this page when you have submitted an analysis and want to know what it is doing right now, or how to check on it from a script. It covers the status page, the states a job moves through, the dashboard view of everything in flight, and how live updates reach the browser.

## Prerequisites
- A submitted job. See [Submitting Analyses](submit-analyses.html).
- Any role that can view results. Tracking is read-only; the exceptions are `Abort` and re-analysis, listed below.

## Quick Example
```bash
    rootcoz results show JOB_ID --fields status,result.analysis_state,result.summary
```

This prints just the three fields a status check needs, instead of the full result payload. The rest of this page covers the same job in the browser.

## Step-by-Step
1. **Open the status page.**

   Jenkins and Prow submissions open `/status/{job_id}` automatically. From the dashboard, clicking a row whose status is `waiting`, `pending`, `running`, `failed`, or `aborted` opens the status page; completed jobs go straight to the report.

   The page shows a status headline, a metadata block, and a live progress list. Because the progress log is stored server-side, reloading the page restores the full history rather than starting over.

2. **Read the status headline.**

   | Status | Badge | What it means |
   | --- | --- | --- |
   | `pending` | `Pending` | The job is queued and has not started |
   | `waiting` | `Waiting` | RootCoz is polling your CI server for the build to finish. Only Jenkins submissions with `Wait for build completion` enabled sit here |
   | `running` | `Running` | Fetching, cloning, or analyzing |
   | `completed` | `Completed` | Finished. The page redirects to the report |
   | `failed` | `Failed` | The job could not complete. The error is shown with its details |
   | `aborted` | `Aborted` | Cancelled by a user before it finished |

   `pending`, `running`, and `waiting` are the active states. The count of active jobs drives the pulsing badge next to `Dashboard` in the sidebar, so a single glance tells you whether the server is busy.

   A seventh badge, `Analysis Timed Out`, is derived rather than stored: when the AI call exceeded its timeout, the failed job is presented as a timeout with the hint that you can re-analyze with a longer one.

3. **Read the metadata rows.**

   | Row | Source |
   | --- | --- |
   | `JOB ID` | The ID to poll from the CLI or the API |
   | `JOB` | The display name, or the source's default name such as `file-analysis` |
   | `BUILD` | Build number or Prow build ID, linked to the CI build when a URL is known |
   | `STATUS` | The badge from the table above |
   | `MAIN AI` | Provider and model for this run |
   | `USAGE / COST` | Token usage, or `GRAFT SAVINGS` when usage is not yet available |
   | `PEERS` | One line per peer model, when peer analysis is enabled |
   | `TEST REPO` | The tests repo URL and ref, linked |
   | `QUEUED` | When the job was accepted |

4. **Follow the progress list.**

   Each phase the pipeline enters is appended to a timestamped list. The most recent entry is highlighted, and the current phase is also shown under the headline.

   | Phase | Shown as |
   | --- | --- |
   | `waiting_for_jenkins`, `waiting_for_build` | Waiting for the build to complete |
   | `fetching` | Fetching test results |
   | `agent_routing` | Routing failure groups to agents |
   | `cross_failure` | Finding cross-failure patterns |
   | `cloning` | Cloning repositories, with a per-repo expandable list showing URL, ref, and state |
   | `analyzing`, `analyzing_failures`, `analyzing_child_jobs` | Analyzing test failures, including the current group number |
   | `peer_review_round_N`, `orchestrator_revising_round_N` | Peer review or main-AI revision, by round |
   | `enriching_jira` | Searching Jira for matching bugs |
   | `saving` | Saving results |

   If a repository fails to clone, that repo is marked in red and the job continues if the remaining context is enough to analyze.

5. **Check for `Source Warnings`.**

   Non-fatal problems from the source appear in their own block, for example GCS access errors or oversized artifacts. The job still completes, so read this block before concluding that a clean-looking report means a clean fetch.

6. **Act on a stuck or unwanted job.**

   `Abort` is available while the job is `waiting`, `pending`, or `running`. Two conditions apply, and the first is easy to miss: the caller must hold **at least the `reviewer` role**, and then it must be either the user who submitted the job or an admin. Being the submitter is not sufficient on its own — if your role is later downgraded from `operator` to `viewer` while your job is still active, the button stays enabled on the status page but the request returns 403. For anyone else the button is disabled with a tooltip explaining why.

   If the job finished and you want a fresh answer, use the header button on the report page: `Analyze` for a submitted job that has not been analyzed yet, `Re-Analyze` for an analyzed one. Both open the same dialog for changing AI settings, and both are disabled while the job is still active. Re-analysis is `operator` or `admin` only. See [Reviewing and Classifying Failures](review-and-classify-failures.html) for the review workflow that follows.

7. **Understand what happens after a server restart.**

   Jobs left in `pending` or `running` cannot be recovered and are marked failed with `Analysis interrupted by server restart. Please re-submit.` Jobs in `waiting` are resumed automatically, so a Jenkins submission that was polling for its build survives a restart.

## Advanced Usage
- **Everything in flight, from the dashboard.** The dashboard lists every job with its status badge, a Mode column reading `Submitted` or `Analyzed`, failure and pass/skip counts, review progress, comment count, and the submitter. Filter by status, review state, analysis state, date range, team, tier, version, and labels. Newest rows appear without a manual refresh, so several submissions can be watched at once.

- **Check status from a script.** `--fields` limits the response to an allowlisted set of paths, which is much cheaper than fetching a full report.

  ```bash
  rootcoz results fields
  rootcoz results show JOB_ID --fields status,result.analysis_state
  rootcoz results list --analysis-state submitted
  rootcoz results analyze JOB_ID
  ```

  `rootcoz results fields` prints the allowlist for your server. Any path outside it is rejected rather than silently ignored, so scripts fail loudly when a field name is wrong. `rootcoz results list` prints `status` and `analysis_state` for every job and takes the same `--analysis-state submitted` filter as the dashboard. `rootcoz results analyze` is the CLI equivalent of `POST /results/{job_id}/analyze` and runs the AI pipeline in place on a submitted job, keeping the same job ID.

  See [CLI Command Reference](cli-reference.html) and [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html).

- **Poll the API directly.** `GET /results/{job_id}` returns the status, the result payload, and the full metadata you need to script your own checks. The same `fields` query parameter is available there. A browser hitting that URL is redirected to the status page whenever the job is `waiting`, `pending`, `running`, **or `failed`** — `ReportPage.load()` sends a `failed` job to `/status/{id}` rather than rendering a report, because there is no analysis to show. The dashboard routes `failed` and `aborted` rows to the status page too, though they differ there: an `aborted` job can still have a result and can render the report if you navigate to it directly. An `aborted` job is the one terminal state where the report renders.

- **Watch a running job for nothing else.** To answer "did anything change on this job right now", subscribe to the `results:{job_id}` stream and listen for `status-changed`. The browser app uses this, but it is the same mechanism an external tool can use.

## Troubleshooting
- **The page looks frozen and the progress list stopped growing.**
  The status page refetches only when a `status-changed` event arrives, so a dropped event stream leaves it showing the last known state. Reload the page. Nothing is lost: the progress log and the current status are both stored server-side.

- **The sidebar badge disagrees with the dashboard.**
  The badge counts active jobs server-wide for every status you can reach, while the dashboard is filtered by whatever filters you have set. Clear the filters or check another view before assuming a job is missing.

- **Only some of my open tabs update.**
  Updates are shared between browser tabs over one connection, and the connection is led by a single tab. If a tab misses events, reload it; it re-subscribes and refetches on mount.

- **A job shows `Analysis interrupted by server restart. Please re-submit.`**
  The background task did not survive the restart. Submit again, or re-analyze from the report header.

- **`Abort` is greyed out.**
  Only the submitter of that job and admins can abort it. The tooltip says the same thing.

- **`Abort` and the progress list are missing entirely.**
  The job is not in an active state. Once it reaches `completed`, `failed`, or `aborted`, the status page stops offering it and the report page takes over.

- **`rootcoz results show --fields` returns an unknown-field error.**
  The path is not on your server's allowlist. Run `rootcoz results fields` to list the valid paths and retry with those.

## Related Pages
- [Submitting Analyses](submit-analyses.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Exploring History and Reports](explore-history-and-reports.html)
- [CLI Command Reference](cli-reference.html)
- [API Reference](api-reference.html)


---

Source: review-and-classify-failures.md

# Reviewing and Classifying Failures

Use the report page when you need to decide whether RootCoz got a failure right and save the final answer your team should trust. This is where you compare the AI’s reasoning, finish human review, and correct the labels before the result drives follow-up work or downstream reporting.

## Prerequisites
- A completed analysis result.
- A signed-in account with at least the `reviewer` role, since `PUT /results/{job_id}/reviewed` requires one and a `viewer` receives `403`.
- `reviewer`, `operator`, or `admin` access to change a failure's classification or pattern.
- `operator` or `admin` access if you plan to re-analyze a job or a single failure.
- If you want to use the CLI examples, a configured CLI profile.

See [Tracking Analysis Progress](track-analysis-progress.html) for details. See [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) for details.

## Quick Example
```bash
    rootcoz results review-status JOB_ID
    rootcoz results set-reviewed JOB_ID --test "TEST_NAME" --reviewed
    rootcoz override-classification JOB_ID --test "TEST_NAME" --classification "PRODUCT BUG"
    rootcoz override-pattern JOB_ID --test "TEST_NAME" --pattern "REGRESSION"
```

Use this when you already know the job ID and test name and only need to confirm review state, mark the failure reviewed, and correct both labels. The rest of this page shows the same workflow in the web report, where you can inspect grouped failures and compare AI reasoning first.

## Step-by-Step
1. Open the result you want to review.

   Start from a completed job in the dashboard or history. If you need to find an older run or a recurring failure first, see [Exploring History and Reports](explore-history-and-reports.html) for details.

2. Triage the report header before you open individual failures.

   The header shows total failures, review progress, and the AI used for the run. If peer analysis was enabled, the `Peer Analysis` summary near the top also shows how many debates reached `Consensus` and how many did not.

3. Expand one failure card and confirm the scope.

   A card can represent one failing test or several tests with the same error. When a group contains more than one test, RootCoz shows `Affected Tests` and gives you a group-level `Review All` button.

> **Note:** Review state is tracked per test, but classification and pattern changes apply to the whole same-error group. Check `Affected Tests` before you change a grouped card.

4. Compare the current answer with the evidence.

   Start with `Analysis`, then read `Artifacts Evidence`. If the failure has been re-run before, open `Previous Analysis` to compare the earlier answer with the current one.

   If peer analysis was enabled, expand `Peer Analysis` on the failure card to inspect each round and see whether the models agreed. This is the fastest way to decide whether the current answer looks solid or needs correction.

5. Mark the review complete.

   Click `Review` on a single failure when you are satisfied with it. On grouped cards, use `Review All` when the whole group is ready, or review individual tests from the `Affected Tests` list if only part of the group is ready.

   The button changes to `Reviewed` and shows who reviewed it. Click it again if you need to reopen the failure.

6. Correct the labels when the default answer is wrong.

   Use the `Classify` row on the failure card to change the root cause and the failure pattern.

   | Change this | Choices | Use it when |
   | --- | --- | --- |
   | Root cause | `CODE ISSUE`, `PRODUCT BUG`, `INFRASTRUCTURE` | The AI blamed the wrong owner or system |
   | Pattern | `NEW`, `REGRESSION`, `FLAKY`, `INTERMITTENT`, `KNOWN BUG`, `PERSISTENT` | The root cause is right, but the failure's behavior over time is wrong |

7. Repeat the same flow inside child jobs when the report contains nested failures.

   If the result includes a `Child Jobs` section, expand the relevant job and review it the same way. Review and classification controls stay scoped to that child job instead of the top-level failure list.

8. Finish the job after everything is reviewed.

   When every failure in the result is marked reviewed and Report Portal is available, RootCoz prompts you to update Report Portal. If you need to create or link follow-up issues after the review is final, see [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html) for details.

## Advanced Usage
Use re-analysis when you want a better answer, not just a different badge. `Re-Analyze` in the page header creates a new result for the whole job, while `Re-analyze` on a failure card updates only that failure in place and keeps the earlier answer under `Previous Analysis`.

If peer analysis showed disagreement, or if the explanation is thin, re-run with different AI settings instead of stacking manual overrides. You can change the AI provider and model, peer analysis settings, tests repo, additional repositories, Jira search, artifact collection, and raw prompt. See [Configuring Analysis Context](configure-analysis-context.html) for details.

For child-job failures, add child scope in the CLI:

```bash
    rootcoz results set-reviewed JOB_ID --test "TEST_NAME" --reviewed --child-job "CHILD_JOB" --child-build 12345
    rootcoz override-pattern JOB_ID --test "TEST_NAME" --pattern "KNOWN_BUG" --child-job "CHILD_JOB" --child-build 12345
```

> **Tip:** The UI label `KNOWN BUG` uses the CLI value `KNOWN_BUG`.

Use `--not-reviewed` instead of `--reviewed` when you need to reopen a failure from the terminal. See [CLI Command Reference](cli-reference.html) for details.

The `AI Spend` section of the report page attributes what the job spent: a breakdown by stage (primary, peer, revision, agent routing, cross-failure), failed-call cost, primary cost per unique failure group with the number of tests sharing it, and peer cost per debate round and agent. Cache tokens are shown apart from input/output because they are not billed the same way. Anything the server cannot attribute — cost it never recorded, or usage from an older run — is reported as unavailable or `N/A`, never as zero, so hover any average to see the exact denominator.

## Troubleshooting
- `I can open the report but my changes are rejected.`
  Changing the root cause or pattern needs `reviewer` access or above (`Reviewer access required. Viewers cannot perform this action.`), and re-analysis needs `operator` or `admin` (`Operator access required.`).

- `My change affected several tests.`
  That card represents a same-error group. RootCoz applies classification and pattern changes across the whole group, so confirm the `Affected Tests` list before you save.

- `The label changed, but the explanation still needs work.`
  Overrides update the labels immediately, but they do not replace the full AI narrative. Use `Re-analyze` if you want a fresh explanation that matches your corrected label.

- `A section disappeared after I changed the classification.`
  That is expected. Switching to `CODE ISSUE` removes old bug-report content, switching to `PRODUCT BUG` removes old suggested-fix content, and switching to `INFRASTRUCTURE` removes both.

- `I never see the Report Portal prompt.`
  The `All failures reviewed. Update Report Portal?` prompt only appears when every failure in the job is reviewed and Report Portal is enabled for the server.

## Related Pages

- [Tracking Analysis Progress](track-analysis-progress.html)
- [Collaborating on Results](collaborate-on-results.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [Exploring History and Reports](explore-history-and-reports.html)
- [Configuring Analysis Context](configure-analysis-context.html)


---

Source: collaborate-on-results.md

# Collaborating on Results

Use per-job chat and comments when you need to understand one analysis together with other people: ask the AI questions about a specific result, leave context for the next reviewer, and pull teammates into the discussion with `@mentions`. Both features live on a single result page and are scoped to one job.

> **Note:** This page covers chat on one result. For questions that span many jobs — trends, review accuracy, cross-job reporting — use the admin-only Server Chat. See [Use Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html) for details.

## Prerequisites

- A completed analysis result. See [Reviewing and Classifying Failures](review-and-classify-failures.html) for details.
- `reviewer`, `operator`, or `admin` access. Both chat and posting comments are gated at reviewer level.
- An AI provider and model you can select when you start a chat, either your own key or server credentials.
- Optional: a `GitHub Token` and Jira credentials on your profile if you want the assistant to search GitHub issues and Jira tickets. See [Managing Your Account and Notifications](manage-account-and-notifications.html) for details.
- For the CLI examples, an authenticated CLI profile. See [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) for details.

## Quick Example

```bash
    rootcoz chat init JOB_ID --provider claude --model claude-opus-4-6
    rootcoz chat send JOB_ID "Which failures share the same error signature?"
    rootcoz comments add JOB_ID --test "tests.test_auth.test_login" -m "Reproduced locally, see PROJ-1234."
```

`chat init` starts the session and is required: `chat send` needs a provider and model selected and returns `409 Start a new chat session first` until you do. Both `--provider` and `--model` are mandatory options on `chat init` — omitting either fails argument validation before the session is created. Both commands need at least the `reviewer` role.

This asks the job's assistant for a cross-failure answer and leaves the evidence in the result's comment thread for the next reviewer.

## Step-by-Step

1. Open the result and start the chat.

   From the report header, click `Chat`. The button is hidden for viewers, and the chat page opens at `/chat/{job_id}` for the job you came from. Pick a provider and model in the header, then click `Start Chat`. `Use server credentials` chooses whose AI key runs the session; the toggle is only available when your account may use server providers.

   Starting a chat creates a per-user, per-job workspace and session. RootCoz clones the job's tests repo and any additional repositories, then copies CI build data into that workspace when the source provides it.

   > **Note:** Chat history and sessions are scoped to your own account on that job. Two reviewers opening the same result get two separate conversations, and RootCoz pins the provider, model, and credential source you started with for the life of the session.

2. Check what the assistant can reach before you ask.

   The welcome message at the top of a new session lists the resources available for that job, and the assistant's working context is this one job's result and comments. Its history tools are the exception: `get_failure_history` and `get_classification_history` query by test name across the server and exclude only the current job, so the assistant can pull how a test has failed in *other* jobs. It has read-only filesystem tools (`read`, `ls`, `find`, `grep`, `subagent`) plus authenticated HTTP tools:

   | Tool | What it returns |
   | --- | --- |
   | `get_job_result` | The full result: failures, classifications, AI analysis |
   | `get_job_comments` | Comments and review discussion on this job |
   | `get_job_tests` | Paginated test entries (`passed`, `skipped`, `failed`) |
   | `get_failure_history` | Pass/fail history for one test, plus prior classifications |
   | `get_classification_history` | Who changed a test's classification, when, and from what to what |
   | `search_jira` / `get_jira_issue` | Jira search and issue lookup — only when your account has Jira credentials |
   | `search_github_issues` / `get_github_issue` | GitHub issue and PR search in the test repository — only when your account has a GitHub token |

   When the workspace has source or CI data, the assistant can also read the cloned repositories and the build files `console-output.txt`, `build-info.json`, and `build-artifacts/`.

   > **Warning:** The assistant has no shell and cannot write or edit files. It reads what RootCoz gave it and calls only the endpoints listed above; it will not execute commands or make changes on your behalf.

3. Ask your question.

   Type in the `Ask a question...` box and press `Enter` to send, `Shift+Enter` for a newline. While the answer is being generated the thread shows `Thinking...` and a `Stop` button. Replies stream into the same conversation and are saved, so you can close the page and come back later.

   > **Tip:** Because the assistant can read the cloned test code, questions like `Why does test_login assert on the old token format?` are often answered faster than asking for a re-analysis with a different model.

4. Comment on the failure, not just in the chat.

   Expand a failure card and use the `Add a comment...` box, then click `Post`. Comments are attached to the specific test — including grouped cards, where the comment covers the group you clicked — and they persist independently of any chat session.

   Type `@` in the comment box to get a list of usernames you can mention. A mentioned teammate gets a navbar badge and unread entry in `Mentions`; if web push is configured on the server, they also get a push notification.

   Jira keys and GitHub PR links pasted into a comment are enriched with their live status as a badge next to the link.

   > **Note:** After you post, RootCoz checks whether the comment suggests the failure is already resolved. If it does, it asks `Mark as reviewed?` and you confirm. That suggestion never marks anything on its own.

5. Delete or clear when you are done.

   Comments show a delete control to their author; admins can delete any comment. For chat, `New Session` clears your messages, session, and cloned repositories for that job so you can start again with a different model.

   > **Warning:** `New Session` deletes your chat history for that job. It is per-user, so it does not touch a colleague's conversation.

6. Use the same workflow from the CLI.

   ```bash
   rootcoz chat init JOB_ID --provider claude --model claude-opus-4-6
   rootcoz chat send JOB_ID "Which failures are infrastructure rather than code issues?"
   rootcoz chat history JOB_ID --limit 20
   rootcoz chat abort JOB_ID
   rootcoz chat clear JOB_ID
   rootcoz chat close JOB_ID

   rootcoz comments list JOB_ID
   rootcoz comments add JOB_ID --test "tests.test_auth.test_login" -m "Fixed in PROJ-1234."
   rootcoz comments delete JOB_ID 42
   ```

   `chat send` waits for the assistant reply and prints it; `chat close` tells the server you left the page, while `chat clear` removes the conversation.

## Advanced Usage

Check your mention inbox and tracker status from the terminal:

```bash
    rootcoz mentionable-users
    rootcoz mentions --unread --limit 20
    rootcoz mentions-mark-read --ids 91,92
    rootcoz mentions-mark-all-read
    rootcoz results enrich-comments JOB_ID
```

`enrich-comments` refreshes the live status badges for Jira tickets and GitHub PRs found in that job's comments, and `analyze-comment-intent` returns the reviewed/not-reviewed judgment behind the `Mark as reviewed?` prompt:

```bash
    rootcoz analyze-comment-intent "Fixed by updating the token fixture." --job-id JOB_ID
```

These prompts match the tools the job assistant actually has:

| Goal | Example question |
| --- | --- |
| Group related failures | `Which failures share this error signature?` |
| Check flakiness | `Has tests.test_auth.test_login failed in previous runs of this job?` |
| Review the discussion | `What did reviewers say about the failures we marked reviewed?` |
| Trace the change | `Which code in the test repo asserts on the old token format?` |
| Check for existing work | `Is there an open Jira issue for this product bug?` |

Use `--json` on any of these commands when you want machine-readable output. For every flag, see [CLI Command Reference](cli-reference.html) for details. For the raw endpoints, see [API Endpoint Reference](api-reference.html) for details.

## Troubleshooting

- The `Chat` button is missing from the report header.
  Chat requires `reviewer` access. Viewers can read results and comments but cannot start or use chat.

- `Start Chat` fails with `Chat session unavailable.`
  The session could not be created. Clear the chat with `New Session` and start again, and check that the provider and model are still available for your credential source.

- The assistant says Jira or GitHub search is not available.
  Those tools are only registered when your account has the matching credentials. Add them on your profile settings page, then start a new chat session so the new tool list is picked up. See [Managing Your Account and Notifications](manage-account-and-notifications.html) for details.

- The assistant refuses to answer your question.
  Job chat is scoped to that one analysis. Off-topic questions are rejected by design. For cross-job or server-wide questions, use [Use Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html) for details.

- My teammate is not notified about an `@mention`.
  The navbar badge and `Mentions` list always update for signed-in users. Browser push notifications additionally require web push to be configured on the server.

- I cannot delete a comment someone else wrote.
  Deleting is limited to the comment author; admins can delete any comment.

## Related Pages

- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Use Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [Managing Your Account and Notifications](manage-account-and-notifications.html)
- [CLI Command Reference](cli-reference.html)


---

Source: create-follow-up-issues-and-push-results.md

# Creating Follow-Up Issues and Pushing Results

Turn a reviewed failure into real follow-up work: file a GitHub issue or Jira ticket from the report, link a failure to a ticket that already exists, and push the finished classifications into Report Portal. Every action here starts from a result page, and the issue content is generated from the same AI analysis you already reviewed.

## Prerequisites

- A completed analysis result, ideally with its failures reviewed. See [Reviewing and Classifying Failures](review-and-classify-failures.html) for details.
- A `GitHub Token` or Jira credentials on your own profile. Issue creation always uses your token, never a stored server credential. See [Managing Your Account and Notifications](manage-account-and-notifications.html) for details.
- `reviewer` or above to link an existing issue with `Track`.
- `operator` or `admin` to push results to an exporter. Push is rejected for reviewers.
- Report Portal enabled on the server (`ENABLE_REPORTPORTAL`, with `REPORTPORTAL_URL`, `REPORTPORTAL_API_TOKEN`, and `REPORTPORTAL_PROJECT`) if you want to push classifications. See [Configuration Reference](configuration-reference.html) for details.
- For the CLI examples, an authenticated CLI profile. See [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) for details.

## Quick Example

```bash
    rootcoz preview-issue JOB_ID --test "tests.test_auth.test_login" --type github --include-links
    rootcoz create-issue JOB_ID --test "tests.test_auth.test_login" --type github \
      --title "Login fails after token refresh" --body "$(cat issue-body.md)"
    rootcoz exporters
    rootcoz push JOB_ID --plugin reportportal
```

This generates and reviews issue content, creates the issue, confirms which exporters are available, and pushes the reviewed classifications.

## Step-by-Step

1. Finish the review first.

   Issue content and pushed classifications are generated from the current analysis, so correct the labels before you file anything. The `Create` action on a failure card also offers `Mark as reviewed?` right after the issue is created.

2. Generate a preview from the failure card.

   Expand a failure, then use `GitHub Issue` or `Jira Ticket` in the `Actions` row. RootCoz opens the issue prompt, which comes from the `issue_prompt` stored with the job or from `.rootcoz/ROOTCOZ_ISSUE_PROMPT.md` in the test repository. Edit it if you want to steer the title and body, then click `Continue`.

   RootCoz calls `POST /results/{job_id}/preview-github-issue` or `POST /results/{job_id}/preview-jira-bug`, which draft a title and body, and search for issues that look like duplicates of the same bug.

   `Include links` adds clickable report and build URLs. `AI for issue generation` lets you pick a different provider or model for the draft. The preview also lists any similar issues it found, so you can link existing work instead of filing a duplicate.

   > **Note:** `Include links` only has an effect when the server has a public base URL configured, because relative links would resolve against the tracker instead of RootCoz. See [Configuration Reference](configuration-reference.html) for details.

3. Set the Jira specifics when you file a Jira ticket.

   In the preview step, pick or type the `Jira Project` (the server searches your projects as you type), choose an `Issue Type` from `Bug`, `Task`, `Story`, `Epic`, `Sub-task`, or `Custom...`, and optionally pick a `Security Level`. A project key must be available on the server or in the request, otherwise ticket creation fails.

   GitHub issues need a repository: RootCoz always uses the deployment's `TESTS_REPO_URL`. It deliberately ignores the job's additional repositories, so you cannot retarget issue creation at another repo — a job's `request_params` are never consulted for this. To file against a different repository, change `TESTS_REPO_URL` on the server (or the `GITHUB` category in Server Settings) first.

4. Create the issue and let RootCoz record the link.

   Edit the `Title` and `Body` if the draft is not right, then click `Create GitHub Issue` or `Create Jira Ticket`. On success RootCoz automatically:

   - adds a comment on the failure containing the new issue link, and
   - adds a tracked-in link for that issue, shown as a `GitHub` or `Jira` badge under the failure, which you can remove later like any other tracked link.

   Creation uses your own credentials. A missing or expired token returns an error that points back to your profile settings.

   > **Warning:** GitHub and Jira issue creation can be disabled per server with `ENABLE_GITHUB_ISSUES` and `ENABLE_JIRA_ISSUES`. When disabled, the buttons are greyed out and the endpoints refuse the request.

5. Link a failure to an issue that already exists.

   Use the `Track` button on the `Classify` row, paste the `Issue URL` into the `Track In` dialog, and click `Save`. RootCoz detects `GitHub` or `Jira` from the URL itself and shows the detected type before you save. You can link several issues to the same failure.

   Creators can remove their own links; admins can remove any link.

6. Review what Jira search already found during analysis.

   When Jira is configured, analysis for a `PRODUCT BUG` already searched for related tickets before you opened the report:

   1. The AI generates search keywords as part of the failure analysis.
   2. Identical keyword sets are deduplicated into one Jira search, and the searches run in parallel.
   3. Each candidate is scored for relevance by the AI.
   4. Only relevant matches are attached to the result, under `Matching Jira Issues` in the `Bug Report` section, with their status.

   Jira problems never break an analysis. Search failures are logged and swallowed, so a result without matching tickets usually means the lookup failed or found nothing relevant — not that the analysis is wrong.

7. Push reviewed classifications to Report Portal.

   Click `Push to Report Portal` in the report header, confirm, and RootCoz matches the job to its Report Portal launch, updates the matching test items, and reports how many classifications were pushed, plus any unmatched items and errors.

   You can also check what is available before pushing:

   ```bash
   rootcoz exporters
   ```

   The list shows each exporter plugin with its display name and whether it is ready. Report Portal is reported as disabled when the integration is unconfigured, when every push-content toggle is off, or when `PUBLIC_BASE_URL` is missing while the rootcoz-URL toggle is on.

   Report Portal push content is controlled by three toggles:

   | Toggle | Effect when enabled |
   | --- | --- |
   | `RP_PUSH_CLASSIFICATIONS` | Maps RootCoz classifications to Report Portal defect types |
   | `RP_PUSH_ROOTCOZ_URL` | Adds the RootCoz report URL as a comment on the test item |
   | `RP_PUSH_TRACKER_LINKS` | Adds linked GitHub and Jira issues as external system issues |

   At least one of the three must be enabled or the push is refused.

## Advanced Usage

Push through the generic exporter endpoint when you want the plugin dispatch rather than the Report Portal shortcut. `POST /results/{job_id}/push/{plugin_name}` takes the same optional `child_job_name` and `child_build_number` parameters and returns the exporter's own result fields plus `success` and `message`. The Report Portal-specific endpoint `POST /results/{job_id}/push-reportportal` remains available and is what the UI button uses.

Let the pipeline push for you. When `AUTO_PUSH_EXPORTERS` is set to a comma-separated plugin list and auto-review is enabled, RootCoz pushes to those exporters once every failure in the job is reviewed, recording the push under the reserved `rootcoz-ai` identity. An empty `AUTO_PUSH_EXPORTERS` disables this, and auto-push failures are logged rather than surfaced in the UI. This is a server-only setting, so the Server Settings page cannot change it — saving it there is rejected with `400 Server-only settings (env var only)`. Set `AUTO_PUSH_EXPORTERS` in the deployment environment, and note that the value is read into the server's settings cache at startup: changing it on a running process does not take effect until the settings are rebuilt or the server restarts.

Create issues from the terminal, including for nested child-job failures:

```bash
    rootcoz get-issue-prompt JOB_ID
    rootcoz preview-issue JOB_ID --test "TEST_NAME" --type jira --jira-project-key PROJ
    rootcoz create-issue JOB_ID --test "TEST_NAME" --type jira \
      --title "Short summary" --body "What failed, and where it was reproduced." \
      --jira-project-key PROJ --jira-issue-type Bug --jira-security-level "Internal"
    rootcoz create-issue JOB_ID --test "TEST_NAME" --type github \
      --title "Short summary" --body "What failed, and where it was reproduced." \
      --child-job "CHILD_JOB" --child-build 12345
    rootcoz push-reportportal JOB_ID --child-job-name "CHILD_JOB" --child-build-number 12345
    rootcoz push JOB_ID --plugin reportportal --json
```

Tracker flags fall back to the values in your CLI server profile, so pass them explicitly only when you need to override. See [CLI Command Reference](cli-reference.html) for details.

Send feedback about RootCoz itself through the `Feedback` button in the application header. You describe the problem, RootCoz collects browser context and recent console and API errors automatically, then drafts a GitHub issue you can edit before it is created. This flow is separate from failure issues and requires feedback to be enabled on the server.

For raw endpoints, see [API Endpoint Reference](api-reference.html) for details.

## Troubleshooting

- The `GitHub Issue` or `Jira Ticket` button is greyed out.
  Issue creation is disabled for this server, or the job was analyzed without a repository or Jira project to target. Ask an administrator to check `ENABLE_GITHUB_ISSUES` and `ENABLE_JIRA_ISSUES`.

- Creation fails with `GitHub token is required` or `Jira token is required`.
  Issue creation uses only your own credentials. Add the token on your profile settings page and try again.

- The preview has no `Include links` effect.
  The server has no public base URL configured. Set `PUBLIC_BASE_URL` so report links can be absolute.

- No `Matching Jira Issues` appear on a product bug.
  The search may have found nothing, or the lookup failed. Failures are logged and ignored, so the analysis result is still valid.

- `Exporters` shows Report Portal as disabled.
  Run `rootcoz exporters` and check the reason: the integration is off, all three `RP_PUSH_*` toggles are off, or `PUBLIC_BASE_URL` is missing while `RP_PUSH_ROOTCOZ_URL` is on.

- The push returns `Operator access required.`
  Exporter pushes need `operator` or `admin`. Reviewers can create and link issues, but not push.

- The push reports unmatched items.
  RootCoz matched the launch but could not pair some failures with Report Portal test items. Check the reported item names against the launch, and confirm the job name and build number match the Report Portal launch.

## Related Pages

- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Collaborating on Results](collaborate-on-results.html)
- [Managing Your Account and Notifications](manage-account-and-notifications.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [CLI Command Reference](cli-reference.html)


---

Source: explore-history-and-reports.md

# Exploring History and Reports

Use RootCoz's history views when you need to answer two practical questions quickly: has this failure happened before, and is it part of a broader trend? These screens help you move from one test failure to per-test history and then out to filtered team summaries.

## Prerequisites
- You are signed in to RootCoz.
- At least one analysis result already exists. See [Submitting Analyses](submit-analyses.html) for details.
- Your account has access to `Reports` if you want the team summary views.

## Quick Example
```bash
    rootcoz history failures --search tests.test_auth.test_login
    rootcoz history test "tests.test_auth.test_login"
    rootcoz reports totals --team alpha --from 2025-01-01 --to 2025-06-01 --review-status reviewed
```

Use the first command to find matching failures, the second to inspect one test over time, and the third to compare reviewed work for one team in a date window.

## Step-by-Step
1. Start in the right place.

| If you want to... | Open | Best for |
| --- | --- | --- |
| Check one failing test or result | `History` | Repeats, classifications, and recent runs |
| Compare filtered job totals across teams or releases | `Reports` | Dates, metadata, labels, and review progress |
| See where reviewers changed RootCoz's answer | `Reports` | Classification override patterns |
| See what follow-up work already exists | `Reports` | GitHub and Jira issue summaries |

2. Search recurring failures in `History`.
Open `History`, use the search box to narrow the table by test name, then add a classification filter or date range if the list is still noisy. Click a row to open that result, or click the test name itself to open that test's dedicated history page.

3. Read the test history page.
Start with `Failure Rate`, `Total Runs`, `Failures`, and `Consecutive` to understand whether the test is stable, degrading, or repeatedly broken. Then read the classification badges, comments, `First seen`, and `Last seen`, and open any row in `Recent Runs` to jump back to a specific result.

4. Move to `Reports` for broader trends.
Choose `Total Failures` when you want summary counts, `Classification Overrides` when you want review changes, or `Issues Created` when you want follow-up visibility. Use the filters across the top — `All teams`, `All tiers`, `All versions`, `All statuses`, `Review:`, the date range picker, and the `Tags:` include/exclude chips — to narrow the slice.

> **Note:** If you leave the status filter empty, report summaries start with completed jobs only.

5. Read the report detail that matches your question.
In `Total Failures`, expand `Job Details` to compare jobs in the current slice. In `Classification Overrides`, expand a transition group such as `CODE ISSUE → PRODUCT BUG` to see which tests changed, on which axis (`Root Cause` or `Pattern`), and who changed them; in `Issues Created`, open the external issue link or the linked result to inspect the original analysis.

When you find a specific result you want to confirm or correct, see [Reviewing and Classifying Failures](review-and-classify-failures.html) for details.

## Advanced Usage
Use the CLI when you want repeatable lookups or want to save filters in shell history.

```bash
    rootcoz history failures --classification "INFRASTRUCTURE" --limit 50
    rootcoz history test "tests.test_auth.test_login" --limit 50
    rootcoz history stats "test-job"
    rootcoz reports overrides --status completed --tags nightly,smoke
    rootcoz reports issues --team alpha --review-status reviewed
```

> **Tip:** The `Reports` page keeps its active tab and filters in the URL, so you can refresh, bookmark, or share the same filtered view.

See [CLI Command Reference](cli-reference.html) for all flags, or see [API Endpoint Reference](api-reference.html) if you want to script the same lookups.

## Troubleshooting
- I do not see `Reports` in the sidebar.
  Reports access is a separate flag from your role. Ask an administrator to grant reports access, then reload the page. Admins always have it.

- `History` is empty for a test I expect to find.
  Remove the classification filter, widen the date range, or search with the full test name.

- A report looks smaller than expected.
  Add the statuses you care about, then re-check your team, version, and label filters.

- There is no data to explore yet.
  Run an analysis first. See [Submitting Analyses](submit-analyses.html) for details.

## Related Pages

- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [Submitting Analyses](submit-analyses.html)
- [Use Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html)
- [CLI Command Reference](cli-reference.html)

---

Source: use-server-chat-for-cross-job-analysis.md

# Use Server Chat for Cross-Job Analysis

Ask RootCoz questions across all jobs when you need trends, summaries, or reusable reports without opening runs one by one. This guide shows administrators how to start a server-wide chat, switch models for one question, follow replies, and keep or clear the session on their own terms.

## Prerequisites

- An admin account in RootCoz. The `Chat` page in the admin sidebar is admin-only.
- A default AI provider and model configured on the server, or a provider and model you can choose before sending a message. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.
- For CLI use, the `rootcoz` CLI installed and authenticated with an admin API key. See [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html) for details.

## Quick Example

```bash
    export ROOTCOZ_SERVER="https://rootcoz.example.com"
    read -rsp "Admin API key: " ROOTCOZ_API_KEY
    export ROOTCOZ_API_KEY
    echo

    rootcoz auth whoami
    rootcoz admin-chat init --provider claude --model claude-opus-4-6
    rootcoz admin-chat send "Summarize the top failure trends from the last 7 days."
```

This confirms the CLI is using an admin identity, then asks a server-wide question and waits for the assistant reply.

> **Note:** Server Chat is separate from chat on a single analysis result. For job-specific chat and comments, see [Collaborating on Results](collaborate-on-results.html) for details.

## Step-by-Step

1. Open Server Chat or confirm CLI admin access.

   In the web UI, open `Chat` from the admin section of the sidebar. The empty state prompts you to `Ask about server analytics`, which is the right place for cross-job questions such as failure trends, user activity, or report summaries.

   In the CLI, run this first when you want to confirm which admin account the shell is using:

   ```bash
   rootcoz auth whoami
   ```

2. Choose how RootCoz should pick the model.

   RootCoz can use the server default AI settings or a one-off override for the next question.

   | If you want | Web UI | CLI |
   | --- | --- | --- |
   | Use the server default | Leave the provider and model already loaded in the header. | Omit `--provider` and `--model`. |
   | Override for one question | Pick a different provider and model in the header before sending. | Add `--provider` and `--model` to `rootcoz admin-chat send`. |
   | Keep using the same override | Leave the header selection in place. | Repeat the same flags on the next `send` command. |

   If you need to change the shared defaults for everyone, see [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

3. Ask a cross-job question and follow the reply.

   In the web UI, type your question and press `Enter` to send it. Use `Shift+Enter` when you want a newline instead. While RootCoz is working, the thread shows `Thinking...` and keeps the response in the same conversation.

   In the CLI, send the question directly:

   ```bash
   rootcoz admin-chat send "Show where humans overrode the AI most often this month."
   ```

   The CLI waits for the assistant reply when it can. If you want to come back later, or the response takes longer than expected, check the stored conversation:

   ```bash
   rootcoz admin-chat history --limit 20
   ```

   > **Tip:** Use the web UI when you want the clearest live experience. It shows the active reply immediately and gives you a `Stop` button while the assistant is still working.

4. Stop the current reply or start over.

   Use the web UI `Stop` button when a reply is still running and you want to cancel it. When you want a completely fresh conversation, click `New Session` in the web UI or clear the current chat from the CLI:

   ```bash
   rootcoz admin-chat clear
   ```

   `New Session` and `clear` both reset your current Server Chat history so the next question starts clean.

   > **Note:** Server Chat history and saved artifacts are scoped to the current admin account, not shared across all admins.


   > **Warning:** Starting a new session in the web UI or running `rootcoz admin-chat clear` in the CLI also removes saved report artifacts for your account.

5. Save or download HTML report artifacts.

   Use artifacts when you want a reusable HTML summary that can be downloaded later. The CLI can upload a local HTML file into the Server Chat artifact store and download it back by ID.

   ```bash
   rootcoz admin-chat save-artifact ./weekly-summary.html
   rootcoz admin-chat save-artifact ./weekly-summary.html --filename "failure-summary-2026-07-31.html"
   rootcoz admin-chat download-artifact <artifact-id> --output ./failure-summary.html
   ```

   `save-artifact` returns a download URL. `download-artifact` writes the file to the path you choose, and when you do not pass `--output`, RootCoz creates a local HTML filename automatically.

   In the web UI, when a Server Chat reply contains one of these artifact links, RootCoz renders it as a download button instead of a raw URL.

## Advanced Usage

Use one-off model overrides when you want to compare answers without changing the server default:

```bash
    rootcoz admin-chat send \
      "Compare issue-creation trends for the last 30 days." \
      --provider claude \
      --model claude-opus-4-6
```

Use JSON history output when you want to hand the conversation to another script or save a machine-readable record:

```bash
    rootcoz --json admin-chat history --limit 50
```

Use custom filenames when you are saving reports for teammates or recurring reviews:

```bash
    rootcoz admin-chat save-artifact ./report.html --filename "platform-weekly-review"
```

If the filename does not end in `.html`, RootCoz adds the extension for you.

These prompt patterns work well in Server Chat because they match the built-in cross-job reporting tools:

| Goal | Example prompt |
| --- | --- |
| Failure trends | `Summarize the top failure patterns from the last 7 days.` |
| Review accuracy | `Show where reviewers changed the AI classification most often this month.` |
| Follow-up volume | `Which jobs created the most GitHub issues or Jira bugs this quarter?` |

For every available command and flag, see [CLI Command Reference](cli-reference.html) for details. For raw endpoints, see [API Endpoint Reference](api-reference.html) for details.

## Troubleshooting

- `The send button stays disabled in the web UI.`
  The chat session only starts once a valid provider and model pair is selected in the header. Pick both, type a message, and wait for any in-flight reply to finish.

- `rootcoz admin-chat send` fails with an admin error, or returns `409 Start a new chat session first`. The 409 means no session exists yet — run `rootcoz admin-chat init` first, which needs `--provider` and `--model`. A genuine admin error is a different failure: check the caller's role.
  The CLI is not using an admin identity. Confirm it with `rootcoz auth whoami`.

- Server Chat says AI is not configured, or the page never finishes initializing.
  Set the default AI provider and model in Server Settings. See [Managing Users and Server Settings](manage-users-and-server-settings.html) and [Configuration Reference](configuration-reference.html) for details.

- `A saved artifact link returns 404.`
  You started a new session or ran `rootcoz admin-chat clear`, which deletes saved artifacts for your account.

- `The CLI did not show a final answer.`
  `admin-chat send` waits up to about two minutes. Check the result later with `rootcoz admin-chat history --limit 20`. For the raw endpoints, see [API Endpoint Reference](api-reference.html) for details.

## Related Pages

- [Exploring History and Reports](explore-history-and-reports.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [Collaborating on Results](collaborate-on-results.html)
- [CLI Command Reference](cli-reference.html)
- [API Endpoint Reference](api-reference.html)

---

Source: manage-account-and-notifications.md

# Managing Your Account and Notifications

This page covers everything you control about your own RootCoz identity: registering, the API key you log in with, the personal tracker tokens and AI keys stored against your account, and how you get told when someone mentions you in a comment.

## Prerequisites

- A running RootCoz server. See [Quickstart](quickstart.html) or [Deploying RootCoz](deploy-rootcoz.html) for details.
- For the tracker tokens, a GitHub personal access token with the `repo` scope, or a Jira Cloud API token (plus your Atlassian account email) or a Jira Server/Data Center personal access token.
- For browser push notifications, a browser that supports Web Push (Chrome, Edge, Firefox, or Safari on a supported platform) and permission to show notifications.

## Quick Example

1. Open `http://localhost:8000` and click `Register`.
2. Enter a username, click `Register`, then copy the API key from the `Registration Complete` screen.
3. Open `Settings`, paste a `GitHub Token`, and click `Save`.
4. Under `Push Notifications`, click `Enable` to receive browser notifications when you are mentioned.

The registration key is the only credential that signs you in; the tracker token is what lets RootCoz create GitHub or Jira issues under your name.

## Step-by-Step

1. **Register an account.**

   Registration takes a username and nothing else — there is no password. Usernames are lowercased before storage, so `JDoe` and `jdoe` are the same account.

   | Rule | Behaviour |
   | --- | --- |
   | Reserved names | Any username starting with `rootcoz` is rejected. This protects the reserved AI identity `rootcoz-ai`, which RootCoz uses for its own comments and chat. |
   | Allowed users | If the server sets `allowed_users`, registration fails with `Registration is restricted. Contact an admin.` unless your username is on the list. |
   | New role | The account gets `DEFAULT_USER_ROLE` (`reviewer` unless the server changes it). |
   | Approval | With `REQUIRE_APPROVAL=true` the account is created as `pending` and cannot log in until an admin approves it. |

   You can optionally supply `GitHub Token`, `Jira Email`, and `Jira Token` on the registration form. They are saved with the account and reused on the `Settings` page.

2. **Save the API key.**

   Registration returns the key once, on the `Registration Complete` screen, under the warning `Save this API key — you won't see it again!`. Click `Copy`, store it in a password manager, then click `I've saved my key — Continue`.

   | Detail | Value |
   | --- | --- |
   | Key format | `rootcoz_` followed by a random token |
   | Storage on the server | Only an HMAC-SHA256 hash, so the raw key cannot be recovered |
   | Reused for | Logging in and as the API credential for the CLI |

   > **Warning:** The key is displayed exactly once. If you lose it, an admin has to generate a new one for you. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

3. **Log in and manage sessions.**

   `Log in` takes the same username plus the API key. On success RootCoz sets an HTTP-only `rootcoz_session` cookie; `Log out` deletes the session server-side and clears the cookie.

   | Session behaviour | Detail |
   | --- | --- |
   | Lifetime | `SESSION_TTL_HOURS`, default 720 hours (30 days) |
   | Cookie flags | `httponly`, `samesite=lax`, and `secure` when the server sets `secure_cookies` |
   | Key rotation | Invalidates every session for that username, then issues a fresh session so you stay signed in |
   | `can_view_reports` | Re-read from the users table on each request, so an admin grant takes effect without signing in again |

   Behind a trusted identity proxy the server can also accept an `X-Forwarded-User` header instead of a key; that behaviour is controlled by the `trust_proxy_headers` server setting.

4. **Keep your profile and tracker tokens current.**

   `Settings` shows your username as read-only, followed by an `API Key` section, `Tracker Tokens`, and `Push Notifications`.

   | Field | What it is used for |
   | --- | --- |
   | `GitHub Token` | Personal access token with `repo` scope. Used to create GitHub issues. It is **not** what enriches comment badges: `enrich_comments` reads the server's `GITHUB_TOKEN` from Server Settings, so status badges resolve against the server-wide credential, not yours. |
   | `Jira Email` | Your Atlassian account email, required for Jira Cloud API-token auth. |
   | `Jira Token` | Jira Cloud API token, or a personal access token on Jira Server/Data Center. |

   Validation happens through `POST /api/validate-token`, and a failed validation leaves the stored value unchanged. The two entry points differ, though. The Settings form (`ProfileForm.handleSubmit`) validates any non-empty GitHub or Jira token that is not already known-good and **aborts the save** if the check fails, so a bad token typed there is never stored. The registration flow does not: `RegisterPage` sends whatever you typed straight to `PUT /api/user/tokens`, which stores it as supplied. A mistyped or already-revoked token entered at registration therefore persists and only fails later, when a tracker action tries to use it. They are kept in your browser for convenience and synced to the server encrypted at rest, so a second browser picks them up automatically.

   > **Clearing a token:** clear the field and save. An empty field in the body of `PUT /api/user/tokens` clears that one credential on the server; the fields you do not send keep their stored values, so clearing your GitHub token leaves your Jira credentials alone. Sending `null` for a field clears it the same way an empty string does, and a request that omits every field changes nothing and reports `"saved": false`. There is no separate `DELETE` route — an empty or `null` field is the clear.

   > **Tip:** Without a tracker token you can still preview generated issue content, but you cannot submit it. See [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html) for details.

5. **Rotate your own API key.**

   In `Settings`, under `API Key`, click `Rotate API Key`. The old key stops working immediately, all your other sessions are signed out, and the new key is shown once with a `Copy` button.

   Two things to know before you rotate:

   - The bootstrap `admin` account cannot rotate through this page. Its key is the `ADMIN_KEY` environment or server setting.
   - Any script or CLI config using the old key breaks until you update it. See [CLI Command Reference](cli-reference.html) for details.

6. **Store your own AI provider keys (reviewer and above).**

   If your role is not `viewer`, `Settings` also shows an `AI provider keys` card. Pick a provider, pick the model to verify against (type the model ID by hand if it is not in the list), paste the key, and save.

   | Outcome | What happens |
   | --- | --- |
   | `accepted` | Key stored for that provider; existing keys are never shown again |
   | `rejected` | Existing credential left unchanged; check the key and the selected model |
   | `inconclusive` | Provider could not be reached; retry when it is available |

   Use `Replace` to swap a key for a provider that already has one, or `Remove key` to delete it — removing a key also revokes the AI sessions that used it. Keys are limited to 8–1024 characters and are stored encrypted on the server.

   > **Note:** The `Use server credentials` toggle on `New Analysis` is a separate, admin-controlled permission. Without that grant you must bring your own provider key. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

7. **Get notified when you are mentioned.**

   Mentions are plain `@username` text in job comments. RootCoz scans each new comment for `@username` tokens, ignoring anything that looks like an email address, and the mentioned user gets an unread badge in the sidebar.

   | Where | What it does |
   | --- | --- |
   | `Mentions` page | Lists every comment that mentions you, newest first, with an unread count and `Mark all as read`. Opening a mention marks it read and jumps to the exact comment on the report. |
   | Sidebar badge | Live unread count, pushed over the `/api/navbar/stream` SSE channel. |
   | `Push Notifications` toggle | Browser notifications titled `Mentioned by @author`. The notification link is built as `/results/{job_id}`, so a tap opens the report for that job. Use the `Mentions` page or the sidebar badge until that is corrected. |

   Enabling push notifications asks the browser for permission and registers the device with the server. Delivery is best effort: a subscription the push service reports as gone is deleted automatically, and at most 10 subscriptions are kept per user.

   > **Note:** Push notifications need VAPID keys. RootCoz generates a key pair automatically when none are configured, so the toggle normally works out of the box. An administrator can also set them under `Web Push` in Server Settings. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

## Advanced Usage

- **Check who can be mentioned.** The mention picker on a comment lists existing usernames. Mention detection is word-boundary aware, so `@jdoe` does not match `@jdoe_extra`.
- **Self-mentions are not pushed.** If you mention yourself in a comment, no browser notification is sent to you.
- **Mention endpoints.** `GET /api/users/mentions` supports `offset`, `limit`, and `unread_only`. `POST /api/users/mentions/read` marks a specific list of comment IDs, and `POST /api/users/mentions/read-all` clears the badge. See [API Endpoint Reference](api-reference.html) for details.
- **Public auth endpoints.** `/api/auth/needs-key` reports whether the current browser already has a key, and `/api/auth/pending-status` returns the waiting message shown to pending accounts. Neither requires a session.
- **Custom pending message.** Administrators can set `admin_wait_approve_msg`; if present, it appears on the pending page and on a blocked login attempt.

## Troubleshooting

- Registration fails with `Usernames starting with 'rootcoz' are reserved for system use`.
  Pick a username that does not start with `rootcoz`. The `rootcoz-ai` identity is reserved for the AI.

- Registration fails with `Registration is restricted. Contact an admin.`
  The server has an allow list configured. Ask an admin to add your username, or have them create the account for you.

- Registration succeeds but `Log in` says `Your account is awaiting admin approval.`
  `REQUIRE_APPROVAL` is on. An admin has to approve you from the Users page. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.

- Login says `Invalid username or API key.`
  The username must match the registered one (case-insensitive), or you are using a key that was rotated. If the admin rotated `ROOTCOZ_ENCRYPTION_KEY`, every stored key hash is invalidated and all users must be given new keys.

- `Rotate API Key` is missing, or returns `Bootstrap admin cannot rotate key via this endpoint.`
  You are signed in as the bootstrap `admin`. That key is managed through the `ADMIN_KEY` setting, not the UI.

- Push notifications show `Push notifications are not configured on server.`
  Web Push is off. An admin must configure both `VAPID_PUBLIC_KEY` and `VAPID_PRIVATE_KEY`, either under `Web Push` in Server Settings or in the environment. Keys are resolved in the order Server Settings, then environment, then an auto-generated key file, so a key saved in Server Settings is the one used for signing. Setting only one of the pair is treated as unset.

- Push notifications show `Notifications blocked.`
  Your browser denied permission for this site. Re-enable notifications in the browser's site settings, then click `Enable` again.

- A stored GitHub or Jira token disappeared after a server restart.
  Tokens are encrypted with `ROOTCOZ_ENCRYPTION_KEY`. If that key changed, previously stored tokens can no longer be decrypted — enter them again.

- Your CLI or a script suddenly returns `401`.
  Your key was rotated or an admin rotated it for you, which signs out every session for that user. Use the newest key.

## Related Pages

- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [Quickstart](quickstart.html)
- [Collaborating on Results](collaborate-on-results.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [API Endpoint Reference](api-reference.html)

---

Source: manage-users-and-server-settings.md

# Managing Users and Server Settings

This page is for RootCoz administrators. It covers the role ladder, what each role is allowed to do, the day-to-day user management actions, and the Server Settings page — including which values are masked and which need a restart before they take effect.

## Prerequisites

- You are signed in as an `admin`.
- The bootstrap `admin` account exists. It authenticates with the username `admin` and the `ADMIN_KEY` value, and it lives outside the users table, so it is not affected by user deletion. It exists **only** while `ADMIN_KEY` is non-empty: login compares the supplied key against `ADMIN_KEY` and refuses an empty one, so a deployment left at the empty default has no bootstrap path into admin at all. Set `admin.key` in the Helm chart, which auto-generates a key on first install when the value is empty — `chart/README.md` shows how to read it back out of the Secret.
- For server settings changes, you have a current server value for anything you are changing, because sensitive values are stored masked.

## Quick Example

1. Open `Users` from the admin sidebar.
2. Click `Add User`, enter a username, pick `operator`, and click `Create`.
3. Copy the API key from the confirmation dialog and send it to the new user over a channel you trust.
4. Open `Settings` in the admin sidebar, expand a category, change a value, and save.

That creates an active operator who can submit analyses, and one server setting change without a restart.

## Step-by-Step

1. **Understand the role ladder.**

   There are four roles. Each one is everything below it plus its own column.

   | Capability | `viewer` | `reviewer` | `operator` | `admin` |
   | --- | --- | --- | --- | --- |
   | View jobs and results | yes | yes | yes | yes |
   | Chat about a job | no | yes | yes | yes |
   | Comment on a job, mentions | no | yes | yes | yes |
   | Rotate own API key, manage own tokens | yes | yes | yes | yes |
   | Submit a new analysis (`POST /analyze`, ingest-only `POST /submit`) | no | no | yes | yes |
   | Re-analyze any job | no | no | yes | yes |
   | Delete own jobs | no | no | yes | yes |
   | Delete any job | no | no | no | yes |
   | Access `Reports` | only with the reports grant | only with the reports grant | only with the reports grant | always |
   | Create and delete users, change roles, grant reports access | no | no | no | yes |
   | Rotate another user's API key | no | no | no | yes |
   | Access `/api/admin/*` (Users, Tokens, Logs, Settings, admin Chat, DB tools) | no | no | no | yes |

   > **Note:** This table describes what the server currently enforces, which is not always what `AGENTS.md` describes. Rotating your own API key works for a `viewer` today because `POST /api/auth/rotate-key` checks no role, so this row reads `yes`; `AGENTS.md` reserves it for `reviewer`. Two further routes used to have the same gap — `PUT /results/{job_id}/reviewed` (which feeds auto-review and the exporter push) and `PUT /results/{job_id}/tags` — and now require `reviewer`. `POST /api/auth/rotate-key` remains tracked in [#289](https://github.com/myk-org/rootcoz/issues/289); treat the current behaviour as a bug rather than a supported capability. `GET /api/admin/token-usage/stream` is correctly admin-only: the authentication middleware rejects every non-admin request under `/api/admin/` before the handler runs, so it does not belong in this list.

   Two flags sit outside the role ladder:

   | Flag | Default | Meaning |
   | --- | --- | --- |
   | `can_view_reports` | off | Grants access to `/api/reports/*` and the `Reports` page to a non-admin. It is orthogonal to role: a `viewer` with the grant can read reports, and an `operator` without it cannot. For non-admins the flag is re-read from the users table on every request, so a grant takes effect without invalidating sessions. Admins always have effective access regardless of the stored value. |
   | `can_use_server_providers` | off | Grants the `Use server credentials` toggle on `New Analysis`, which lets the user run analyses on the server's AI keys instead of their own. Admins always have it. |
   | `ENABLE_FREE_MODELS` (server setting) | off | When on, every user may run analyses and chat on the server's keys for the admin-declared **free provider+model pairs**. Admins and users with the server-providers grant already cover it. Change it under Server Settings → AI, where the *Free AI models* card below the AI settings manages the pair list itself. |

   > **Note:** `DEFAULT_USER_ROLE` decides the role new registrations get, and it accepts only `viewer`, `reviewer`, or `operator` — it cannot grant `admin`. Setting it before startup does **not** by itself let self-registered users submit analyses: `REQUIRE_APPROVAL` defaults to `true`, and registration then creates the account with `pending` status, which cannot reach protected endpoints until an admin approves it. Set `REQUIRE_APPROVAL=false` alongside the role for that outcome.

2. **Create a user.**

   On `Users`, click `Add User`. The dialog takes a username, a role, and two switches: `Allow viewing reports` and `Allow server providers`. Click `Create`.

   | Detail | Behaviour |
   | --- | --- |
   | Username rules | Lowercased, must be non-empty, and must not start with `rootcoz` (that prefix is reserved, including the AI identity `rootcoz-ai`). |
   | Status | The account is created as `active` — admin-created users never go through approval. |
   | API key | Returned once in the response and shown once in the dialog. Copy it before closing. |
   | Reports flag | Stored exactly as submitted, even for an `admin`. Note that changing a user's role does not clear it: a user created through the API with the flag set keeps the grant after being demoted, so clear it explicitly if that is not what you want. The Settings UI does not offer the flag on admin creation, which is why the UI avoids this case. |
   | Your session | Creating a user does not change or replace your own session. |

3. **Change a role.**

   Use the role dropdown in the user's row. Every user except yourself and the reserved `admin` account can be changed; the server rejects `Cannot change your own role`. Confirm in the `Change Role to …` dialog.

   | Transition | What happens |
   | --- | --- |
   | To `admin` | If the user has no API key yet, one is generated and shown once. Otherwise the existing key is kept. |
   | Any other change | The existing API key is preserved. |

4. **Approve or reject a pending registration.**

   Pending users appear with a `pending` badge and are counted in the `{n} pending` pill next to the `User Management` heading.

   | Action | Effect |
   | --- | --- |
   | `Approve` | Sets the status to `active` so the user can log in. The approval dialog also decides whether that user may use server AI credentials. |
   | `Reject` | Sets the status to `rejected`. The user cannot log in and is told to contact an admin. |

   Pending users only exist when the server runs with `REQUIRE_APPROVAL=true`.

5. **Rotate another user's key.**

   Use the key icon in the user's row. The dialog warns `The old key will stop working immediately.` and, on completion, shows the new key once with a copy button.

   | Detail | Behaviour |
   | --- | --- |
   | Effect on the old key | Invalidated at once. |
   | Effect on sessions | All sessions for that user are deleted, so they are signed out everywhere. |
   | Admin accounts | An admin-role user gets a generated key; the bootstrap `admin` key is `ADMIN_KEY` and is not touched by this action. |

6. **Grant or revoke reports access.**

   Use the `View reports` switch in the user's row. Admins always have effective access, so the switch is a courtesy for downgrading later. The same access is available from the `Reports` page in the sidebar and from the CLI.

7. **Delete a user.**

   Use the delete icon in the user's row and confirm in the `Delete user` dialog. Deleting removes the account together with its stored tokens and AI credentials. You cannot delete your own account, and the bootstrap `admin` is not a row you can delete — it exists outside the users table.

8. **Read the Users table.**

   | Column | Meaning |
   | --- | --- |
   | `Username` | Lowercase account name; admins are marked with a shield. |
   | `Status` | `active`, `pending`, or `rejected`. |
   | `Role` | Current role, or a dropdown when you are allowed to change it. |
   | `View reports` | Effective reports access for that user. |
   | `Server providers` | Whether the user may use the server's AI credentials. Pending users cannot be toggled until they are active. |
   | `Created`, `Last Seen` | Registration time and most recent authenticated activity. |

9. **Change a server setting.**

   Open `Settings` in the admin sidebar. Settings are grouped into collapsible categories, and the header reports how many settings and categories exist. Search narrows the list across all categories.

   | Category | Settings it holds |
   | --- | --- |
   | `Jenkins` | `jenkins_url`, `jenkins_user`, `jenkins_password`, `jenkins_ssl_verify`, `jenkins_timeout`, `jenkins_artifacts_max_size_mb`, `get_job_artifacts` |
   | `AI` | `ai_provider`, `ai_model`, `force_server_credentials`, `ai_call_timeout`, `max_concurrent_ai_calls`, `peer_ai_configs`, `peer_analysis_max_rounds` |
   | `Jira` | `jira_url`, `jira_email`, `jira_api_token`, `jira_pat`, `jira_project_key`, `jira_ssl_verify`, `jira_max_results`, `enable_jira`, `enable_jira_issues` |
   | `GitHub` | `github_token`, `tests_repo_url`, `tests_repo_token`, `enable_github_issues` |
   | `Report Portal` | `reportportal_url`, `reportportal_api_token`, `reportportal_project`, `reportportal_verify_ssl`, `enable_reportportal`, `rp_push_classifications`, `rp_push_rootcoz_url`, `rp_push_tracker_links` |
   | `Auth & Security` | `admin_key`, `secure_cookies`, `trust_proxy_headers`, `require_approval`, `admin_wait_approve_msg`, `allowed_users`, `default_user_role` |
   | `Prow` | `prow_url`, `gcs_bucket` |
   | `Server` | `public_base_url`, `additional_repos`, `max_concurrent_repo_clones`, `wait_for_completion`, `poll_interval_minutes`, `max_wait_minutes`, `metadata_rules_file`, `enable_auto_review`, `auto_push_exporters` |
   | `Web Push` | `vapid_public_key`, `vapid_private_key`, `vapid_claim_email` |

   Each row shows the environment variable name, a source badge, and where relevant three other signals:

   | Signal | Meaning |
   | --- | --- |
   | Source `default` | Nothing set this value; the built-in default is in effect. |
   | Source `env` | The environment variable is in effect. |
   | Source `db` | A database override set from this page is in effect. Database values take priority over environment variables. |
   | Warning triangle next to the source | An environment variable is also set. The tooltip shows whether it matches the database override — if it does, the override is redundant. |
   | ⚠️ badge | The setting requires a server restart to take effect. |

10. **Understand masked and restart-required settings.**

   Some settings hold credentials and are never sent to the browser in cleartext. They display as `••••••••` until you use the reveal button, which asks the server for that one value on demand. Change history entries for those settings are masked too, and saving stores them encrypted.

   | Group | Settings |
   | --- | --- |
   | Masked and encrypted (`_SENSITIVE_SETTINGS`) | `jenkins_password`, `jenkins_user`, `jira_api_token`, `jira_pat`, `jira_email`, `github_token`, `tests_repo_token`, `reportportal_api_token`, `admin_key`, `vapid_private_key` |
   | Requires a restart (`_RESTART_REQUIRED_SETTINGS`) | `default_user_role`, `secure_cookies`, `trust_proxy_headers`, `metadata_rules_file`, `vapid_public_key`, `vapid_private_key`, `vapid_claim_email` |

   Most of those are also server-only, so the restart is not the operative constraint — the save is rejected outright. `vapid_private_key` is the only one here that this page can actually store; the rest must be set in the environment.

   Everything else applies to the running server as soon as you save. The exceptions are the server-only settings, which this page cannot save at all: `PUT /api/admin/settings` rejects every one of them with `400 Server-only settings (env var only)`, so they must be set in the deployment environment instead. They are `admin_wait_approve_msg`, `allowed_users`, `auto_push_exporters`, `db_path`, `debug`, `default_user_role`, `enable_auto_review`, `enable_github_issues`, `enable_reportportal`, `log_level`, `metadata_rules_file`, `public_base_url`, `require_approval`, `rp_push_classifications`, `rp_push_rootcoz_url`, `rp_push_tracker_links`, `secure_cookies`, `trust_proxy_headers`, `vapid_claim_email`, and `vapid_public_key`. A setting described elsewhere on this page as editable, or as needing only a restart, is still rejected here if it appears in that list — the restart-required distinction only applies to settings that can be saved at all.

   > **Warning:** Saving an empty value is a reset, not a blank value. The database override is deleted and resolution falls back to the environment variable, then the default.

11. **Review and reset individual settings.**

   Each row offers a history view (who changed it, when, and the previous value) and a reset action that deletes the database override. Resetting a setting that has no override fails rather than silently doing nothing. Because masked values are shown as `••••` in history, capture the real value elsewhere before you reset one.

12. **Watch usage and logs.**

   Two more admin pages round out the picture:

   | Page | Shows |
   | --- | --- |
   | `Tokens` | Token usage broken down by provider, model, job, or day, with calls, input, output, cache read/write, cost, and average duration. Filter by date range and provider. |
   | `Logs` | The live server log tail, filterable by level and text, streamed over SSE. |

## Advanced Usage

- **Reset to environment values.** Database overrides always win over environment variables. Reset the setting in the UI (or delete it with `DELETE /api/admin/settings/{key}`) when you want the deployment's env var back in charge.
- **Audit trail.** Logins, registrations, key rotations, approvals, role changes, report grants, and deletions are all written to the audit log with the acting admin's username, and both actions and log lines are visible on the `Logs` page.
- **Automate with the CLI.** User creation, role changes, `can_view_reports`, and pending approvals are all available from the CLI (`admin users ...`). See [CLI Command Reference](cli-reference.html) for details.
- **Give analysis more context.** The `AI`, `GitHub`, and `Server` categories hold most of the knobs that change how much context an analysis gets. See [Configuring Analysis Context](configure-analysis-context.html) for details.
- **Declare free models.** In Server Settings → AI, the *Free AI models* card below the settings rows manages the free provider+model pair list. Pick a provider and model from the server catalog and click `Add`; remove pairs with the per-row `Remove` button. "Free" is this list — never the catalog's cost fields — and every pair is validated against the server catalog at save time. The `ENABLE_FREE_MODELS` toggle above it decides whether the list is live. The CLI mirrors it: `admin models free-list` / `free-add` / `free-remove`.
- **Keep the bootstrap key reachable.** `ADMIN_KEY` authenticates a superuser that lives outside the database. If you rotate it while signed in as a database admin, keep a copy of the new value or you can lock yourself out.
- **Encryption key rotation.** `ROOTCOZ_ENCRYPTION_KEY` is not a server setting in the UI. Changing it re-derives both the Fernet key used for at-rest secrets and the HMAC secret used to hash API keys. Existing API key hashes stop matching, so every user must be given a new key; stored user tokens, AI credentials, and encrypted settings become undecryptable and must be re-entered. Sessions survive, because a session row stores only the username, admin flag, role, and expiry.

## Troubleshooting

- `Users` or `Settings` is missing from the sidebar.
  You are not an admin. Ask an admin to change your role; only an admin can change your own role via another admin.

- `Admin access required` (403).
  The signed-in account is not an admin, or the session predates a role change. Sign out and back in.

- A new registration does not appear as pending.
  `REQUIRE_APPROVAL` is off, so registrations are created `active` immediately. To require approval, set it in the deployment environment — it is a server-only setting, so saving it through Server Settings is rejected with `400 Server-only settings (env var only)` and restart the server afterwards.

- A registered username was rejected.
  Usernames starting with `rootcoz` are reserved. Pick a different username.

- `DEFAULT_USER_ROLE` refuses to save `admin`.
  It only accepts `viewer`, `reviewer`, or `operator`. Grant `admin` to a specific user from the Users page instead.

- Reports still do not appear for a user you just granted.
  The grant is read per request, so reload the page. If the user has not signed in again and still sees nothing, confirm they are not a pending account.

- A masked setting looks empty after you reveal it.
  There is no database override, so there is nothing stored to reveal. Reveal uses the environment value when one is set.

- A setting did not take effect after saving.
  Check for the ⚠️ restart badge. `default_user_role`, `secure_cookies`, `trust_proxy_headers`, `metadata_rules_file`, and the three `Web Push` settings only apply after the server restarts.

- Everyone's logins and tokens broke after an infrastructure change.
  `ROOTCOZ_ENCRYPTION_KEY` changed. Sessions survive, but stored API key hashes and encrypted tokens do not — rotate keys and re-enter tokens.

- Bootstrap `admin` login fails after you edited `admin_key`.
  The login path compares the username `admin` against the current value in constant time. Confirm the value you saved, and remember that setting an empty value resets the override instead of storing a blank.

## Related Pages

- [Managing Your Account and Notifications](manage-account-and-notifications.html)
- [Configuration Reference](configuration-reference.html)
- [Deploying RootCoz](deploy-rootcoz.html)
- [Configuring Analysis Context](configure-analysis-context.html)
- [Exploring History and Reports](explore-history-and-reports.html)
- [CLI Command Reference](cli-reference.html)

---

Source: api-reference.md

# API Endpoint Reference

Everything the web UI does goes through the same HTTP API the CLI and the AI chat tools use, so you can script RootCoz from CI, from a shell, or from your own tooling. This page groups the endpoints by what they do, states the minimum role each one needs, and points at the live OpenAPI schema for exact field-level detail.

## Prerequisites
- A running RootCoz server. The examples use `http://localhost:8000`; see [Quickstart](quickstart.html).
- An API key. Register or log in in the web UI, or send `Authorization: Bearer <API_KEY>` with any request.
- The role your call needs: `viewer`, `reviewer`, `operator`, or `admin`. See [Managing Users and Server Settings](manage-users-and-server-settings.html).

## Quick Example

```bash
    curl -H "Authorization: Bearer $ROOTCOZ_API_KEY" \
      http://localhost:8000/results/JOB_ID
```

That single authenticated GET returns the stored analysis for one job, including failures, classifications, patterns, classification overrides, tracker links, capabilities, and child jobs. It does **not** include the comment thread — call `GET /results/{job_id}/comments` separately for that.

## Step-by-Step

1. **Authenticate.**

   Send your API key as a bearer token. The `rootcoz_session` cookie set at login works too, and `X-Forwarded-User` works when the server runs behind a trusted proxy with `TRUST_PROXY_HEADERS=true`.

   ```bash
   curl -H "Authorization: Bearer $ROOTCOZ_API_KEY" \
     http://localhost:8000/api/auth/me
   ```

   `GET /api/auth/me` is the cheapest way to confirm your identity, role, and reports access. Send `OPTIONS` requests freely — CORS preflight bypasses authentication everywhere.

2. **Discover the schema instead of guessing field names.**

   | Path | What it is |
   | --- | --- |
   | `GET /openapi.json` | Full OpenAPI schema, public |
   | `GET /docs` | Swagger UI, public |
   | `GET /redoc` | ReDoc UI, public |

   These three are public so an integrator can read the contract without credentials. Every route in the schema sets an explicit camelCase `operation_id` derived from the handler name, so generated clients get stable method names such as `getJobResult` and `analyzeSubmittedJob`.

   > **Note:** This page is a curated map of the endpoints people actually call. The live schema at `/openapi.json` is the exhaustive list.

3. **Know the role ladder before you script anything.**

   | Role | Can do |
   | --- | --- |
   | `viewer` | Read jobs, results, tests, history, dashboards, reports when granted |
   | `reviewer` | Everything a viewer can, plus comments, chat, review state, classification and pattern overrides, tags, tracker links, personal tokens |
   | `operator` | Everything a reviewer can, plus submit analyses, re-analyze, abort, delete own jobs, push to exporters |
   | `admin` | Everything an operator can, plus delete any job, manage users and roles, server settings, token usage, log stream, admin chat |

   `can_view_reports` is a separate flag from role. Admins always have it; anyone else needs an admin to grant it. Most calls that create or modify data also run against the `ALLOWED_USERS` allow list, which is empty (open) by default and always bypassed for admins. The check is per-handler, not global, so it is enforced on data-modifying routes rather than as a middleware boundary; `tests/test_allow_list_coverage.py` enumerates the mutating routes and fails if one that can write data is added without the check. The exemptions are listed exactly in that test: the authentication routes, the read-only Jira and token-validation lookups, the metadata rules preview, and `DELETE /api/user/ai-credentials/{provider}` — revoking your own key stays possible even if you are off the list, otherwise being removed would leave you holding a credential you cannot remove.

   > **Warning:** A role you do not hold returns `403`, not `404`. `403 Admin access required`, `403 Reviewer access required`, and `403 Operator access required` name the exact gate you hit.

### Public and unauthenticated routes

| Method | Path | Notes |
| --- | --- | --- |
| GET | `/health` | Liveness, returns immediately |
| GET | `/api/health` | Detailed health including database and AI sidecar |
| GET | `/metrics` | Prometheus metrics |
| GET | `/api/releases/latest` | Proxies GitHub release metadata (version, changelog) |
| GET | `/openapi.json`, `/docs`, `/redoc` | Schema and interactive API browsers |
| POST | `/api/auth/register` | Create a user, returns the raw API key once |
| POST | `/api/auth/login` | Exchange username + API key for a session cookie |
| GET | `/api/auth/needs-key` | Browser-only identity probe |
| GET | `/api/auth/pending-status` | Poll while an account awaits approval |

### Authentication and your account

| Method | Path | Min role | Notes |
| --- | --- | --- | --- |
| GET | `/api/version` | viewer | Running version |
| POST | `/api/auth/logout` | authenticated | Clears the session cookie |
| GET | `/api/auth/me` | viewer | Username, role, `can_view_reports`, `can_use_server_providers` |
| POST | `/api/auth/rotate-key` | viewer | Rotates your own API key; the old key stops working. Known gap — see the note below |
| GET | `/api/user/ai-credentials` | viewer | Supported provider IDs with a `configured` flag each. Keys are never returned |
| PUT | `/api/user/ai-credentials/{provider}` | reviewer | Store a personal key. Body is `{api_key, model}` — `model` is required (`400` without it, since the key is verified against it) and the key is only persisted when the provider returns `accepted`. A `rejected` or `inconclusive` check is not an HTTP error: it returns `200` with `ok: false`, an `outcome`, and an explanatory `message`, leaving any existing credential unchanged, so clients must check `ok` rather than the status code. `{provider}` is an exact provider ID from `GET /api/user/ai-credentials` (for example `anthropic` or `google`), not the `claude`/`gemini`/`cursor` names used for analysis settings. Unknown IDs are rejected with 400 |
| DELETE | `/api/user/ai-credentials/{provider}` | reviewer | Remove a personal provider key |
| GET | `/api/user/tokens` | viewer | Stored Jira and GitHub tracker tokens |
| PUT | `/api/user/tokens` | reviewer | Save Jira and GitHub tracker tokens. Each field is independent: a string sets it, `""` or `null` clears just that field, and an omitted field keeps its stored value, so clearing one credential leaves the others intact. A body with none of the three fields writes nothing and returns `saved: false`; anything other than a string or `null` is rejected with `400` |
| POST | `/api/validate-token` | viewer | Validate a GitHub or Jira token before saving |
| POST | `/api/jira-projects` | viewer | List Jira projects visible to the supplied credentials |
| POST | `/api/jira-security-levels` | viewer | List Jira security levels for a project |

Registration returns the raw API key exactly once and sends `Cache-Control: no-store`. Store it before you navigate away.

### Submitting and driving analyses

All of these continue in the background and return `202 Accepted`, except `POST /results/{job_id}/abort`, which returns `200 OK`. `POST /submit` ingests only and stops before cloning or AI work; `POST /analyze` runs the full pipeline.

| Method | Path | Min role | Body highlights |
| --- | --- | --- | --- |
| POST | `/analyze` | operator | `type` (`jenkins`, `file`, `raw`, `prow`) plus source-specific fields |
| POST | `/submit` | operator | Same body; stops at ingest, `analysis_state=submitted` |
| POST | `/results/{job_id}/analyze` | operator | Runs the AI pipeline on a submitted job; optional `ReAnalyzeRequest` |
| POST | `/re-analyze/{job_id}` | operator | Re-runs analysis with a new `ReAnalyzeRequest` |
| POST | `/api/failures/{failure_uuid}/re-analyze` | operator | Re-analyzes one failure in place, keeps the previous answer |
| POST | `/results/{job_id}/abort` | reviewer | Cancels a running analysis |

The shared body accepts the server defaults as per-request overrides:

| Field | Purpose |
| --- | --- |
| `type` | `jenkins`, `file`, `raw`, or `prow` |
| `job_name`, `build_number` | Jenkins job and build |
| `raw_xml`, `failures`, `passed_tests`, `skipped_tests` | File and raw sources |
| `prow_job_name`, `build_id`, `gcs_prefix` | Prow build identity and artifact prefix |
| `prow_url`, `gcs_bucket` | Prow server defaults, overridable per request |
| `ai_provider`, `ai_model`, `force_server_credentials` | `claude`, `gemini`, or `cursor` |
| `ai_call_timeout`, `max_concurrent_ai_calls` | AI budget for this run |
| `peer_ai_configs`, `peer_analysis_max_rounds` | Multi-AI debate settings |
| `additional_repos` | Extra repositories to clone for context |
| `tests_repo_url`, `tests_repo_token` | Tests repository used for history and context |
| `jira_url`, `jira_email`, `jira_api_token`, `jira_pat`, `jira_project_key`, `jira_ssl_verify`, `jira_max_results`, `enable_jira` | Jira search during analysis |
| `github_token` | GitHub access for comment enrichment |
| `raw_prompt`, `issue_prompt` | Additive prompt instructions |
| `labels`, plus name and tag fields | Metadata for the job |

> **Note:** Resolution order for provider, model, timeout, concurrency, peers, and additional repos is request → `.rootcoz/settings.json` in the cloned tests repo → server setting → environment variable. See [Configuring Analysis Context](configure-analysis-context.html).

### Reading results

| Method | Path | Min role | Query parameters |
| --- | --- | --- | --- |
| GET | `/results` | viewer | `limit` (max 100), `analysis_state` (`submitted`, `analyzed`) |
| GET | `/results/{job_id}` | viewer | `fields` — comma-separated allowlisted paths |
| GET | `/api/results/fields` | viewer | none; returns the allowlist |
| GET | `/api/results/{job_id}/tests` | viewer | `status` (repeatable: `passed`, `skipped`, `failed`), `child_job_name`, `child_build_number`, `offset`, `limit` (1–200, default 50) |
| GET | `/api/failures/{failure_uuid}` | viewer | none |
| GET | `/results/{job_id}/review-status` | viewer | none |
| GET | `/api/dashboard` | viewer | `limit`, `offset` |
| GET | `/api/dashboard/filtered` | viewer | `search`, `status`, `date_from`, `date_to`, `review_status`, `analysis_state`, `limit`, `offset`, plus metadata filters `team`, `tier`, `version`, `label`, `exclude_label` (each repeatable) |
| GET | `/api/dashboard/active-count` | viewer | none |

`fields` returns full values for allowlisted paths only and is never truncated. Unknown paths return `400`, so fetch `/api/results/fields` first when you build against it.

### Review, classification, and job maintenance

| Method | Path | Min role | Body |
| --- | --- | --- | --- |
| PUT | `/results/{job_id}/reviewed` | reviewer | `test_name`, `reviewed`. |
| PUT | `/results/{job_id}/override-classification` | reviewer | `test_name`, `classification` |
| PUT | `/results/{job_id}/override-pattern` | reviewer | `test_name`, `pattern` |
| PUT | `/results/{job_id}/tags` | reviewer | JSON body `{"tags": ["a", "b"]}` — a list of strings. A non-list value is rejected with `400 tags must be a list of strings` |
| DELETE | `/results/{job_id}` | operator | none; deletes your own job, admins delete any |
| DELETE | `/api/results/bulk` | operator | `job_ids` |

Classification values are `CODE ISSUE`, `PRODUCT BUG`, and `INFRASTRUCTURE`. Pattern values are `NEW`, `REGRESSION`, `FLAKY`, `INTERMITTENT`, `KNOWN_BUG`, and `PERSISTENT`. These are the exact API values; the UI renders `KNOWN_BUG` as `Known Bug` and spaces the words for readability.

> **Note:** This table lists the role each route **actually enforces**. One route marked "Known gap" enforces no role beyond authentication — `viewer` is therefore its effective minimum, even though the documented model reserves it for `reviewer`: `POST /api/auth/rotate-key`, tracked in [#289](https://github.com/myk-org/rootcoz/issues/289). Treat it as a bug, not as a supported way to operate. `PUT /results/{job_id}/reviewed` and `PUT /results/{job_id}/tags` were previously gaps as well and now require `reviewer`. Review state is tracked per test; classification and pattern changes apply across the whole same-error group.

### Comments and mentions

| Method | Path | Min role | Notes |
| --- | --- | --- | --- |
| GET | `/results/{job_id}/comments` | viewer | Full thread for one job |
| POST | `/results/{job_id}/comments` | reviewer | `test_name`, `comment`; returns `201` |
| DELETE | `/results/{job_id}/comments/{comment_id}` | reviewer | Comment author or admin |
| POST | `/results/{job_id}/enrich-comments` | viewer | Live GitHub PR and Jira ticket status |
| POST | `/api/analyze-comment-intent` | reviewer | Classifies an incoming comment as a question or an action |
| GET | `/api/users/mentions` | viewer | Mentions for the current user |
| GET | `/api/users/mentions/unread-count` | viewer | Navbar badge count |
| POST | `/api/users/mentions/read` | viewer | Mark selected mentions read |
| POST | `/api/users/mentions/read-all` | viewer | Clear the unread badge |
| GET | `/api/users/mentionable` | viewer | Users who can be `@`-mentioned |

### Trackers, issues, and exporters

| Method | Path | Min role | Notes |
| --- | --- | --- | --- |
| GET | `/results/{job_id}/issue-prompt` | viewer | Custom `.rootcoz/ROOTCOZ_ISSUE_PROMPT.md` for the repo |
| POST | `/results/{job_id}/preview-github-issue` | reviewer | `test_name`, `include_links`, AI overrides; returns preview content |
| POST | `/results/{job_id}/preview-jira-bug` | reviewer | Same shape, Jira flavour |
| POST | `/results/{job_id}/create-github-issue` | reviewer | `test_name`, `title`, `body`; returns `201` |
| POST | `/results/{job_id}/create-jira-bug` | reviewer | Adds `jira_issue_type`; returns `201` |
| PUT | `/results/{job_id}/tracked-in` | reviewer | `test_name`, `url`, `type` |
| GET | `/results/{job_id}/tracked-in` | reviewer | Current tracker links |
| DELETE | `/results/{job_id}/tracked-in/{link_id}` | reviewer | Remove one link |
| GET | `/api/exporters` | viewer | Available exporter plugins and enabled state |
| POST | `/results/{job_id}/push/{plugin_name}` | operator | Generic push; `child_job_name` and `child_build_number` optional |
| POST | `/results/{job_id}/push-reportportal` | operator | Backward-compatible Report Portal push |

> **Note:** All four issue preview and create endpoints call `_require_reviewer`, so a `viewer` receives `403` before the allow list is consulted. They additionally require valid tracker credentials and the matching `ENABLE_GITHUB_ISSUES` or `ENABLE_JIRA_ISSUES` toggle.

### History and classification lookup

| Method | Path | Min role | Query parameters |
| --- | --- | --- | --- |
| GET | `/history/failures` | viewer | `search`, `job_name`, `classification`, `from`, `to`, `limit`, `offset`. The date bounds are named `from` and `to` on the wire, not `date_from`/`date_to` |
| GET | `/history/test/{test_name}` | viewer | `limit`, `job_name`, `exclude_job_id` |
| GET | `/history/search` | viewer | `signature`, `exclude_job_id` |
| GET | `/history/stats/{job_name}` | viewer | `exclude_job_id` |
| GET | `/history/classifications` | viewer | `test_name`, `classification`, `job_name`, `parent_job_name`, `job_id` |
| POST | `/history/classify` | reviewer | **JSON body** (not query string): `test_name`, `classification`, `reason`, `job_id`, `source`; returns `201` |

Send `source="ai"` to attribute the classification to the reserved `rootcoz-ai` identity. The backend blocks AI callers from overriding classifications a human has already set.

### Chat

| Method | Path | Min role | Notes |
| --- | --- | --- | --- |
| GET | `/api/chat/{job_id}` | viewer | `limit` (1–500), `offset` |
| POST | `/api/chat/{job_id}` | reviewer | `message` plus optional AI overrides; returns `202`, answer arrives over SSE |
| POST | `/api/chat/{job_id}/init` | reviewer | Explicit `ai_provider`, `ai_model`, `force_server_credentials` |
| POST | `/api/chat/{job_id}/abort` | reviewer | Cancel the in-flight answer |
| POST | `/api/chat/{job_id}/close` | reviewer | Close the AI session |
| DELETE | `/api/chat/{job_id}` | reviewer | Clear chat history |
| GET | `/api/chat/{job_id}/stream` | viewer | SSE stream of new messages |

### Server-Sent Events streams

| Path | Min role | Pushes |
| --- | --- | --- |
| `/api/stream` | viewer | Multiplexed stream; subscribe with `?topics=navbar,dashboard,results:JOB_ID,comments:JOB_ID,chat:JOB_ID` |
| `/api/navbar/stream` | viewer | Active analysis count and unread mentions |
| `/api/dashboard/stream` | viewer | Job list changes |
| `/api/results/{job_id}/stream` | viewer | Per-job status changes |
| `/api/results/{job_id}/comments/stream` | viewer | Per-job comment changes |
| `/api/chat/{job_id}/stream` | viewer | Per-job chat messages |
| `/api/admin/token-usage/stream` | admin | Token usage changes. Enforced by middleware before the handler runs |
| `/api/admin/logs/stream` | admin | Live server log tail; `lines` and `level` query parameters |
| `/api/admin/settings/stream` | admin | Server setting changes |

Use `/api/stream` when you want one connection instead of six. It accepts up to 50 comma-separated topics; `token-usage` and `settings` are delivered only to admins.

### Reports and analytics

| Method | Path | Min role | Query parameters |
| --- | --- | --- | --- |
| GET | `/api/reports/totals` | admin or `can_view_reports` | `team`, `tier`, `version`, `from`, `to`, `status`, `tags`, `exclude_tags`, `review_status`, `limit`, `offset` |
| GET | `/api/reports/classification-overrides` | admin or `can_view_reports` | same filter set |
| GET | `/api/reports/issues-created` | admin or `can_view_reports` | same filter set |

Date bounds use the `from` and `to` query names. Without the flag you get `403 Reports access required`.

### Metadata assignment

| Method | Path | Min role |
| --- | --- | --- |
| GET | `/api/jobs/metadata` | viewer |
| GET | `/api/jobs/{job_name}/metadata` | viewer |
| PUT | `/api/jobs/{job_name}/metadata` | admin |
| DELETE | `/api/jobs/{job_name}/metadata` | admin |
| PUT | `/api/jobs/metadata/bulk` | admin |
| GET | `/api/jobs/metadata/rules` | viewer |
| POST | `/api/jobs/metadata/rules/preview` | viewer |

### Notifications and AI discovery

| Method | Path | Min role | Notes |
| --- | --- | --- | --- |
| GET | `/api/notifications/vapid-public-key` | viewer | Web Push public key |
| POST | `/api/notifications/subscribe` | viewer | `endpoint`, `p256dh_key`, `auth_key` |
| POST | `/api/notifications/unsubscribe` | viewer | Removes a subscription |
| GET | `/api/ai-models` | viewer | `provider`, `force_server_credentials`; lists models for `claude`, `gemini`, `cursor` |
| GET | `/api/capabilities` | viewer | Feature toggles and whether server credentials exist |
| GET | `/api/default-server-settings` | viewer | Non-sensitive settings and defaults only |
| POST | `/api/feedback/preview` | viewer | AI-drafted feedback issue |
| POST | `/api/feedback/create` | viewer | Creates the GitHub issue; returns `201` |

### Admin

| Method | Path | Notes |
| --- | --- | --- |
| GET | `/api/admin/users` | List users |
| GET | `/api/admin/users/pending` | Accounts awaiting approval |
| POST | `/api/admin/users/create` | `username`, `role`, `can_view_reports`, `can_use_server_providers` |
| DELETE | `/api/admin/users/{username}` | Delete a user |
| PUT | `/api/admin/users/{username}/role` | Change role |
| POST | `/api/admin/users/{username}/approve` | Approve a pending registration |
| POST | `/api/admin/users/{username}/reject` | Reject a pending registration |
| POST | `/api/admin/users/{username}/rotate-key` | Re-issue a user's API key |
| PUT | `/api/admin/users/{username}/can-view-reports` | Grant or revoke reports access |
| PUT | `/api/admin/users/{username}/can-use-server-providers` | Grant or revoke server AI credentials |
| GET | `/api/admin/free-models` | List admin-declared free provider+model pairs (with `version` fingerprint) |
| PUT | `/api/admin/free-models` | Replace the free pair list (validated against the catalog); optional `expected_version` from a prior GET rejects concurrent edits with `409` |
| GET | `/api/admin/settings` | All settings with value, source, and metadata; `reveal_key` unmasks one key or `__all__` |
| PUT | `/api/admin/settings` | `{"settings": {"KEY": "value"}}` |
| GET | `/api/admin/settings/history` | `key`, `limit` change history |
| DELETE | `/api/admin/settings/{key}` | Reset one setting to its default |
| GET | `/api/admin/token-usage` | `start_date`, `end_date`, `ai_provider`, `ai_model`, `call_type`, `group_by` |
| GET | `/api/admin/token-usage/summary` | Dashboard totals |
| GET | `/api/admin/token-usage/{job_id}` | Per-job usage |
| POST | `/api/admin/ai-models/refresh` | Re-discover models from the sidecar |
| GET | `/api/admin/component-versions` | Backend, sidecar, and chart versions |
| GET | `/api/admin/db/schema` | Read-only schema dump |
| POST | `/api/admin/db/query` | Read-only SQL for the admin chat |
| GET | `/api/admin/chat` | Admin chat history |
| POST | `/api/admin/chat` | Send an admin chat message; returns `202` |
| POST | `/api/admin/chat/init`, `/close`, `/abort` | Admin chat session control |
| DELETE | `/api/admin/chat` | Clear admin chat history |
| GET | `/api/admin/chat/stream` | Admin chat SSE |
| POST | `/api/admin-chat/artifacts` | Save a generated HTML report artifact |
| GET | `/api/admin-chat/artifacts/{artifact_id}` | Download that artifact |

See [Configuration Reference](configuration-reference.html) for the full environment variable surface behind these admin settings.

## Advanced Usage

Prefer the CLI when you can. Most endpoints above have a `rootcoz` command counterpart. SSE streams have none, because the CLI is a one-shot tool, and the user-token endpoints (`GET`/`PUT /api/user/tokens`) have client methods but no registered CLI command either. See [CLI Command Reference](cli-reference.html) and [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html).

A three-call integration that skips the UI entirely:

```bash
    KEY="$ROOTCOZ_API_KEY"
    BASE=http://localhost:8000

    JOB=$(curl -s -X POST "$BASE/analyze" \
      -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
      -d '{"type":"file","raw_xml":"<testsuite>...</testsuite>","labels":["nightly"]}' \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["job_id"])')

    curl -s -H "Authorization: Bearer $KEY" "$BASE/results/$JOB" | jq '.status'

    curl -s -H "Authorization: Bearer $KEY" \
      "$BASE/results/$JOB?fields=status,result.summary"
```

For long-lived integrations, open `/api/stream?topics=navbar,results:$JOB` once instead of polling the result endpoint.

## Troubleshooting
- `403 Authentication required` or a `401` on a supposedly public path.
  The path is not in the public allow list. Use `POST /api/auth/login` for a session cookie or send `Authorization: Bearer <API_KEY>`.

- `403 Admin access required` on `/api/admin/*`.
  Your account is not `admin`. The bootstrap `admin` superuser is created from `ADMIN_KEY`; see [Managing Users and Server Settings](manage-users-and-server-settings.html).

- `403 Reports access required` from `/api/reports/*`.
  You need the `can_view_reports` flag, which is independent of role. Admins always have it.

- `400` from `GET /results/{job_id}?fields=...`.
  One path is not on the allowlist. Fetch `GET /api/results/fields` and use an exact path.

- `404` on `/api/results/{job_id}/tests` for tests you expect.
  A `404` here means the job ID does not exist — the endpoint checks that before applying any filter, so no amount of `child_job_name` or `child_build_number` will help. An existing job that simply has no matching entries returns `200` with an empty `entries` list; to narrow or widen that, confirm the status filter matches `passed`, `skipped`, or `failed`, and add `child_job_name` and `child_build_number` when the tests live under a child job.

- `403` on `POST /analyze` right after registering.
  Your default role is `viewer` or the account is still pending approval. Check `GET /api/auth/me`, then ask an admin to raise your role or approve the account.

## Related Pages
- [Configuration Reference](configuration-reference.html)
- [CLI Command Reference](cli-reference.html)
- [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html)
- [Submitting Analyses](submit-analyses.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Using Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html)
- [Exploring History and Reports](explore-history-and-reports.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)

---

Source: cli-reference.md

# CLI Command Reference

Every RootCoz web action has a `rootcoz` CLI counterpart. This page is the lookup table: what each command group does, the exact usage line, and the flags that actually exist. For task-oriented walkthroughs instead of a flag list, see [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html).

## Prerequisites

- The `rootcoz` CLI on your `PATH`. It is installed with the project, and the web UI's Docker image already includes it.
- A reachable RootCoz server, either from `--server`/`ROOTCOZ_SERVER` or from `~/.config/rootcoz/config.toml`.
- An API key from `rootcoz register`, an admin key, or a key issued by an administrator.

## Quick Example

```bash
    rootcoz --json results dashboard --limit 5
```

That single command resolves the server, authenticates, and returns machine-readable JSON you can pipe into `jq`.

## Step-by-Step

1. **Set up the global options.** These go before the command group and apply to everything that follows.

| Option | Env var | Purpose |
| --- | --- | --- |
| `--server`, `-s` | `ROOTCOZ_SERVER` | Config server name or full URL. |
| `--json` | — | Emit JSON instead of tables. |
| `--user` | `ROOTCOZ_USERNAME` | Username shown in comments and reviews. |
| `--api-key` | `ROOTCOZ_API_KEY` | Bearer token for a user or admin account. |
| `--no-verify-ssl` | `ROOTCOZ_NO_VERIFY_SSL` | Skip TLS certificate verification. |
| `--verify-ssl` | — | Force verification on, overriding the config profile. |
| `--insecure` | — | Alias for `--no-verify-ssl`. |
| `--install-completion` | — | Install shell completion. |
| `--show-completion` | — | Print completion script to stdout. |

> **Note:** `--json` is also accepted after the subcommand, so `rootcoz --json results list` and `rootcoz results list --json` are equivalent. `--verify-ssl` and `--insecure` together are an error.

2. **Check server state.**

| Command | Purpose |
| --- | --- |
| `rootcoz register USERNAME` | Register a user and print the new API key once. |
| `rootcoz health` | Server health, per-check status, and recent error rate. |
| `rootcoz version` | Server version string. |

3. **Manage authentication and AI keys.**

| Command | Purpose |
| --- | --- |
| `rootcoz auth login --username NAME --api-key KEY` | Validate credentials and print the resolved role. Does not persist anything. |
| `rootcoz auth whoami` | Current username, role, admin flag, and report access. |
| `rootcoz auth rotate-key` | Issue a new API key for yourself, shown once. |
| `rootcoz auth logout` | Clear the browser session cookie. See the note below — this does not remove a configured API key. |
| `rootcoz auth ai-keys list` | Show which providers already have a key for your account. |
| `rootcoz auth ai-keys set PROVIDER --model MODEL [--stdin]` | Store a provider key, verified against `MODEL`. |
| `rootcoz auth ai-keys delete PROVIDER` | Remove your stored key for one provider. |

Roles reported by `auth whoami` are `viewer`, `reviewer`, `operator`, and `admin`.

> **Note:** `auth logout` posts to `POST /api/auth/logout`, which deletes the `rootcoz_session` and `jji_session` browser cookies. The CLI never holds those cookies, and it stores no session of its own — `auth login` only validates the key and prints your role. So `logout` is effectively a no-op for CLI use: it leaves the `api_key` in `~/.config/rootcoz/config.toml` untouched, and every later command keeps authenticating with it. To end CLI access, remove the key from the config file (or unset `ROOTCOZ_API_KEY`), or run `rootcoz auth rotate-key` to issue a replacement — the old key stops working immediately, but you must then update the config file or environment with the new key, which is shown once.

4. **Manage the local config file.** These read `~/.config/rootcoz/config.toml` (or `$XDG_CONFIG_HOME/rootcoz/config.toml`) and do not need a running server, except `config defaults`.

| Command | Purpose |
| --- | --- |
| `rootcoz config` or `rootcoz config show` | Config path, default server, and each server's URL, user, and SSL mode. |
| `rootcoz config servers` | List servers; supports `--json`. |
| `rootcoz config completion [bash\|zsh]` | Print the rc-file snippet that wires up completion. |
| `rootcoz config defaults` | Server-side analysis defaults (non-sensitive). Requires a server. |

Config keys are grouped as `[default] server = "NAME"`, `[servers.NAME]` entries, and a shared `[defaults]` table merged into every server. Per-server keys include `url`, `username`, `api_key`, `no_verify_ssl`, `ai_provider`, `ai_model`, `jenkins_*`, `jira_*`, `github_token`, `peers`, `additional_repos`, `labels`, `prow_url`, and `gcs_bucket`. See [Configuration Reference](configuration-reference.html) for the full list.

5. **Submit analyses.**

`rootcoz analyze [OPTIONS]` and `rootcoz submit [OPTIONS]` share one flag set. They are the same command registered under two names, and the CLI switches behaviour on the name you invoke: `submit` posts to `POST /submit` so the run ingests and stores CI results without cloning a repo or calling the AI, while `analyze` posts to `POST /analyze` and runs the full pipeline. Pair `submit` with `rootcoz results analyze JOB_ID` to analyze the stored job later on the same job ID.

| Option | Purpose |
| --- | --- |
| `--source` | `jenkins` (default), `file`, `prow`, or `raw`. |
| `--job-name`, `-j` | Job name. Required for `jenkins` and `prow`. |
| `--build-number`, `-b` | Build number, or Prow build ID as a numeric string. Required for `jenkins` and `prow`. |
| `--file`, `-f` | Path to a JUnit XML file. Required for `file`. |
| `--prow-url`, `--gcs-bucket`, `--gcs-prefix` | Prow Deck URL, artifact bucket, and object prefix. `PROW_URL` and `GCS_BUCKET` also come from the environment. |
| `--name`, `-n` | Display name on the dashboard. |
| `--provider`, `--model` | Exact provider ID and model to use. |
| `--jira` / `--no-jira` | Enable or disable Jira integration for this run. |
| `--jenkins-url`, `--jenkins-user`, `--jenkins-password` | Jenkins credentials. `JENKINS_URL`, `JENKINS_USER`, `JENKINS_PASSWORD`. |
| `--jenkins-ssl-verify` / `--no-jenkins-ssl-verify`, `--jenkins-timeout`, `--jenkins-artifacts-max-size-mb` | Jenkins transport tuning. |
| `--get-job-artifacts` / `--no-get-job-artifacts` | Download build artifacts for AI context. |
| `--tests-repo-url`, `--tests-repo-token` | Tests repository and its token. `TESTS_REPO_URL`, `TESTS_REPO_TOKEN`. |
| `--jira-url`, `--jira-email`, `--jira-api-token`, `--jira-pat`, `--jira-project-key` | Jira connection. Each has a matching `JIRA_*` env var. |
| `--jira-ssl-verify` / `--no-jira-ssl-verify`, `--jira-max-results` | Jira search tuning. |
| `--github-token` | GitHub API token. `GITHUB_TOKEN`. |
| `--ai-call-timeout` | AI timeout in minutes. |
| `--force-server-credentials` / `--no-force-server-credentials` | Use server-managed AI credentials instead of your own key. |
| `--raw-prompt` | Extra instructions appended to the AI prompt. |
| `--issue-prompt` | Custom issue-generation prompt, overriding `.rootcoz/ROOTCOZ_ISSUE_PROMPT.md`. |
| `--peers` | Peer models as `"provider:model,provider:model"`. |
| `--peer-analysis-max-rounds` | Debate rounds, 1-10. Server default is 3. |
| `--additional-repos` | Extra context repos as `"name:url,name:url"`. |
| `--wait` / `--no-wait`, `--poll-interval`, `--max-wait` | Wait for a still-running Jenkins build. |
| `--max-concurrent` | Cap concurrent AI calls. `0` keeps the config or server default. |
| `--failures`, `--passed-tests`, `--skipped-tests` | JSON arrays of raw test entries. Accepted only with `--source raw`; any other source rejects them with a validation error. `--source raw` requires at least one of the three. |
| `--tag` | Categorization tag. Repeatable. |
| `--label`, `-l` | Job metadata label merged on analyze. Repeatable, and distinct from `--tag`. |

Related lifecycle commands:

| Command | Purpose |
| --- | --- |
| `rootcoz results analyze JOB_ID` | Run AI on a job that was only ingested. |
| `rootcoz re-analyze JOB_ID` | Re-run a finished analysis with its original settings. |
| `rootcoz status JOB_ID` | Job status; also names the origin job for a re-analysis. |
| `rootcoz abort JOB_ID` | Abort a running or waiting analysis. |

6. **Read and review results.** `rootcoz results ...`

| Command | Flags | Purpose |
| --- | --- | --- |
| `list` | `--limit`/`-l` (50), `--analysis-state` | Recent jobs. Without `--analysis-state`, both `submitted` and `analyzed` jobs are returned; pass `--analysis-state analyzed` to restrict it. |
| `dashboard` | `--label`/`-l` (repeatable), `--exclude-tag` (repeatable), `--search`/`-s`, `--review-status` (`all`, `reviewed`, `not_reviewed`), `--analysis-state`, `--limit` (500) | Jobs with failure counts and review progress. |
| `show JOB_ID` | `--full`/`-f`, `--fields` | Summary by default, full JSON with `--full`, or a sparse projection with `--fields`. |
| `fields` | — | Print the allowlist of field paths accepted by `--fields`. |
| `tests JOB_ID` | `--status`/`-s` (`passed`, `skipped`, `failed`, repeatable), `--child-job`, `--child-build`, `--offset` (0), `--limit` (50, max 200) | Paginated test entries with `total` and `has_more`. |
| `review-status JOB_ID` | — | Failure, reviewed, and comment counts. |
| `set-reviewed JOB_ID` | `--test`/`-t`, `--reviewed`/`--not-reviewed`, `--child-job`, `--child-build` | Mark one failure reviewed. |
| `set-tracked-in JOB_ID` | `--test`/`-t`, `--url`/`-u`, `--type` (`jira` or `github`; omit for auto-detection), `--child-job`, `--child-build` | Link a failure to a ticket. |
| `delete-tracked-in JOB_ID` | `--link-id` | Remove one tracked-in link. |
| `enrich-comments JOB_ID` | — | Refresh live PR and ticket status on comments. |
| `delete [JOB_ID ...]` | `--all`, `--confirm` | Delete jobs and related data. `--all` requires `--confirm`. |

Sparse field paths include `result.summary`, `result.failed_count`, `result.failures.test_name`, `result.failures.classification`, and `result.failures.details`. Unknown paths are rejected with HTTP 400, so run `rootcoz results fields` first.

> **Note:** Pass `--child-job` before `--child-build`. The CLI validates the pair and rejects a build number with no job name, because the client drops `child_build_number` when `--child-job` is absent — an unpaired `--child-build` would otherwise be applied unscoped and silently. `results set-tracked-in` is included in that validation.

7. **Query failure history.** `rootcoz history ...`

| Command | Flags | Purpose |
| --- | --- | --- |
| `test TEST_NAME` | `--limit`/`-l` (20), `--job-name`/`-j`, `--exclude-job-id` | Failure rate, last classification, recent runs, and comments for one test. |
| `search` | `--signature`/`-s` (required), `--exclude-job-id` | Other tests that failed with the same error signature. |
| `stats JOB_NAME` | `--exclude-job-id` | Aggregate failure rate and most common failures. |
| `failures` | `--limit`/`-l` (50), `--offset`/`-o` (0), `--search`/`-s`, `--classification`/`-c`, `--job-name`/`-j` | Paginated failure list. |

8. **Classify failures and discuss them.**

| Command | Flags | Purpose |
| --- | --- | --- |
| `rootcoz classify TEST_NAME` | `--type`/`-t` (required: `FLAKY`, `REGRESSION`, `INFRASTRUCTURE`, `KNOWN_BUG`, `INTERMITTENT`), `--job-id` (required), `--reason`/`-r`, `--job-name`/`-j`, `--references`, `--child-job`, `--child-build` | Record a classification. |
| `rootcoz classifications list` | `--job-id`, `--test-name`/`-t`, `--type`/`-c`, `--job-name`/`-j`, `--parent-job-name` | Existing classifications. |
| `rootcoz comments list JOB_ID` | — | Comments on a job. |
| `rootcoz comments add JOB_ID` | `--test`/`-t`, `--message`/`-m`, `--child-job`, `--child-build` | Add a comment. |
| `rootcoz comments delete JOB_ID COMMENT_ID` | — | Remove a comment. |
| `rootcoz override-classification JOB_ID` | `--test`/`-t`, `--classification`/`-c` (`CODE ISSUE`, `PRODUCT BUG`, `INFRASTRUCTURE`), `--child-job`, `--child-build` | Correct the classification. |
| `rootcoz override-pattern JOB_ID` | `--test`/`-t`, `--pattern`/`-p` (`NEW`, `REGRESSION`, `FLAKY`, `INTERMITTENT`, `KNOWN_BUG`, `PERSISTENT`), `--child-job`, `--child-build` | Correct the failure pattern. |
| `rootcoz analyze-comment-intent COMMENT` | `--job-id`, `--ai-provider`, `--ai-model` | Ask AI whether a comment implies the failure was reviewed. |
| `rootcoz mentionable-users` | — | Usernames valid for `@mentions`. |
| `rootcoz mentions` | `--limit`/`-l` (50), `--offset`/`-o`, `--unread` | Your mentions across all reports. |
| `rootcoz mentions-mark-read` | `--ids` (comma-separated) | Mark specific mentions read. |
| `rootcoz mentions-mark-all-read` | — | Mark everything read. |

9. **Look up and re-analyze single failures.** `rootcoz failure ...`

| Command | Purpose |
| --- | --- |
| `failure show FAILURE_UUID` | Job, test, error, classification, and details for one failure. |
| `failure re-analyze FAILURE_UUID` | Re-run analysis for just that failure. |

10. **Manage job metadata.** `rootcoz metadata ...`

| Command | Flags | Purpose |
| --- | --- | --- |
| `list` | `--team`, `--tier`, `--version`, `--label`/`-l` (repeatable) | Filtered metadata rows. |
| `get JOB_NAME` | — | Metadata for one job. |
| `set JOB_NAME` | `--team`, `--tier`, `--version`, `--label`/`-l` (repeatable) | Create or update. |
| `delete JOB_NAME` | — | Remove metadata. |
| `import FILE` | — | Bulk load a JSON or YAML array of `{job_name, team, tier, version, labels}`. |
| `rules` | — | Auto-assignment rules and their source file. |
| `preview JOB_NAME` | — | What the rules would assign to a job name. |

11. **Create follow-up issues.**

| Command | Flags | Purpose |
| --- | --- | --- |
| `rootcoz capabilities` | — | Whether the server can create GitHub issues and Jira bugs. |
| `rootcoz jira-projects` | `--query`, `--jira-token`, `--jira-email` | Available Jira projects. |
| `rootcoz jira-security-levels PROJECT_KEY` | `--jira-token`, `--jira-email` | Security levels for a project. |
| `rootcoz preview-issue JOB_ID` | `--test`/`-t`, `--type` (`github` or `jira`), `--child-job`, `--child-build`, `--include-links`, `--ai-provider`, `--ai-model`, `--github-token`, `--github-repo-url`, `--jira-token`, `--jira-email`, `--jira-project-key`, `--jira-security-level`, `--issue-prompt` | Render issue title, body, and similar issues without creating anything. |
| `rootcoz get-issue-prompt JOB_ID` | — | The issue prompt attached to the job's tests repo. |
| `rootcoz create-issue JOB_ID` | `--test`/`-t`, `--type`, `--title`, `--body`, `--child-job`, `--child-build`, `--github-token`, `--github-repo-url`, `--jira-token`, `--jira-email`, `--jira-project-key`, `--jira-security-level`, `--jira-issue-type` (default `Bug`) | Create the issue. |
| `rootcoz validate-token github\|jira` | `--token` (hidden prompt), `--email` | Check a tracker token before using it. |
| `rootcoz ai-models` | `--provider`/`-p` | Model IDs per provider, with credential sources. |

12. **Push to exporters.**

| Command | Flags | Purpose |
| --- | --- | --- |
| `rootcoz exporters` | — | Available exporter plugins and their enabled state. |
| `rootcoz push JOB_ID` | `--plugin`/`-p` (required), `--child-job-name`, `--child-build-number` | Push results through a named exporter plugin. |
| `rootcoz push-reportportal JOB_ID` | `--child-job-name`, `--child-build-number` | Report Portal shortcut; `push-rp` is a hidden alias. |

13. **Chat with the analysis.**

| Command | Flags | Purpose |
| --- | --- | --- |
| `rootcoz chat init JOB_ID` | `--provider`/`-p`, `--model`/`-m` (both required), `--server-credentials` | Start a job-scoped chat and clone its repos. |
| `rootcoz chat send JOB_ID MESSAGE` | `--provider`/`-p`, `--model`/`-m`, `--server-credentials` | Send a message and wait up to about two minutes for the reply. |
| `rootcoz chat history JOB_ID` | `--limit`/`-l` (200) | Full transcript. |
| `rootcoz chat abort JOB_ID` | — | Stop the message being processed. |
| `rootcoz chat clear JOB_ID` | — | Delete job chat history. |
| `rootcoz chat close JOB_ID` | — | Signal you left the chat view. |
| `rootcoz admin-chat init` | `--provider`/`-p`, `--model`/`-m` (both required), `--server-credentials` | Start a server-wide admin chat. |
| `rootcoz admin-chat send MESSAGE` | `--provider`/`-p`, `--model`/`-m`, `--server-credentials` | Ask a cross-job question. |
| `rootcoz admin-chat history` | `--limit`/`-l` (200) | Admin transcript. |
| `rootcoz admin-chat clear` | — | Clear admin history and saved artifacts. |
| `rootcoz admin-chat save-artifact HTML_FILE` | `--filename`/`-f` | Upload a local HTML report. |
| `rootcoz admin-chat download-artifact ARTIFACT_ID` | `--output`/`-o` | Download an artifact; without `-o` it writes `report-<id>.html`, where `<id>` is the first eight characters of the artifact id. |

See [Use Server Chat for Cross-Job Analysis](use-server-chat-for-cross-job-analysis.html) for prompt patterns and artifact behaviour.

14. **Run analytics reports.** `rootcoz reports ...` — requires `admin` or `can_view_reports`.

`totals`, `overrides`, and `issues` take the same filter set:

| Option | Purpose |
| --- | --- |
| `--team`, `--tier`, `--version` | Metadata filters. |
| `--from`, `--to` | Date window as `YYYY-MM-DD`. |
| `--status` | Job status. |
| `--tags` | Comma-separated include list. |
| `--exclude-tags` | Comma-separated exclude list. |
| `--review-status` | `reviewed` or `not_reviewed`. |

15. **Administer the server.** `rootcoz admin ...`

| Command | Flags | Purpose |
| --- | --- | --- |
| `component-versions` | — | Installed AI-sidecar component versions. |
| `token-usage` | `--period` (`today`, `week`, `month`, `all`), `--start-date`, `--end-date`, `--provider`, `--model`, `--call-type`, `--group-by` (`provider`, `model`, `call_type`, `day`, `week`, `month`, `job`), `--job-id`, `--format` (`table`, `json`, `csv`) | AI token usage and cost. No filters prints the summary dashboard. |
| `users list` | — | All users, roles, and grants. |
| `users create USERNAME` | `--role` (required), `--can-view-reports`/`--no-can-view-reports`, `--can-use-server-providers`/`--no-can-use-server-providers` | Create a user; the API key is shown once. |
| `users delete USERNAME` | `--force`/`-f` | Delete a user without the confirmation prompt. |
| `users rotate-key USERNAME` | — | Issue a new key for a user. |
| `users change-role USERNAME ROLE` | — | Change role. Promoting to `admin` issues a key. |
| `users set-can-view-reports USERNAME VALUE` | — | Grant or revoke report access. `VALUE` accepts `true`/`false`, `1`/`0`, `yes`/`no`, `on`/`off`. |
| `users set-can-use-server-providers USERNAME VALUE` | — | Grant or revoke server AI credentials. |
| `users pending` | — | Registrations awaiting approval. |
| `users approve USERNAME` | `--can-use-server-providers`/`--no-can-use-server-providers` | Approve a registration. |
| `users reject USERNAME` | — | Reject a registration. |
| `settings list` | `--category`/`-c`, `--reveal` | Current values, sources, and categories. Values are masked unless `--reveal`. |
| `settings set KEY VALUE` | — | Persist a server setting. |
| `settings history` | `--key`/`-k`, `--limit`/`-l` (50) | Change log. |
| `settings reset KEY` | — | Drop the DB override and fall back to env or default. |
| `models refresh` | — | Re-read the model list from the sidecar. |
| `models free-list` | — | List admin-declared free provider+model pairs. |
| `models free-add` | `--provider`, `--model` | Declare one provider+model pair as free (validated against the catalog). |
| `models free-remove` | `--provider`, `--model` | Remove one pair from the free list. |

See [Managing Users and Server Settings](manage-users-and-server-settings.html) for what these settings mean.

## Advanced Usage

- `rootcoz results show JOB_ID --fields "result.summary,result.failures.test_name,result.failures.classification"` returns only what a report needs, which keeps large results cheap to poll.
- `rootcoz admin token-usage --group-by model --format csv > usage.csv` is the only command with a first-class CSV mode; everything else uses `--json`.
- `rootcoz --server staging` switches servers without editing any file, and `rootcoz config servers` shows which name maps to which URL.
- `rootcoz analyze --source file -f results.xml` works with no Jenkins or Prow access at all.
- `rootcoz auth ai-keys set claude --model <MODEL_ID> --stdin` reads the key from a pipe, so it never lands in shell history or a CI log.

## Troubleshooting

- `Error: No server specified.`
  Pass `--server NAME` or `URL`, set `ROOTCOZ_SERVER`, or add a `[default]` server to `~/.config/rootcoz/config.toml`.
- `Error: Server 'name' not found in config.`
  The `--server` value was treated as a config name and no `[servers.name]` entry exists. Run `rootcoz config servers`, or pass a full `http://` or `https://` URL to bypass the config lookup.
- `Error: 401` or a hint to use `--api-key`.
  No credentials reached the server. Set `ROOTCOZ_API_KEY`, add `api_key` to the server entry, or pass `--api-key`.
- `Error: 403` and a hint about the allow list or a higher role.
  Your account is not approved, or your role is too low for that action. An admin can fix this with `rootcoz admin users approve USERNAME` or `rootcoz admin users change-role USERNAME operator`.
- `rootcoz config show` prints `No config file found`.
  That is normal on a fresh machine. `rootcoz config show` prints the exact `mkdir` and heredoc needed to create one.
- `rootcoz results show JOB_ID --fields ...` returns HTTP 400.
  The path is not on the allowlist. Run `rootcoz results fields` and copy the path exactly.
- `--child-build requires --child-job`.
  Pass both flags together; there is no way to target a child build without naming its job.
- A `push` reports every test as unmatched.
  The exporter could not match test names. Confirm the plugin is enabled with `rootcoz exporters`, and check the child scoping flags if the job is a pipeline parent.

## Related Pages

- [Automating Common Tasks with the CLI](automate-common-tasks-with-the-cli.html)
- [Quickstart](quickstart.html)
- [Configuration Reference](configuration-reference.html)
- [API Endpoint Reference](api-reference.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Exploring History and Reports](explore-history-and-reports.html)


---

Source: automate-common-tasks-with-the-cli.md

# Automating Common Tasks with the CLI

The `rootcoz` CLI is the scripting surface for RootCoz. This page walks the workflows an integrator actually automates — authenticating without a prompt, submitting an analysis, waiting for it, pulling structured results, reviewing, and pushing to an exporter — and then composes them into one script. For the full flag list, see [CLI Command Reference](cli-reference.html).

## Prerequisites

- The `rootcoz` CLI on your `PATH`.
- A server URL and an API key. Get the key with `rootcoz register USERNAME` or from an administrator.
- `jq` for the examples below. Everything except the CSV mode of `rootcoz admin token-usage` is JSON with `--json`.

## Quick Example

```bash
    export ROOTCOZ_SERVER=https://rootcoz.example.com
    export ROOTCOZ_API_KEY=your-api-key

    JOB_ID=$(rootcoz --json analyze --source jenkins -j nightly -b 412 \
      --provider claude --model claude-opus-4-6 | jq -r .job_id)

    until [ "$(rootcoz --json status "$JOB_ID" | jq -r .status)" = "completed" ]; do sleep 15; done

    rootcoz --json results show "$JOB_ID" --fields result.summary,result.failed_count
```

That submits a Jenkins analysis, waits for it to finish, and prints a two-field summary — the whole submit-poll-read loop in six lines.

> **Warning:** The `until` loop above exits only on `completed`. A job that ends as `failed` or `aborted` never satisfies the condition, so the script polls forever instead of reporting the problem. In automation, branch on the terminal statuses and exit non-zero:

```bash
    STATUS=$(rootcoz --json status "$JOB_ID" | jq -r .status)
    case "$STATUS" in
      completed) rootcoz --json results show "$JOB_ID" --fields result.summary,result.failed_count ;;
      failed|aborted|error) echo "analysis $JOB_ID ended as $STATUS" >&2; exit 1 ;;
      *) echo "unexpected status $STATUS" >&2; exit 1 ;;
    esac
```

Bound the wait as well (for example `for i in $(seq 1 240)`) so a job stuck in `pending` or `running` cannot hang the pipeline. See [Tracking Analysis Progress](track-analysis-progress.html) for every status value.

## Step-by-Step

1. **Authenticate without any interactive prompt.**

   The CLI has no session store. `rootcoz auth login` only validates a key and prints your role; it does not save anything. For unattended runs, supply credentials in one of these ways, in order of preference:

   | Method | How |
   | --- | --- |
   | Environment | `export ROOTCOZ_SERVER=... ROOTCOZ_API_KEY=... ROOTCOZ_USERNAME=...` |
   | Config file | `api_key` and `username` under `[servers.NAME]` in `~/.config/rootcoz/config.toml` |
   | Per command | `rootcoz --server NAME --api-key KEY ...` |

   ```toml
   # ~/.config/rootcoz/config.toml
   [default]
   server = "prod"

   [servers.prod]
   url = "https://rootcoz.example.com"
   username = "ci-bot"
   api_key = "rootcoz_..."
   ```

   With that file in place, every later command drops `--server` and `--api-key`.

   ```bash
   rootcoz config servers   # confirm which name is the default
   rootcoz auth whoami      # confirm the role the key resolves to
   ```

   > **Warning:** A key written into a config file is plaintext. On a shared CI runner, prefer `ROOTCOZ_API_KEY` from a masked secret over a checked-in config file.

2. **Submit an analysis and capture the job ID.**

   `analyze` prints human-readable confirmation lines, so add `--json` when you need to pipe it.

   ```bash
   # Jenkins
   JOB_ID=$(rootcoz --json analyze --source jenkins -j nightly -b 412 \
     --provider claude --model claude-opus-4-6 | jq -r .job_id)

   # JUnit XML, no CI connectivity needed
   JOB_ID=$(rootcoz --json analyze --source file -f results.xml | jq -r .job_id)

   # Prow
   JOB_ID=$(rootcoz --json analyze --source prow -j my-periodic-test -b 20250731 \
     --prow-url https://prow.example.com --gcs-bucket prow-artifacts | jq -r .job_id)
   ```

   Use `rootcoz submit` instead of `analyze` when you want the CI results stored without paying for AI, then `rootcoz results analyze JOB_ID` to run the analysis later on the same job ID. Both take the same flags — `submit` is the same command registered under a second name, and the CLI posts to the ingest-only endpoint when you invoke it under that name.

   Tag and label the run so it shows up in filtered dashboards and reports:

   ```bash
   rootcoz --json analyze --source jenkins -j nightly -b 412 \
     --tag smoke --label team-alpha --label tier-1 | jq -r .job_id
   ```

   > **Note:** `--tag` categorizes the run; `--label` attaches job metadata. They are different fields and both are repeatable.

3. **Poll until the analysis completes.**

   `rootcoz status JOB_ID` returns the job status. In-progress statuses are `pending`, `running`, and `waiting`; `completed` is the terminal success state.

   ```bash
   while true; do
     STATE=$(rootcoz --json status "$JOB_ID" | jq -r .status)
     case "$STATE" in
       completed) echo "done: $JOB_ID"; break ;;
       failed|error|aborted) echo "analysis did not complete: $JOB_ID ($STATE)"; exit 1 ;;
     esac
     sleep 15
   done
   ```

   Cap the loop in real automation so a stuck job does not spin forever:

   ```bash
   for _ in $(seq 1 80); do
     case "$(rootcoz --json status "$JOB_ID" | jq -r .status)" in
       completed) break ;;
       failed|aborted) echo "analysis did not complete"; exit 1 ;;
     esac
     sleep 15
   done
   ```

   Handle the terminal status explicitly: a loop that only breaks on `completed` will fall through to the result-fetch step for a job that failed, was aborted, or is still running once the iterations run out, and the script will read as though the analysis succeeded. Check the status after the loop and treat `failed` and `aborted` as errors.

   `rootcoz abort JOB_ID` cancels a run you no longer want.

4. **Fetch the result.**

   `rootcoz results show JOB_ID` prints a one-line summary by default. For anything machine-readable, add `--json`, or project a subset with `--fields`.

   ```bash
   rootcoz results show "$JOB_ID"

   # Sparse payload: cheap to poll even for a large job
   rootcoz --json results show "$JOB_ID" \
     --fields result.summary,result.failed_count,result.ai_provider \
     | jq '.result'
   ```

   `rootcoz results fields` lists every accepted path. Unknown paths are rejected with HTTP 400, so check that list before building long `--fields` strings.

5. **Paginate through test entries.**

   `rootcoz results tests` returns `entries`, `total`, and `has_more` under `--json`. Page size is capped at 200.

   ```bash
   OFFSET=0
   while :; do
     PAGE=$(rootcoz --json results tests "$JOB_ID" --status failed --limit 200 --offset "$OFFSET")
     echo "$PAGE" | jq -r '.entries[] | "\(.test_name)\t\(.duration)"'
     [ "$(echo "$PAGE" | jq -r '.has_more')" = "true" ] || break
     OFFSET=$((OFFSET + $(echo "$PAGE" | jq -r '.entries | length')))
   done
   ```

   For pipeline jobs, scope to one child with `--child-job` and `--child-build`.

6. **Review and classify.**

   ```bash
   # Look at progress first
   rootcoz results review-status "$JOB_ID"

   # Mark a failure reviewed
   rootcoz results set-reviewed "$JOB_ID" --test "TEST_NAME" --reviewed

   # Override what the AI decided
   rootcoz override-classification "$JOB_ID" --test "TEST_NAME" --classification "PRODUCT BUG"
   rootcoz override-pattern "$JOB_ID" --test "TEST_NAME" --pattern "REGRESSION"

   # Add your own classification with a reason and a ticket link
   rootcoz classify "TEST_NAME" --type REGRESSION --job-id "$JOB_ID" \
     --reason "Fails only on the new checkout path" --references "PLAT-1234"

   # Record where the follow-up work lives
   rootcoz results set-tracked-in "$JOB_ID" --test "TEST_NAME" --url "https://jira.example.com/browse/PLAT-1234"

   # Leave a note for the next reviewer
   rootcoz comments add "$JOB_ID" --test "TEST_NAME" --message "Reproduced on the staging pool"
   ```

   If the job is a pipeline parent, add `--child-job CHILD_JOB --child-build 12345` to the write commands — `results set-reviewed`, `override-classification`, `override-pattern`, `classify`, `results set-tracked-in`, and `comments add` — so the change lands on the right failure. `results review-status` takes neither flag, so it always reports across the whole job.

7. **Push to an exporter.**

   ```bash
   rootcoz exporters                                        # which plugins are enabled
   rootcoz --json push "$JOB_ID" --plugin reportportal | jq .
   rootcoz --json push "$JOB_ID" --plugin reportportal \
     --child-job-name CHILD_JOB --child-build-number 12345
   ```

   Report Portal has a shortcut that skips `--plugin`:

   ```bash
   rootcoz --json push-reportportal "$JOB_ID"
   ```

   The response includes `pushed`, `errors`, `unmatched`, and `launch_id`, so an automation can fail loudly when `unmatched` is non-empty.

8. **Script report output.**

   Reports need `admin` or `can_view_reports`. All three subcommands take the same filters.

   ```bash
   rootcoz --json reports totals --team alpha --from 2025-01-01 --to 2025-06-30 \
     --review-status reviewed | jq '.total_jobs, .total_failures, .total_reviewed'

   rootcoz --json reports overrides --tags nightly,smoke | jq '.groups'
   rootcoz --json reports issues --team alpha | jq -r '.issues[] | "\(.issue_type)\t\(.title)"'

   # Trend lookup before you even submit anything
   rootcoz --json history failures --search tests.test_auth --classification INFRASTRUCTURE \
     --limit 50 | jq -r '.failures[].test_name'
   ```

   For cost reporting, `rootcoz admin token-usage` is the one command with a native CSV mode:

   ```bash
   rootcoz admin token-usage --period month --group-by model --format csv > usage.csv
   rootcoz --json admin token-usage --job-id "$JOB_ID" | jq '.records[].cost_usd'
   ```

9. **Compose it into one script.**

   ```bash
   #!/usr/bin/env bash
   set -euo pipefail

   : "${ROOTCOZ_SERVER:?set ROOTCOZ_SERVER}"
   : "${ROOTCOZ_API_KEY:?set ROOTCOZ_API_KEY}"

   JOB_NAME="${1:?usage: analyze.sh JOB_NAME BUILD_NUMBER}"
   BUILD_NUMBER="${2:?usage: analyze.sh JOB_NAME BUILD_NUMBER}"

   JOB_ID=$(rootcoz --json analyze --source jenkins -j "$JOB_NAME" -b "$BUILD_NUMBER" \
     --provider claude --model claude-opus-4-6 --tag ci \
     | jq -r '.job_id')
   echo "queued $JOB_ID"

   for _ in $(seq 1 80); do
     STATE=$(rootcoz --json status "$JOB_ID" | jq -r '.status')
     [ "$STATE" = "completed" ] && break
     [ "$STATE" = "failed" ] && { echo "analysis failed" >&2; exit 1; }
     sleep 15
   done

   SUMMARY=$(rootcoz --json results show "$JOB_ID" \
     --fields result.summary,result.failed_count | jq -c '.result')
   echo "summary: $SUMMARY"

   # Page through failed tests. --limit bounds a single page on its own, so a
   # job with more than 200 failed tests would leave the rest unreviewed and
   # unpushed. The API returns has_more, so follow it until it is false.
   OFFSET=0
   while :; do
     PAGE=$(rootcoz --json results tests "$JOB_ID" --status failed \
       --limit 200 --offset "$OFFSET")
     # Process the page first, then decide whether another one exists.
     jq -r '.entries[].test_name' <<<"$PAGE" | while read -r TEST; do
       rootcoz results set-reviewed "$JOB_ID" --test "$TEST" --reviewed
       rootcoz results set-tracked-in "$JOB_ID" --test "$TEST" \
         --url "https://jira.example.com/browse/PLAT-$BUILD_NUMBER"
     done
     [ "$(jq -r '.has_more' <<<"$PAGE")" = "true" ] || break
     OFFSET=$((OFFSET + 200))
   done

   rootcoz --json push "$JOB_ID" --plugin reportportal | jq '{pushed, unmatched, errors}'
   ```

   The same shape works for a cron-driven report: swap step 2 for `reports totals` and drop the polling loop entirely.

## Advanced Usage

- Pass `--api-key` from a CI secret and `--server` from a variable to run the same script against dev, staging, and prod without editing it.
- Use `rootcoz analyze --source file -f results.xml` to analyze artifacts downloaded after the fact, which is the easiest way to backfill historical CI runs.
- `rootcoz results delete --all --confirm` deletes jobs in bulk. The `--confirm` requirement exists so an accidental flag cannot wipe the server. The command collects job IDs from a single unpaginated `/api/dashboard` request, so on a server with more than 500 jobs it only sees the first page and leaves the rest in place. For a full cleanup, delete the remainder by explicit `JOB_ID` values from `rootcoz --json results list`.
- `rootcoz re-analyze JOB_ID` re-runs a finished analysis by replaying the job's stored request parameters, which is faster than resubmitting when the inputs are unchanged. Note what that means for models: because the replay includes the `ai_model` resolved when the job first ran, a model you have since changed on the server is **not** picked up. The `re-analyze` command itself takes no model option — it accepts only `JOB_ID` — so there is no CLI path to override it. To re-run with a different model, call `POST /re-analyze/{job_id}` directly with a JSON body containing `ai_model` (and `ai_provider`); the endpoint takes a `ReAnalyzeRequest` and merges it over the stored parameters. Do not use `POST /results/{job_id}/analyze` for this — that route only accepts jobs in the `submitted` state and returns `409` for an analyzed one.
- `rootcoz history search --signature HASH` finds every other test that failed with the same error signature, useful for deciding whether a new failure is really a new problem.
- `rootcoz --json admin-chat send "Which jobs regressed this week?"` is the fastest way to get a cross-job answer in a pipeline log.

## Troubleshooting

- The script hangs on the first command.
  The CLI is waiting for a prompt. `auth ai-keys set` prompts unless you pass `--stdin`, `validate-token` always prompts for `--token`, and `admin users delete` prompts unless you pass `--force`.
- `Error: No server specified.` on the first call.
  `ROOTCOZ_SERVER` is not exported in the script's environment, and no config file exists. Set both variables, or add a `[default]` server to `~/.config/rootcoz/config.toml`.
- `Error: 401` partway through a long script.
  The key expired or was rotated. Re-run `rootcoz auth whoami` to confirm, then issue a new one with `rootcoz auth rotate-key`.
- The poll loop never exits.
  A `--wait` submission stays in `waiting` until the CI build finishes. Add an iteration cap, and check `rootcoz status JOB_ID` manually to see which state it is stuck in.
- `results tests` returns fewer entries than the dashboard failure count.
  The dashboard counts analysis failures, while `results tests` pages the CI test entries. Raise `--limit` to 200 and keep incrementing `--offset` until `has_more` is `false`.
- `push` reports unmatched tests.
  The exporter could not line the test names up. Run `rootcoz exporters` to confirm the plugin is enabled, and add `--child-job-name` and `--child-build-number` for pipeline jobs.
- A `403` mentions `can_view_reports`.
  Your role is fine but the reports grant is missing. An admin can add it with `rootcoz admin users set-can-view-reports USERNAME true`.

## Related Pages

- [CLI Command Reference](cli-reference.html)
- [Quickstart](quickstart.html)
- [Submitting Analyses](submit-analyses.html)
- [Tracking Analysis Progress](track-analysis-progress.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [Exploring History and Reports](explore-history-and-reports.html)


---

Source: configure-analysis-context.md

# Configuring Analysis Context

RootCoz's analysis quality comes from what you give it: the repository that owns the failing tests, any additional repositories the failure touches, the AI models that review each other, the tracker it searches for duplicates, and the build artifacts it reads. This page covers those inputs, plus the per-project `.rootcoz/` files a repository can ship to teach RootCoz about itself.

## Prerequisites

- You can submit analyses, so your role is `operator` or `admin`.
- A reachable clone URL for the tests repository. Only `https://` and `git://` URLs are accepted.
- For a private tests repository, a token with read access to it.
- For Jira search, a Jira URL, project key, and credentials that can read the project.
- For per-project customization, write access to the repository that holds the `.rootcoz/` files.

## Quick Example

In `New Analysis`, fill the `Source Repositories` section:

| Field | Value |
| --- | --- |
| `Tests Repo URL` | `https://github.com/org/my-tests` |
| `Ref / Branch` | `main` |
| `Additional Repositories` | Two rows, one per repository: `Name` `api` with `URL` `https://github.com/org/api`, and `Name` `web` with `URL` `https://github.com/org/web` |

`Additional Repositories` is a list of inputs, not a single string: click `Add Repository` once per extra repository and fill the `Name`, `URL`, and optional `Ref` fields on that row. The comma-separated `name:url` spelling belongs to the CLI `--additional-repos` flag and the `additional_repos` server setting, not to this form.

This clones the tests repository plus both extra repositories into the analysis workspace and tells the AI that all three are available to read.

## Step-by-Step

1. **Point RootCoz at the tests repository.**

   `Tests Repo URL` is the repository that owns the failing tests — usually a test repository rather than the product repository. `Ref / Branch` pins the checkout; leave it empty to use the remote default branch. RootCoz clones with a shallow history and exposes the clone to the AI as a readable resource.

   The field is per submission. Setting it once on the server (`tests_repo_url` in the `GitHub` category of Server Settings) makes it the default for every later submission.

   | Detail | Behaviour |
   | --- | --- |
   | URL schemes | `https://` and `git://` only |
   | Clone directory | Derived from the repository name; it must not collide with an additional repository name |
   | Files it unlocks | Any `.rootcoz/` files in that repository, which are loaded after the clone completes |

2. **Authenticate with the tests repository.**

   Private repositories need a token. There are two ways to supply one:

   | Source | Field | Notes |
   | --- | --- | --- |
   | Per submission | `tests_repo_token` on the request | Encrypted at rest with the rest of the job parameters |
   | Server default | `tests_repo_token` in the `GitHub` server settings category | A database override wins over the environment variable |

   Additional repositories use their own per-repository token, set in the request, not in `.rootcoz/settings.json`.

   > **Note:** A token given in the request takes priority over the server setting for that job. Leave both empty for public repositories.

3. **Add repositories the failure actually touches.**

   `Additional Repositories` accepts a list of `name`, `url`, and optional `ref` entries. Each clone becomes a subdirectory of the workspace, and the AI is told the path and what is in it — for example that a repository contains VCS metadata it can browse.

   | Rule | Behaviour |
   | --- | --- |
   | `name` | Becomes the clone directory name. No slashes, no `..`, no leading dot. |
   | Reserved name | `build-artifacts` is reserved for downloaded artifacts and is rejected. |
   | Collision | A name matching the tests repository clone directory fails the job. |
   | `ref` | Branch, tag, or commit. Empty means the remote default branch. |

   The server-side equivalent is the `additional_repos` setting, stored as `name:url` or `name:url:ref` separated by commas. `max_concurrent_repo_clones` caps how many clones run at once.

4. **Add peer AI configs for cross-model review.**

   Turn on `Enable peer review` in the `Peer Analysis` section and add one entry per peer model. The main model analyzes first, the peers review in parallel, and the loop continues until they agree or the round limit is reached. Nobody has veto power — it is a debate that ends in a consensus answer.

   | Field | Server setting | Default |
   | --- | --- | --- |
   | Peer list | `peer_ai_configs`, stored as `provider:model,provider:model` | empty (no peers) |
   | Debate rounds | `peer_analysis_max_rounds`, between 1 and 10 | 3 |

   `claude`, `gemini`, and `cursor` are convenience aliases, not the only accepted values — as with `AI_PROVIDER`, any exact provider ID that pi-sidecar reports is accepted, and the chart's restricted `ai.provider` enum does not apply here. Peer models cost real tokens and real wall-clock time, so start with one peer and a small round limit.

   > **Tip:** Peer models need credentials. Either they use your own stored key or the server's, depending on the `Use server credentials` permission you were granted.

5. **Turn on Jira search for duplicate bugs.**

   Enable `Enable Jira search` in the `Jira Integration` section and supply `Jira URL` and `Project Key`. While analyzing, RootCoz asks the model for three to five short `jira_search_keywords` for each `PRODUCT BUG` instead of generic words, and uses them to find existing bugs before creating a new one. The same pattern produces `tests_repo_search_keywords` for matching GitHub issues.

   | Server setting | What it controls |
   | --- | --- |
   | `enable_jira` | Whether the search runs |
   | `jira_url` | Base URL of the Jira instance |
   | `jira_project_key` | Project to search in |
   | `jira_max_results` | Maximum matches returned per search; default 5 |
   | `enable_jira_issues` | Whether new Jira bugs can be created, independently of search |

   Search is enrichment only. Creating issues still requires working credentials, and a per-user `Jira Email` and `Jira Token` lets created issues land under your name. See [Managing Your Account and Notifications](manage-account-and-notifications.html) for details.

6. **Collect build artifacts when text output is not enough.**

   For a Jenkins run, `Fetch build artifacts` downloads the build's artifacts into a `build-artifacts/` directory in the workspace, subject to `Max Size (MB)` (`jenkins_artifacts_max_size_mb`, default 500). For a Prow run the same toggle downloads the non-JUnit artifacts from GCS.

   The prompt tells the AI that `build-artifacts/` exists, and every classification it produces carries an `artifacts_evidence` field with file references in the form `[build-artifacts/logs/app.log]: ...`. That is how screenshots and log files become evidence in the report instead of guesswork from the JUnit XML alone.

   | Setting | Default | Notes |
   | --- | --- | --- |
   | `get_job_artifacts` | on | Runtime toggle per submission or server-wide |
   | `jenkins_artifacts_max_size_mb` | 500 | Size cap for the download |

   Turn artifact collection off for large builds where the console output already tells the whole story — it is the slowest part of setting up a job.

7. **Let the repository configure itself with `.rootcoz/`.**

   Any cloned repository can carry a `.rootcoz/` directory that changes how it is analyzed:

   ```text
   <analyzed-repo>/
     .rootcoz/
       settings.json                  # Per-repo analysis settings
       ROOTCOZ_PROMPT.md              # Custom analysis instructions
       ROOTCOZ_HISTORY_PROMPT.md      # Custom history analysis instructions
       ROOTCOZ_ISSUE_PROMPT.md        # Custom issue generation prompt
       agents/                        # Project pi agents
       skills/                        # Project pi skills
       extensions/                    # Project pi extensions
   ```

   RootCoz discovers this after the tests repository is cloned. Only files under `.rootcoz/` are recognized; the old prompt filenames in the repository root are no longer supported.

8. **Use `.rootcoz/settings.json` for per-repo defaults.**

   This file is the only schema-validated file in the directory. Every key is optional, unknown keys are rejected, and secrets are not allowed.

   ```json
   {
     "ai_provider": "claude",
     "ai_model": "claude-sonnet-4",
     "ai_call_timeout": 15,
     "max_concurrent_ai_calls": 4,
     "peer_ai_configs": [
       { "ai_provider": "gemini", "ai_model": "gemini-2.5-pro" }
     ],
     "peer_analysis_max_rounds": 2,
     "force_server_credentials": true,
     "additional_repos": [
       { "name": "api", "url": "https://github.com/org/api", "ref": "main" }
     ]
   }
   ```

   | Key | Type | Purpose |
   | --- | --- | --- |
   | `ai_provider` | string | Provider for this repository's analyses |
   | `ai_model` | string | Model for this repository's analyses |
   | `ai_call_timeout` | integer > 0 | Per-call timeout, in minutes |
   | `max_concurrent_ai_calls` | integer > 0 | Parallel AI calls |
   | `peer_ai_configs` | array of `{ai_provider, ai_model}` | Peer reviewers |
   | `peer_analysis_max_rounds` | integer 1–10 | Debate round limit |
   | `force_server_credentials` | boolean | Use server keys instead of personal keys |
   | `additional_repos` | array of `{name, url, ref}` | Repositories to clone alongside the tests repo |

   Resolution order for every one of these keys is **request → `.rootcoz/settings.json` → server settings**. Because the file is read from a repository you do not control, loading it is deliberately strict:

   | Constraint | Behaviour |
   | --- | --- |
   | Schema | Validated against `src/rootcoz/schemas/rootcoz-settings.schema.json` |
   | Unknown or secret keys | Rejected. A `token` key anywhere in the file fails validation. |
   | Size | 256 KiB maximum |
   | Path | Must be a regular file inside the repository. Symlinks and paths escaping the repo are rejected. |
   | Invalid file | The job fails with a validation error rather than silently ignoring the file. |

   > **Warning:** Do not put credentials in this file. It is read from an untrusted clone and is not encrypted anywhere in the pipeline. Use per-repository tokens in the request or server settings instead.

9. **Write the prompt files for project-specific instructions.**

   | File | Applies to | How it is loaded |
   | --- | --- | --- |
   | `ROOTCOZ_PROMPT.md` | Failure analysis | Discovered in each cloned repository; the prompt advertises the path and the AI is instructed to open it before analyzing. |
   | `ROOTCOZ_HISTORY_PROMPT.md` | History-aware classification | Only advertised when history-aware classification is active — the job has a server URL and a bearer token. |
   | `ROOTCOZ_ISSUE_PROMPT.md` | Issue and bug generation | Fetched from the tests repository through the GitHub Contents API, so a tests repo token is needed for private repositories. |

   The prompt body is never inlined into the analysis prompt. RootCoz tells the AI that the file exists and requires it to read the file with its `read` tool, which keeps prompts small and lets you iterate on instructions without touching RootCoz.

   > **Note:** `ROOTCOZ_ISSUE_PROMPT.md` is only consulted when the job does not already carry an issue prompt from the request.

10. **Ship project pi resources.**

    After cloning, `agents/`, `skills/`, and `extensions/` are copied from `.rootcoz/` into the workspace's `.pi/` directory so the agent runtime discovers them for analysis, re-analysis, and chat. Project agents are listed to the AI in a first-step gate and named in the resources section. Built-in agents are copied afterwards and never overwrite a project agent with the same name.

    | Directory | Purpose |
    | --- | --- |
    | `agents/` | Project agents whose files the AI is pointed at for analysis guidance |
    | `skills/` | Project skills the agent runtime loads |
    | `extensions/` | Project extensions the agent runtime loads |

    Symlinks are skipped rather than followed. However, a `settings.json` that exists but fails validation is not merely logged — the analysis stops and the job is marked failed with the reason redacted. Treat a malformed `.rootcoz/settings.json` as a job-failing condition, not a warning.

    > **Warning:** Anything you add under `.rootcoz/` ships with your repository and steers AI behavior for everyone who analyzes it. Treat it as reviewed code, and document every new file you add — the repository's `AGENTS.md` requires that each repo-side customization file is described alongside the code that reads it.

## Advanced Usage

- **Per-request raw prompt.** `Raw Prompt` on `New Analysis` is appended to the analysis prompt for that one submission. Use it for one-off nudges; use `ROOTCOZ_PROMPT.md` for instructions that should apply every time.
- **Tune timeouts and concurrency.** `ai_call_timeout` (minutes, default 10) and `max_concurrent_ai_calls` (default 3) are the two knobs most worth raising for large, slow analyses. Both can be set per submission, in `.rootcoz/settings.json`, or server-wide.
- **Defer the AI choice.** When a tests repository is set and no provider or model is chosen, RootCoz can resolve the AI configuration from `.rootcoz/settings.json` after the clone, so the repository picks its own model.
- **Repository-wide defaults.** `tests_repo_url`, `additional_repos`, `peer_ai_configs`, `peer_analysis_max_rounds`, `ai_call_timeout`, and `max_concurrent_ai_calls` all exist as server settings in the `AI`, `GitHub`, and `Server` categories, which makes them defaults for every submission. See [Managing Users and Server Settings](manage-users-and-server-settings.html) for details.
- **Metadata rules.** `metadata_rules_file` extracts team, tier, and version metadata from job names so history and reports can filter by them. It requires a restart when changed.
- **Auto-review.** `enable_auto_review` marks a failure reviewed automatically when the same job, test, and error signature was reviewed by a human before — useful once you have history to match against.

## Troubleshooting

- The AI cannot find the test source.
  The tests repository is not set for this submission, or the clone failed. Check `Tests Repo URL` on the submission and the clone progress on the status page.

- The analysis does not mention `.rootcoz/ROOTCOZ_PROMPT.md`.
  The file lives in the repository root, not in `.rootcoz/`, or it is in a repository that was not cloned. Only `.rootcoz/` is recognized.

- `settings.json` is ignored.
  It is only read from the tests repository, after that repository is cloned. A `settings.json` in an additional repository is not read.

- The job fails with `Unknown keys in .rootcoz/settings.json`.
  Only the eight keys in the table above are allowed. Remove anything else — including secrets like `token`.

- The job fails with `Invalid additional_repos url`.
  The URL must be a valid HTTP(S) URL — `additional_repos` entries are validated as `HttpUrl`, so `git://` is rejected — and the entry must have a non-empty `name`.

- The job fails with `additional_repos contains name '…' which collides with the tests repo clone directory`.
  Rename the additional repository so it differs from the tests repository's clone directory name.

- Peer review never converges and burns tokens.
  Lower `peer_analysis_max_rounds` or reduce the number of peers. Ten rounds across several models is the most expensive configuration.

- Jira or GitHub search returns nothing.
  Check that search is enabled, that the credentials can read the target, and that the generated keywords are specific. The model is told to avoid generic terms like `timeout` or `failure`; keyword quality is what narrows the results.

- A private repository fails to clone.
  Supply `tests_repo_token`, or a per-repository token for an additional repository. Embedded credentials in the clone URL are not used.

- Artifacts are missing from the report.
  Artifact collection was off, the size cap was hit, or the artifacts live outside what the job uploaded. `build-artifacts/` only contains what the CI system published.

## Related Pages

- [Submitting Analyses](submit-analyses.html)
- [Reviewing and Classifying Failures](review-and-classify-failures.html)
- [Managing Users and Server Settings](manage-users-and-server-settings.html)
- [Creating Follow-Up Issues and Pushing Results](create-follow-up-issues-and-push-results.html)
- [Configuration Reference](configuration-reference.html)

---
