# Agent Message Bus — Bootstrap Guide

This document covers the **stage 0** prerequisites that must exist before
`pulumi up` can provision infrastructure (stage 1) and before the application
can be deployed (stage 2).

## Deployment Stages

| Stage | What | How |
|-------|------|-----|
| **0 — Bootstrap** | Manual one-time setup (this doc) | `gcloud` CLI / GCP Console |
| **1 — Infrastructure** | All GCP resources (28 total) | `pulumi up --stack prod` |
| **2 — Application** | Build & push Docker image, populate secrets | `docker build` + `docker push` + `gcloud secrets versions add` |

## Current Deployment Status

| Resource | Value |
|----------|-------|
| **GCP Project** | `internal-airbyte-ai-infra` (project number: 351139268520) |
| **Load Balancer IP** | `34.54.112.44` |
| **Internal Cloud Run Service** | `https://internal-agent-bus-cloudrun-vuy6wf6uta-uc.a.run.app` (INTERNAL_LOAD_BALANCER ingress) |
| **Slack Cloud Run Service** | `https://slack-webhook-cloudrun-vuy6wf6uta-uc.a.run.app` (INGRESS_TRAFFIC_ALL) |
| **Firestore Database** | `(default)` in `us-central1` |
| **Artifact Registry** | `us-central1-docker.pkg.dev/internal-airbyte-ai-infra/agent-message-bus` |
| **Cloud Armor Policy** | `agent-message-bus-armor` |
| **Pulumi Stack** | `prod` on `gs://internal-airbyte-ai-infra-pulumi-state` |

## Stage 0: Bootstrap Checklist

> **Status: COMPLETE** — All bootstrap items have been executed.

### 1. Create GCP Project

```bash
gcloud projects create internal-airbyte-ai-infra --name="Internal Airbyte AI Infra"
```

Link a billing account:

```bash
gcloud billing projects link internal-airbyte-ai-infra --billing-account=BILLING_ACCOUNT_ID
```

### 2. Create GCS Bucket for Pulumi State

```bash
gcloud storage buckets create gs://internal-airbyte-ai-infra-pulumi-state \
  --project=internal-airbyte-ai-infra \
  --location=us-central1 \
  --uniform-bucket-level-access

gcloud storage buckets update gs://internal-airbyte-ai-infra-pulumi-state \
  --versioning
```

Then configure the Pulumi backend:

```bash
pulumi login gs://internal-airbyte-ai-infra-pulumi-state
```

### 3. Authenticate

The operator running `pulumi up` needs GCP credentials with Owner or
Editor role on `internal-airbyte-ai-infra`:

```bash
gcloud auth application-default login
```

### 4. Build and Push Docker Image (before first `pulumi up`)

Cloud Run requires the container image to exist before the service can be created:

```bash
cd infra/agent-message-bus/app
docker build -t us-central1-docker.pkg.dev/internal-airbyte-ai-infra/agent-message-bus/agent-message-bus:latest .
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
docker push us-central1-docker.pkg.dev/internal-airbyte-ai-infra/agent-message-bus/agent-message-bus:latest
```

### 5. Create Placeholder Secret Versions (before first `pulumi up`)

Cloud Run also needs at least one version of each secret to exist:

```bash
for secret in github-webhook-secret subscription-api-bearer-token slack-signing-secret devin-ai-api-key zendesk-webhook-signing-secret; do
  echo -n "PLACEHOLDER_REPLACE_ME" | \
    gcloud secrets versions add "$secret" --project=internal-airbyte-ai-infra --data-file=-
done
```

---

## Stage 1: Infrastructure (`pulumi up`)

> **Status: COMPLETE** — 31 resources deployed.

```bash
cd infra/agent-message-bus
source venv/bin/activate
pulumi up --stack prod --yes
```

---

## Post-Deploy Manual Steps (after stage 1)

These are **not** chicken-and-egg problems — they require stage 1 outputs.

### Populate Real Secret Values

Replace the placeholder secret values with real ones:

```bash
echo -n "YOUR_GITHUB_WEBHOOK_SECRET" | \
  gcloud secrets versions add github-webhook-secret --data-file=- --project=internal-airbyte-ai-infra

echo -n "YOUR_SLACK_SIGNING_SECRET" | \
  gcloud secrets versions add slack-signing-secret --data-file=- --project=internal-airbyte-ai-infra

echo -n "YOUR_BEARER_TOKEN" | \
  gcloud secrets versions add subscription-api-bearer-token --data-file=- --project=internal-airbyte-ai-infra

echo -n "YOUR_DEVIN_API_KEY" | \
  gcloud secrets versions add devin-ai-api-key --data-file=- --project=internal-airbyte-ai-infra

echo -n "YOUR_ZENDESK_WEBHOOK_SIGNING_SECRET" | \
  gcloud secrets versions add zendesk-webhook-signing-secret --data-file=- --project=internal-airbyte-ai-infra
```

After updating secrets, redeploy both Cloud Run services to pick up new values:

```bash
for svc in internal-agent-bus-cloudrun slack-webhook-cloudrun; do
  gcloud run services update "$svc" --region=us-central1 \
    --project=internal-airbyte-ai-infra --update-env-vars="DEPLOY_TS=$(date +%s)"
done
```

### Register GitHub Org Webhook

Admin URL: <https://github.com/organizations/airbytehq/settings/hooks>

- **Payload URL**: `http://34.54.112.44/github/webhook`
- **Content type**: `application/json`
- **Secret**: Same value used for `github-webhook-secret` above
- **Events**: `issue_comment`, `issues`, `pull_request`

### Register Zendesk Webhook

Admin URL: Zendesk Admin Center → Apps & Integrations → Webhooks → Create Webhook

- **Endpoint URL**: `http://34.54.112.44/zendesk/webhook`
- **Request method**: POST
- **Request format**: JSON
- **Authentication**: None (verified via HMAC signing secret + Cloud Armor IP allowlist)
- **Signing secret**: Copy from Zendesk → store as `zendesk-webhook-signing-secret` in Secret Manager

Then create a Trigger (Admin Center → Objects & Rules → Triggers):
- **Conditions**: Ticket is Created
- **Actions**: Notify webhook → JSON body with `ticket_id`, `subject`, `description`, `comments`

### Configure Slack App Request URL

Admin URL: <https://api.slack.com/apps/A0AFSNS2Q7Q/interactive-messages>

- **Request URL**: `https://slack-webhook-cloudrun-vuy6wf6uta-uc.a.run.app/slack/webhook`

> **Note:** The Slack webhook uses the dedicated `slack-webhook-cloudrun` service URL
> (not the GCLB IP) because Slack requires HTTPS with a valid certificate.

---

## CI/CD: Workload Identity Federation (WIF) Setup

GitHub Actions workflows (`pulumi-preview.yml`, `pulumi-apply-command.yml`, `docker-build-deploy.yml`)
use **Workload Identity Federation** for keyless GCP authentication. This must be configured once.

### 1. Create a Workload Identity Pool

```bash
gcloud iam workload-identity-pools create "github-actions" \
  --project="internal-airbyte-ai-infra" \
  --location="global" \
  --display-name="GitHub Actions"
```

### 2. Create an OIDC Provider

```bash
gcloud iam workload-identity-pools providers create-oidc "github" \
  --project="internal-airbyte-ai-infra" \
  --location="global" \
  --workload-identity-pool="github-actions" \
  --display-name="GitHub" \
  --attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository" \
  --attribute-condition="assertion.repository=='airbytehq/airbyte-ops-mcp'" \
  --issuer-uri="https://token.actions.githubusercontent.com"
```

### 3. Create a Service Account for CI

```bash
gcloud iam service-accounts create github-actions-ci \
  --project="internal-airbyte-ai-infra" \
  --display-name="GitHub Actions CI"
```

Grant the required roles:

```bash
# For Pulumi (infra management)
gcloud projects add-iam-policy-binding internal-airbyte-ai-infra \
  --member="serviceAccount:github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com" \
  --role="roles/editor"

# For Docker push to Artifact Registry
gcloud projects add-iam-policy-binding internal-airbyte-ai-infra \
  --member="serviceAccount:github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com" \
  --role="roles/artifactregistry.writer"

# For Cloud Run deploy
gcloud projects add-iam-policy-binding internal-airbyte-ai-infra \
  --member="serviceAccount:github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com" \
  --role="roles/run.admin"

# For Pulumi GCS state backend
gcloud projects add-iam-policy-binding internal-airbyte-ai-infra \
  --member="serviceAccount:github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com" \
  --role="roles/storage.admin"
```

### 4. Allow GitHub Actions to Impersonate the Service Account

```bash
gcloud iam service-accounts add-iam-policy-binding \
  github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com \
  --project="internal-airbyte-ai-infra" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/351139268520/locations/global/workloadIdentityPools/github-actions/attribute.repository/airbytehq/airbyte-ops-mcp"
```

### 5. Workflow Configuration

The WIF provider and service account values are hardcoded directly in the workflow files
(this is an internal repo and these are not secrets):

| Value | Location in workflow files |
|-------|---------------------------|
| `projects/351139268520/locations/global/workloadIdentityPools/github-actions/providers/github` | `workload_identity_provider:` field in `google-github-actions/auth` steps |
| `github-actions-ci@internal-airbyte-ai-infra.iam.gserviceaccount.com` | `service_account:` field in `google-github-actions/auth` steps |
