---
name: infra-ops
description: >
  Pysae runtime ops on Kubernetes (EKS), ArgoCD, Helm — inspect pods,
  check sync state, scale, restart, rollback, debug crashes, exec into
  pods, view logs and events, look at resource usage. Use when the user
  mentions kubectl, argocd, helm, a pod/deployment/namespace/cluster,
  or asks "is X synced", "why is X down", "scale X", "restart X",
  "rollback X", "logs of X", or any runtime state question on
  dev/prod/review/test. The service/deployment/argo-app/datadog-service
  catalogue lives in references/pysae-projects.md — always consult it
  first to map a service alias to its canonical names. Does NOT trigger
  a CI deploy (use /ci-deploy),
  does NOT cover API performance metrics (use /infra-perf), does NOT
  manage AWS Secrets (use /env-secret).
argument-hint: "[service] [env: dev|prod|review|test]"
allowed-tools:
  - Bash
  - Read
  - Glob
  - Grep
  - AskUserQuestion
  - Skill
  - mcp__kubernetes
  - mcp__datadog
  - mcp__gitlab
  - mcp__mongodb-dev
  - mcp__mongodb-prod
  - mcp__mongodb-atlas-mcp-dev
  - mcp__mongodb-atlas-mcp-prod
---

Pysae DevOps agent — operate, inspect, and troubleshoot the Pysae infrastructure across all environments.

## Required tools

- `aws`
- `kubectl`
- `argocd`
- `helm`
- `glab`
- `atlas`
- `gitlab-mcp`
- `datadog-mcp`
- `mongo-tools`
- `mongo-mcp-dev`
- `mongo-mcp-prod`
- `mongodb-atlas-mcp`

Run `pysae-ai-tools tools require <tools>` before starting; if it exits non-zero, abort and report the install command from stderr to the user.

## Mode detection

```bash
CTX=$(pysae-ai-tools internal detect-context)
IS_CI=$(echo "$CTX" | jq -r '.is_ci')
```

In CI mode (`IS_CI=true`):
- **Never use `AskUserQuestion`** — no interactive prompts.
- **Read-only only**: every mutating verb (`kubectl scale`, `kubectl rollout undo`, `argocd app sync`, `argocd app rollback`, `kubectl delete`, `kubectl exec` with state-changing commands) is **forbidden**. If the user request implies a mutation, refuse with a clear message and exit 1.
- **Output structured for MR comments** — prefer concise tables/markdown over conversational prose.

## Tool routing

When the user asks about runtime infra, use the **most specific tool**, not the most familiar one. ArgoCD apps are stored as Kubernetes CRDs, so `kubectl get application` works — but it bypasses ArgoCD's auth, cache, and sync semantics. Always prefer the dedicated CLI/MCP for the layer the user is asking about.

| User intent | Use this | Don't use |
|---|---|---|
| « is X synced », « argo status », « sync drift », « last sync » | `argocd app get <app>`, `argocd app diff <app>`, `argocd app history <app>` | `kubectl get application` (works but shunts ArgoCD auth/cache) |
| « manual sync », « refresh argo », « force sync » | `argocd app sync <app>`, `argocd app refresh <app> --hard` | `kubectl patch application` |
| « pod status », « why is X crashing », « ready/restart count » | `kubectl get pods` / `mcp__kubernetes__kubectl_get` | `argocd app get` (gives sync, not pod health) |
| « pod logs », « crash logs », « previous logs » | `kubectl logs --previous` / `mcp__kubernetes__kubectl_logs` | Datadog logs (slower, ingestion lag) |
| « events », « what happened », « recent events » | `kubectl get events -n <ns> --sort-by=.lastTimestamp` | — |
| « scale », « replicas », « HPA » | check HPA first, then `kubectl scale` / `mcp__kubernetes__kubectl_scale` | Edit Helm values (bypasses HPA) |
| « rollback deployment », « undo », « previous version » | `kubectl rollout undo` (immediate fix) then `argocd app rollback` (realign GitOps) | — |
| « helm release », « deployed values », « chart version » | `helm get values`, `helm get manifest`, `helm history` | `kubectl get cm -l owner=helm` (raw, unparsed) |
| « diff git vs deployed » | `argocd app diff <app>` (rendered manifests) or `helm diff upgrade` (values) | `kubectl diff` (doesn't read the source repo) |
| « error logs » (cross-service correlation) | `mcp__datadog__get_logs` | `kubectl logs` (one pod at a time) |
| « latency / error rate / P99 » | delegate to `/infra-perf` | — |
| « monitor state », « SLO » | `mcp__datadog__list_monitors`, `mcp__datadog__list_slos` | — |
| « find/aggregate on Mongo » | `mcp__mongodb-{dev,prod}__find` / `aggregate` | direct `mongosh` (only if MCP is down) |
| « MongoDB Atlas cluster state », « tier », « network access » | `mcp__mongodb-atlas-mcp-{dev,prod}__*` | — |
| « slow Mongo queries », « performance advisor », « index suggestions » | `atlas -P {pysae-dev,pysae-prod} performanceAdvisor slowQueryLogs list` / `suggestedIndexes list` | — |
| « Atlas host metrics » (CPU, IOPS, RAM, connections) | `atlas -P {pysae-dev,pysae-prod} metrics processes <host>` / `metrics disks` / `metrics databases` | Datadog (Atlas integration may lag) |
| « cluster events », « failover », « maintenance window » | `atlas -P {pysae-dev,pysae-prod} events list` | — |
| « mongod logs » (audit, slow, full) | `atlas -P {pysae-dev,pysae-prod} logs download <host>` | `kubectl logs` (Atlas is managed, not in cluster) |
| « open Atlas alerts » | `atlas -P {pysae-dev,pysae-prod} alerts list` | — |
| « deployment status from GitLab » (who deployed what when) | `mcp__gitlab__get_deployment`, `mcp__gitlab__list_environments` | `kubectl describe deployment` (annotations only) |
| « secret value » | delegate to `/env-secret` | `kubectl get secret -o yaml` (forbidden — leaks) |

## Architecture overview

Pysae runs on **AWS EKS** with **ArgoCD** for GitOps deployments.

| Layer | Stack |
|-------|-------|
| Orchestration | ArgoCD (GitOps, syncs from `argo-apps` repo) |
| Runtime | AWS EKS (Kubernetes) — `dev` and `prod` clusters |
| Charts | Helm charts colocated in each repo's `helm/` directory |
| Registry | AWS ECR (`596368530845.dkr.ecr.eu-west-3.amazonaws.com/apps/`) |
| Monitoring | Datadog (EU endpoint: `api.datadoghq.eu`) — use MCP tools when available |
| DB | MongoDB Atlas (managed) |
| Auth | Auth0 |
| CI/CD | GitLab CI (shared templates from `pysae/infra/gitlab-templates` branch `2.x`) |

## Reference files

- **[../../references/pysae-projects.md](../../references/pysae-projects.md)** — canonical source for all project mappings: repos, K8s deployments, ArgoCD apps, Datadog services, AWS resources, Prefect flows. **Always consult this first.**
- **[references/commands.md](references/commands.md)** — exact kubectl, helm, argocd, and Datadog commands

## Environments

| Env | Namespace | kubectl context | ArgoCD app pattern | Usage |
|-----|-----------|-----------------|-------------------|-------|
| dev | `dev` | `dev` | `dev-<service>` | Development, main branch |
| prod | `prod` | `prod` | `prod-<service>` | Production |
| review | `mr` | `dev` | `review-<slug>-<service>` | Per-MR review environments |

## Workflow by intent

For multi-step scenarios. Single-tool questions go through the Tool routing table above.

### "is X synced on argo"
1. `argocd app get <env>-<service>` → Sync + Health status
2. If `OutOfSync`, `argocd app diff <env>-<service>` → show drift
3. If user wants to fix: ask before `argocd app sync <env>-<service>` (especially prod)

### "why is X down / crashing"
1. `kubectl get pods -n <ns> -l app=<service>` → CrashLoopBackOff, OOMKilled, ImagePullBackOff, …
2. `kubectl describe pod <pod>` → events (image pull, scheduling, OOM)
3. `kubectl logs --previous <pod>` → crash trace
4. Cross-reference Datadog logs (broader window) via `mcp__datadog__get_logs`
5. If root cause is a CI/CD failure → delegate to `/ci-debug`

### "scale X"
1. Check current state: `kubectl get hpa -n <ns> <service>` + replicas
2. If HPA active: adjust min/max via Helm values (don't manual scale, HPA reverts)
3. If no HPA: `kubectl scale deployment/<service> --replicas=<N>` after user confirmation
4. Verify pod readiness

### "rollback X"
1. `kubectl rollout history deployment/<service> -n <ns>` → list revisions
2. `kubectl rollout undo deployment/<service> -n <ns>` → immediate fix
3. `argocd app rollback <env>-<service>` → realign GitOps source of truth
4. Or delegate to `/ci-rollback` if the user wants a CI-driven rollback

### "exec into a pod"
1. `kubectl exec -it -n <ns> <pod> -- /bin/sh` (or `/bin/bash` if available)
2. For a one-liner: `kubectl exec -n <ns> <pod> -- <command>`

### "top RAM/CPU consumers"
1. `kubectl top pods -n <ns> --sort-by=memory` (or `--sort-by=cpu`)
2. Compare with requests/limits: `kubectl describe pod <pod> | grep -A 2 Limits`
3. Cross-reference with Helm values for tuning suggestions

### "is the DB slow / Mongo is laggy"
1. `atlas -P pysae-prod clusters describe <cluster>` → tier, state, ongoing maintenance
2. `atlas -P pysae-prod performanceAdvisor slowQueryLogs list --processName <host>` → top slow ops
3. `atlas -P pysae-prod metrics processes <host> --period PT1H` → CPU, IOPS, connections over the last hour
4. `atlas -P pysae-prod performanceAdvisor suggestedIndexes list` → missing indexes
5. Cross-reference with API latency (`/infra-perf`) to confirm DB is the bottleneck

The Atlas profile (`-P pysae-dev` / `-P pysae-prod`) is configured at install time from AWS Secrets Manager; never pass API keys on the command line.

### "finops / cost analysis"
1. List resources per namespace: deployments, pods, PVCs (`kubectl get all,pvc -n <ns>`)
2. Check resource requests/limits in Helm values
3. Cross-reference with `pysae-projects.md` for service ownership
4. Suggest right-sizing, idle resources, unused PVCs

## Critical rules

- **NEVER delete resources** without explicit user confirmation
- **NEVER expose secrets** — `kubectl get secret` shows names only, never `-o yaml` on secrets
- **NEVER modify prod** without double confirmation ("Tu confirmes pour la PROD ?")
- **Prefer read-only operations** — inspect before acting
- **Rollback/scale**: always ask confirmation, show current state first
- **ArgoCD is the source of truth** for deployments — prefer ArgoCD sync over manual kubectl apply
- **Prefer MCP tools** for Datadog and Kubernetes when available (e.g. `mcp__datadog__get_logs`, `mcp__kubernetes__kubectl_get`)
- **Language**: explanations in French, commands in English

## Self-check before output

- [ ] Target service/namespace correctly identified (cross-checked with `pysae-projects.md`)
- [ ] Environment confirmed (dev/prod/review)
- [ ] Read-only operations done before any mutation
- [ ] User confirmation obtained for any write operation
- [ ] No secrets exposed in output
- [ ] Post-action verification suggested or performed

$ARGUMENTS
