Nika (AI-SRE-Agent) Production Architecture

Multi-Account Autonomous Incident Investigation, Root Cause Analysis (RCA), and Vector-Based Memory System

Alert Event Ingestion Query Response P1 Fallback Generate Embeddings Recall Memory (Vector Search) Publish Episode (Async) Embed Write Memory Archive Trace Query RCAs Slack Webhooks (RCA Threads) Socket Mode Feedback (👍/👎/Corrections) sts:AssumeRole (OIDC Read-Only Access) ALERT SOURCES (5 SPOKE AWS ACCOUNTS) CloudWatch Alarms Native AWS Metrics & Logs Prometheus / Alertmgr EKS Cluster Alerts Datadog / PagerDuty SaaS Monitors & Webhooks Zenduty / Grafana Incident Escalation / Dashboards Slack Workspace #oncall-support • Threaded RCA Reports • Engineer Feedback (👍/👎) • Multi-Turn Q&A in Thread ai-sre-alerts-queue SQS (Central Account) AWS Bedrock Claude 3.5 Sonnet CENTRAL ACCOUNT: Vegapay-AI (g5.xlarge EC2 Spot Instance - docker-compose) HolmesGPT Operator ReAct Loop / Slack Bot • AWS CLI / Kubernetes Tools • Action Execution Engine vLLM Server Qwen3.6-27B-AWQ (GPU) • High-Throughput Local Reasoning • Configurable Severity trigger Embedder (CPU) bge-base-en-v1.5 • 440MB RAM Footprint • 200 embeddings / sec Memory Worker Async Processing • Episode Extraction & Decay • Feedback Loop Classifier RCA Dashboard FastAPI + Web UI (Tier 2) • Search & Filter Past RCAs • Feedback Stats & MTTR VPN / SSO Protected S3 RCA Archive Cold Compliance (Tier 3) • SSE-KMS Encrypted • 2-Year Regulatory Retention • Full Tool Execution Traces PostgreSQL + pgvector Hot Memory Store (Tier 1) • Investigations (Episodic) • Knowledge & Facts (Semantic) • Runbook Chunks (Procedural) SPOKE AWS ACCOUNTS (Banking Partners - Read-Only access via OIDC AssumeRole) Yes Bank Dedicated Account • payment-service • core-banking Unity Bank Dedicated Account • settlement-service • ledger Capri Global Shared Cluster • capri-api • db-replica Ebix Cash Shared Cluster • ebix-gateway • auth-proxy BOB Dedicated Account • bob-cards • settlement Legend Frontend / UI Backend / Agent Cloud / Spoke Database / Memory OIDC / AssumeRole Alert Ingestion

Multi-Account Trust & Security

  • • Hub-and-Spoke OIDC Trust: Central Vegapay-AI account assumes roles in spoke accounts using temporary read-only credentials (1hr TTL).
  • • Zero Secrets Stored: No banking passwords or long-lived API keys are kept in the database.
  • • Read-Only Scoping: IAM policies limit the agent to logs (CloudWatch), metrics (Prometheus), and container states (K8s) — no write permissions.

Three-Tier Memory System

  • • Tier 1: Hot Memory (Postgres/pgvector): HNSW indexing on 1024-dimensional embeddings allows instant semantic search of past incidents.
  • • Tier 2: Visible Memory: Slack threads hold real-time investigation transcripts. FastAPI Dashboard allows full-text search and MTTR analytics.
  • • Tier 3: Cold Memory (S3): Complete raw tool execution traces are archived in S3 with SSE-KMS encryption for a 2-year regulatory retention period.

Feedback Loop & Learning

  • • Socket Mode Listener: Captures engineer reactions (👍/👎) and direct text corrections in Slack threads.
  • • Auto-Decay & Hygiene: Memory Worker updates confidence scores based on feedback. Low-confidence memories decay over time to prevent LLM context pollution.
  • • Auto-Evolving Runbooks: Successfully resolved incidents promote runbook chunks, making subsequent investigations faster and more precise.

Nika v0.2.2 • Designed for Vegapay Production Environments