Yes, Cloud Run can support thousands of users, but the current implementation is still a pilot, not production-ready.

  Current Gaps

  - Widget agency/public-key validation exists only in browser JavaScript. Anyone can bypass it.
  - API endpoints are public without rate limiting or server-side tenant validation.
  - --allowed-origin=* should not be used for an agency deployment.
  - Google Sheets writes happen synchronously during chat completion.
  - Current Cloud Run settings allow only 3 × 8 = 24 simultaneous requests.
  - Sessions use Redis correctly, but are not associated with an agency.
  - A session UUID alone grants access to that conversation.

  Recommended Architecture

  Agency website
      |
  Production widget from CDN
      |
  HTTPS Load Balancer
      |
  Cloud Armor: rate limits and bot protection
      |
  Cloud Run API: stateless, autoscaling
      |-- Redis Standard Tier: conversation sessions and locks
      |-- Firestore/PostgreSQL: agencies, domains, leads, configuration
      |-- LLM provider
      `-- Cloud Tasks: asynchronous CRM/Sheets delivery

  Required API Changes

  1. Add an agency registry containing:
      - agency_id
      - allowed domains
      - branding and agent configuration
      - enabled/disabled status
      - destination CRM configuration

  2. Add a bootstrap endpoint:

  POST /api/widget/session
  Origin: https://agency.example
  {
    "agency_id": "mccone",
    "public_key": "pk_live_..."
  }

  The server validates the agency and origin, then returns a short-lived signed session token.

  3. Require that token for chat requests:

  Authorization: Bearer <short-lived-session-token>

  4. Store sessions using an agency namespace:

  agency:{agency_id}:session:{session_id}

  5. Submit completed leads to Cloud Tasks. A worker saves the lead and delivers it to Sheets or the CRM with retries and idempotency.
  6. Keep service-account credentials and LLM keys in Secret Manager.

  Capacity
  Do not size from registered users. Size from simultaneous LLM requests:

  required concurrency = requests/second × average LLM latency

  Example:

  - 1,000 active visitors
  - One message every 30 seconds
  - Approximately 33 requests/second
  - Seven-second LLM latency
  - Approximately 231 simultaneous requests
  - At concurrency 8: approximately 29 Cloud Run instances

  Your current maximum of three instances would be insufficient for that scenario. The LLM provider’s rate and token quotas may become the bottleneck before Cloud Run.

  Start with:

  CPU: 2
  Memory: 2 GiB
  Concurrency: 8
  Minimum instances: 1-2
  Maximum instances: determined by LLM quota, initially perhaps 20
  Redis: Standard Tier
  Session TTL: 24 hours

  Then load-test complete conversations before increasing limits. Cloud Run currently supports up to 1,000 concurrent requests per instance, but Google recommends beginning with lower
  concurrency, such as 8, and tuning from measurements.

  Deployment Phases

  1. Single-agency pilot: fixed server-side domain, Redis, Cloud Armor, Cloud Tasks, durable lead storage.
  2. Limited production: tenant registry, signed widget sessions, monitoring, billing limits, backups and deletion policies.
  3. Multi-agency platform: agency administration, per-tenant quotas, isolated configuration, usage accounting, audit records and CRM connectors.
  validation in demo/widget.js:5 with server-side tenant/session authentication.

  Sources: Cloud Run concurrency (https://cloud.google.com/run/docs/about-concurrency), Cloud Run maximum instances (https://cloud.google.com/run/docs/configuring/max-instances), Cloud
  Armor rate limiting (https://cloud.google.com/armor/docs/rate-limiting-overview), Redis tiers (https://cloud.google.com/memorystore/docs/redis/redis-tiers), Cloud Tasks queues
  (https://cloud.google.com/tasks/docs/configuring-queues).
