# MCA SDK Demo Notebooks

## end-to-end-demo.ipynb

**Comprehensive demonstration of MCA SDK v0.6.1 with ALL features**

### What This Notebook Demonstrates

#### Core Infrastructure (Cells 1-4)
1. **GCP ADC Authentication** - Verify Application Default Credentials
2. **OTel Collector Connectivity** - Test OTLP HTTP/gRPC ports
3. **Model Config from Registration API** - Fetch model configuration
4. **SDK Initialization** - Initialize with registry integration

#### Feature Demonstrations (Cells 5-11)

5. **Model Input/Output Capture** ⭐ NEW in v0.6.0 (Fixed in v0.6.1)
   - Decorator method: `@predict(capture_input=True, capture_output=True)`
   - Manual method: `record_prediction(input_data=..., output=...)`
   - Automatic credential sanitization (AWS/GCP keys, JWT tokens)
   - Configurable via `capture_input_data` and `capture_output_data` settings

6. **GenAI/LLM Tracking with Token Costs**
   - Track LLM completions with `track_llm_completion()`
   - Automatic cost estimation (GPT-4, GPT-3.5, etc.)
   - Metrics: `genai.requests_total`, `genai.tokens_total`, `genai.cost_usd`

7. **Custom Metrics**
   - Counters: `record_counter()` for incrementing values
   - Gauges: `record_gauge()` for point-in-time measurements
   - Histograms: `record_metric()` for distributions

8. **Agentic AI Goal Tracking**
   - `record_goal_started()` / `record_goal_completed()`
   - Goal lifecycle spans with duration tracking
   - Metrics: `agent.goals_started`, `agent.goals_completed`, `agent.goal_duration_seconds`

9. **Human Intervention Recording**
   - `record_human_intervention()` for HITL tracking
   - Override reasons and decision provenance
   - Metric: `agent.human_interventions_total`

10. **Error Tracking**
    - `record_error()` for exception capture
    - Metric: `model.errors_total` by error type
    - Stack traces in structured logs

11. **Telemetry Flush**
    - `client.shutdown()` to export all pending telemetry
    - Wait for GCP propagation

#### Export Verification (Cells 12-14)

12. **Cloud Logging** - Query `emms-model-telemetry` logs
13. **Cloud Trace** - Verify distributed traces
14. **Prometheus Metrics** - Check collector self-monitoring

### Prerequisites

- **Environment**: Vertex AI Workbench or GCP compute instance
- **Network**: VPN connection if outside GKE cluster
- **Access**: Cloud Logging, Cloud Trace read permissions
- **Collector**: OTel Collector running at 10.164.76.55:4318

### How to Use

1. **Open notebook** in Jupyter or Vertex AI Workbench
2. **Run cells sequentially** (top to bottom)
3. **Wait 30 seconds** after cell 11 for GCP propagation
4. **Verify exports** in cells 12-14

### Expected Results

✓ All SDK features demonstrated
✓ Telemetry exported to OTel Collector
✓ Logs visible in Cloud Logging
✓ Traces visible in Cloud Trace
✓ Metrics available in Prometheus (internal cluster only)

### Complete Architecture

```
Registration API
    ↓
SDK v0.6.1 (all features with critical bug fixes)
    ├─ Input/Output Capture
    ├─ Prediction Tracking
    ├─ GenAI/LLM Tracking
    ├─ Custom Metrics
    ├─ Agentic AI Tracking
    ├─ Human Intervention
    └─ Error Tracking
    ↓ OTLP HTTP :4318
OTel Collector
    ├─→ Cloud Logging
    ├─→ Cloud Trace
    └─→ Prometheus
```

### Key Features

**v0.6.1 Critical Fixes:**
- Fixed context token handling in agentic AI goal tracking
- All users should upgrade to v0.6.1 immediately

**v0.6.0 Features:**

- **Input/Output Capture** (Story 1-18): Drift monitoring with credential sanitization
- **Comprehensive Testing**: End-to-end, integration, load tests
- **GCP Auth**: Automatic credential handling for Registry API
- **High Availability**: Collector HA features (HPA, persistent volumes, DLQ)

### Access Your Data

- **Logs**: https://console.cloud.google.com/logs/query
  - Filter: `logName="projects/bhsf-ai-team-projects/logs/emms-model-telemetry"`
- **Traces**: https://console.cloud.google.com/traces/list
  - Search: `service.name="clinical-notes-summarizer-v2"`
- **Metrics**: Contact DevOps for Grafana dashboard URLs

### Troubleshooting

**Collector not reachable**
- Check VPN connection
- Verify firewall rules
- Test ports: `nc -zv 10.164.76.55 4318`

**No logs/traces found**
- Wait 1-2 minutes for propagation
- Re-run feature demos (cells 5-11)
- Check collector is running: `kubectl get pods -n observability`

**Import errors**
- Upgrade SDK: `pip install mca-sdk[gcp,genai]>=0.6.1 --upgrade`
- Verify installation: `pip show mca-sdk`

### Related Documentation

- [MCA SDK README](../README.md)
- [API Contracts](../../docs/api-contracts.md)
- [Integration Guides](../../docs/integration-guides/)
- [Registry GCP Auth Guide](../../docs/registry-gcp-auth-guide.md)
