# MCA SDK for Go

OpenTelemetry instrumentation library for ML model monitoring in Go.

## Installation

```bash
go get gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go
```

## Quick Start

```go
package main

import (
    "context"
    "log"

    mca "gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go"
    "gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go/config"
)

func main() {
    // Create client configuration
    cfg := &config.Config{
        ServiceName: "readmission-model",
        ModelID:     "mdl-001",
        TeamName:    "clinical-ai",
        TLSInsecure: true, // Only for local development
    }

    // Initialize client with context
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    client, err := mca.NewClient(ctx, cfg)
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }
    defer client.Shutdown()

    // Record predictions
    latency := 0.15
    pred := &mca.Prediction{
        Latency:      &latency, // Use pointer to distinguish 0.0 from nil
        PredictionID: "pred-123",
        Attributes: map[string]string{
            "model_version": "2.0",
            "input_size":    "1024",
        },
    }

    if err := client.RecordPrediction(context.Background(), pred); err != nil {
        log.Printf("Failed to record prediction: %v", err)
    }
}
```

## Configuration

### Required Fields

- `ServiceName` - Service name for resource attributes
- `ModelID` - Model identifier
- `TeamName` - Team name for resource attributes

### Optional Fields (with defaults)

- `ModelVersion` - Model version (default: "1.0.0")
- `ModelType` - Model type: internal, vendor, genai, agentic (default: "internal")
- `ModelCategory` - Model category: internal, external (default: "internal")
- `CollectorEndpoint` - OTel Collector endpoint (default: "http://localhost:4318")
- `Protocol` - OTLP protocol: http, grpc (default: "http")
- `TLSInsecure` - Allow insecure TLS (default: false, set true only for local dev)
- `FilterSystemMetrics` - Drop process.* and runtime.* metrics to reduce costs (default: true)

### Configuration from Environment

```go
cfg := config.LoadFromEnv()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
client, err := mca.NewClient(ctx, cfg)
```

Environment variables:
- `MCA_SERVICE_NAME` - Service name (required)
- `MCA_MODEL_ID` - Model ID (required)
- `MCA_TEAM_NAME` - Team name (required)
- `MCA_MODEL_VERSION` - Model version
- `MCA_COLLECTOR_ENDPOINT` - Collector endpoint
- `MCA_OTEL_PROTOCOL` - Protocol (http or grpc)
- `MCA_TLS_INSECURE` - Allow insecure TLS (true/false)
- `MCA_FILTER_SYSTEM_METRICS` - Filter system metrics (true/false, default: true)

## API Reference

### `NewClient(ctx context.Context, cfg *config.Config) (*Client, error)`

Creates and initializes a new MCA Client. The context is used for provider initialization with timeout to prevent hangs.

### `RecordPrediction(ctx context.Context, pred *Prediction) error`

Records a model prediction with metrics, logs, and traces. Creates a trace span for correlation, increments the prediction counter, and records latency if provided (use `*float64` pointer - `nil` means no latency, `&0.0` records zero latency).

### `Flush(ctx context.Context) error`

Forces all OpenTelemetry providers to flush their buffers immediately. Useful for batch processing scenarios where you want to ensure telemetry is exported without shutting down the client.

### `Shutdown() error`

Gracefully shuts down all OpenTelemetry providers. Use with `defer` for automatic cleanup.

### Agentic AI Tracking

#### `RecordGoalStarted(ctx context.Context, description string, goalType string, attributes map[string]interface{}) (string, error)`

Starts tracking an agent goal and returns a unique goal ID. Creates a span and increments the goals_started counter.

#### `RecordGoalCompleted(ctx context.Context, goalID string, status string, attributes map[string]interface{}) error`

Completes tracking of an agent goal. Valid statuses: "success", "failure", "partial".

#### `TrackTool(ctx context.Context, toolName string, goalID string, attributes map[string]interface{}, fn func(context.Context) error) error`

Wraps tool execution with automatic timing and error handling. The provided function is executed and metrics are recorded.

#### `RecordHumanIntervention(ctx context.Context, reason string, waitTime float64, interventionType string, timestamp time.Time) error`

Records when the agent requires human assistance. Adds a span event to the current active span.

#### `RecordPolicyViolation(ctx context.Context, policyType string) error`

Records a security policy violation event. Adds a span event and increments the policy violations counter.

### Advanced Core Tracking

#### `RecordError(ctx context.Context, err error, latency *float64, predictionID string, attributes map[string]interface{}) error`

Records a prediction error with dual metrics (predictions_total{status="error"} and errors_total) and detailed telemetry.

#### `RecordEvaluation(ctx context.Context, metrics map[string]float64, dataset string, attributes map[string]interface{}) error`

Records model evaluation metrics (MAE, RMSE, etc.). The metrics map should contain keys like "mae" and "rmse".

## Important Notes

### Latency as Pointer

`Prediction.Latency` uses `*float64` to distinguish between:
- `nil` - No latency provided
- `&0.0` - Valid zero latency (e.g., cache hit)

```go
// No latency
pred := &mca.Prediction{PredictionID: "pred-1"}

// Zero latency
latency := 0.0
pred := &mca.Prediction{Latency: &latency, PredictionID: "pred-2"}

// Non-zero latency
latency := 0.15
pred := &mca.Prediction{Latency: &latency, PredictionID: "pred-3"}
```

### Attribute Limits

- Maximum 50 custom attributes per prediction
- String values truncated to 256 characters
- Excess attributes are ignored with warning logged

## Idiomatic Go Patterns

### Error Handling

```go
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
client, err := mca.NewClient(ctx, cfg)
if err != nil {
    return fmt.Errorf("failed to create client: %w", err)
}
```

### Context Propagation

```go
ctx := context.Background()
err := client.RecordPrediction(ctx, pred)
```

### Resource Cleanup with Defer

```go
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
client, err := mca.NewClient(ctx, cfg)
if err != nil {
    return err
}
defer client.Shutdown()
```

### Goroutine Safety

The client is goroutine-safe and can be used concurrently:

```go
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
    wg.Add(1)
    go func(id int) {
        defer wg.Done()
        latency := 0.15
        pred := &mca.Prediction{Latency: &latency}
        client.RecordPrediction(ctx, pred)
    }(i)
}
wg.Wait()
```

## Advanced Usage Examples

### Agentic AI Workflow

```go
// Start a goal
goalID, err := client.RecordGoalStarted(ctx, "Research treatment options", "research", map[string]interface{}{
    "priority": "high",
})
if err != nil {
    log.Fatalf("Failed to start goal: %v", err)
}

// Track tool execution
err = client.TrackTool(ctx, "pubmed_search", goalID, map[string]interface{}{
    "query": "diabetes treatment",
}, func(ctx context.Context) error {
    // Execute tool logic
    results, err := pubmedClient.Search(ctx, "diabetes")
    if err != nil {
        return err
    }
    log.Printf("Found %d results", len(results))
    return nil
})
if err != nil {
    log.Printf("Tool execution failed: %v", err)
}

// Record human intervention if needed
if needsApproval {
    err = client.RecordHumanIntervention(ctx, "expert_review_needed", 120.0, "review", time.Now())
    if err != nil {
        log.Printf("Failed to record intervention: %v", err)
    }
}

// Complete the goal
err = client.RecordGoalCompleted(ctx, goalID, "success", map[string]interface{}{
    "tasks_completed": 5,
})
if err != nil {
    log.Fatalf("Failed to complete goal: %v", err)
}
```

### Error Recording

```go
func predict(ctx context.Context, data interface{}) (interface{}, error) {
    start := time.Now()
    result, err := model.Predict(data)
    latency := time.Since(start).Seconds()
    
    if err != nil {
        // Record the error with latency
        if recordErr := client.RecordError(ctx, err, &latency, "", nil); recordErr != nil {
            log.Printf("Failed to record error: %v", recordErr)
        }
        return nil, err
    }
    
    // Record successful prediction
    pred := &mca.Prediction{
        Latency:      &latency,
        PredictionID: uuid.New().String(),
    }
    if err := client.RecordPrediction(ctx, pred); err != nil {
        log.Printf("Failed to record prediction: %v", err)
    }
    
    return result, nil
}
```

### Model Evaluation

```go
// After evaluating model on test set
metrics := map[string]float64{
    "mae":  0.05,
    "rmse": 0.08,
}
err := client.RecordEvaluation(ctx, metrics, "test", map[string]interface{}{
    "model_variant": "v2",
    "dataset_size":  10000,
})
if err != nil {
    log.Printf("Failed to record evaluation: %v", err)
}
```

### Policy Violation Tracking

```go
func accessSensitiveData(ctx context.Context, userID string) error {
    if !user.HasPermission("admin") {
        // Record policy violation
        client.RecordPolicyViolation(ctx, "unauthorized_admin_access")
        return errors.New("permission denied")
    }
    // ... proceed with access
    return nil
}
```

### Batch Processing with Flush

```go
// Process batch of predictions
for i := 0; i < 1000; i++ {
    pred := &mca.Prediction{PredictionID: fmt.Sprintf("pred-%d", i)}
    client.RecordPrediction(ctx, pred)
}

// Flush telemetry after batch
if err := client.Flush(ctx); err != nil {
    log.Printf("Failed to flush telemetry: %v", err)
}
```

## Testing

```bash
# Run unit tests
go test -v ./...

# Run tests with race detection
go test -race ./...

# Run tests with coverage
go test -cover ./...
```

## Architecture

The SDK connects to OpenTelemetry Collector via OTLP/HTTP:

```
Go Application
    ↓
MCA SDK Client
    ↓
OTLP/HTTP (port 4318)
    ↓
OpenTelemetry Collector
    ↓
Backends (Prometheus, Cloud Trace, Cloud Logging)
```

## Requirements

- Go 1.21+
- OpenTelemetry Collector running on localhost:4318 (or configured endpoint)

## License

Apache 2.0
