# Testing Go MCA SDK in Google Workbench

## Prerequisites
- Go 1.21+ installed
- Docker and Docker Compose available
- Repository cloned to Workbench

## Step 1: Start OTel Collector

```bash
cd /home/coder/emms/sdk/mca-prototype
docker-compose up -d otel-collector

# Verify collector is running
curl http://localhost:13133/
# Should return: Alive!

# Check collector logs
docker-compose logs otel-collector
```

## Step 2: Install Go Dependencies

```bash
cd /home/coder/emms/sdk/mca-sdk-go
go mod tidy
```

## Step 3: Run Unit Tests

```bash
# Run all unit tests
go test -v ./...

# Run with race detection (important for concurrent code)
go test -race ./...

# Run with coverage
go test -cover ./...

# Expected output:
# PASS
# coverage: ~XX% of statements
# ok      gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go    X.XXXs
```

## Step 4: Create Integration Test Script

Create `test_integration_test.go` in `mca-sdk-go/`:

```go
package mca

import (
	"context"
	"fmt"
	"testing"
	"time"

	"gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go/config"
)

// TestIntegration verifies end-to-end SDK functionality with OTel Collector.
func TestIntegration(t *testing.T) {
	fmt.Println("Starting Go MCA SDK Integration Test...\n")

	// Create client configuration
	cfg := &config.Config{
		ServiceName:       "go-test-model",
		ModelID:           "mdl-go-001",
		ModelVersion:      "1.0.0",
		TeamName:          "test-team",
		ModelType:         "internal",
		CollectorEndpoint: "http://localhost:4318",
		Protocol:          "http",
		TLSInsecure:       true,
	}

	// Initialize client with timeout
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	client, err := NewClient(ctx, cfg)
	if err != nil {
		t.Fatalf("Failed to create MCA client: %v", err)
	}
	defer client.Shutdown()
	fmt.Println("✓ MCAClient initialized successfully")

	// Test 1: Record prediction with latency
	latency1 := 0.123
	pred1 := &Prediction{
		Latency:      &latency1,
		PredictionID: "pred-001",
		Attributes: map[string]string{
			"model_version": "v1.0",
		},
	}
	if err := client.RecordPrediction(context.Background(), pred1); err != nil {
		t.Errorf("Failed to record prediction: %v", err)
	}
	fmt.Println("✓ Recorded prediction with latency")

	// Test 2: Record prediction without latency
	pred2 := &Prediction{
		PredictionID: "pred-002",
		Attributes: map[string]string{
			"custom_attribute": "test-value",
		},
	}
	if err := client.RecordPrediction(context.Background(), pred2); err != nil {
		t.Errorf("Failed to record prediction: %v", err)
	}
	fmt.Println("✓ Recorded prediction without latency")

	// Test 3: Multiple rapid predictions
	for i := 0; i < 5; i++ {
		latency := 0.1 + float64(i)*0.05
		pred := &Prediction{
			Latency:      &latency,
			PredictionID: fmt.Sprintf("pred-batch-%d", i),
		}
		if err := client.RecordPrediction(context.Background(), pred); err != nil {
			t.Errorf("Failed to record batch prediction %d: %v", i, err)
		}
	}
	fmt.Println("✓ Recorded 5 batch predictions")

	// Wait for metrics to export
	fmt.Println("\n⏳ Waiting 10 seconds for metrics to export...")
	time.Sleep(10 * time.Second)

	fmt.Println("✓ Client will shutdown gracefully via defer\n")
	fmt.Println("Integration test completed!")
}
```

## Step 5: Run Integration Test

```bash
# Run integration test
go test -v -run TestIntegration

# Expected output:
# === RUN   TestIntegration
# Starting Go MCA SDK Integration Test...
#
# ✓ MCAClient initialized successfully
# ✓ Recorded prediction with latency
# ✓ Recorded prediction without latency
# ✓ Recorded 5 batch predictions
# ⏳ Waiting 10 seconds for metrics to export...
# ✓ Client will shutdown gracefully via defer
#
# Integration test completed!
# --- PASS: TestIntegration (11.XXs)
# PASS
```

## Step 6: Verify Telemetry Data

### Check OTel Collector Logs
```bash
docker-compose logs otel-collector | grep -i "go-test-model"
docker-compose logs otel-collector | grep -i "model.predictions"

# Look for:
# - OTLP/HTTP requests received on port 4318
# - Metrics exported
# - No errors
```

### Query Prometheus (if configured)
```bash
curl -s http://localhost:9090/api/v1/query?query=model_predictions_total | jq .

# Expected: Counter showing prediction count
```

### Check Debug Exporter Output
```bash
docker-compose logs otel-collector | grep "model.predictions_total"

# Should show:
# - Metric name: model.predictions_total
# - Value: incremented count
# - Attributes: service.name, model.id, etc.
```

## Step 7: API Parity Validation with Python SDK

Run both SDKs side-by-side to verify equivalent behavior:

### Python SDK Test
```bash
cd /home/coder/emms/sdk/mca-prototype
python3 << 'EOF'
from mca_sdk import MCAClient

client = MCAClient(
    service_name="python-test-model",
    model_id="mdl-python-001",
    model_version="1.0.0",
    team_name="test-team",
    model_type="internal",
    otel_endpoint="http://localhost:4318",
    strict_validation=False
)

client.record_prediction(latency=0.123, prediction_id="pred-py-001")
print("✓ Python SDK prediction recorded")

client.shutdown()
print("✓ Python SDK shutdown complete")
EOF
```

### Go SDK Test (already done in Step 5)

### Compare Outputs
```bash
# Check that both SDKs produce similar OTLP payloads
docker-compose logs otel-collector | grep "python-test-model" | tail -20
docker-compose logs otel-collector | grep "go-test-model" | tail -20

# Verify:
# ✓ Same metric names (model.predictions_total, model.latency_seconds)
# ✓ Same resource attributes (service.name, model.id, model.version, team.name)
# ✓ Similar payload structure
```

## Step 8: Performance Validation

Create `test_performance_test.go`:

```go
package mca

import (
	"context"
	"fmt"
	"testing"
	"time"

	"gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go/config"
)

// TestPerformance measures SDK overhead and throughput.
func TestPerformance(t *testing.T) {
	cfg := &config.Config{
		ServiceName:       "perf-test",
		ModelID:           "mdl-perf",
		ModelVersion:      "1.0",
		TeamName:          "test",
		ModelType:         "internal",
		CollectorEndpoint: "http://localhost:4318",
		TLSInsecure:       true,
	}

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	client, err := NewClient(ctx, cfg)
	if err != nil {
		t.Fatalf("NewClient() error: %v", err)
	}
	defer client.Shutdown()

	iterations := 1000
	start := time.Now()

	for i := 0; i < iterations; i++ {
		latency := 0.1
		pred := &Prediction{Latency: &latency}
		if err := client.RecordPrediction(context.Background(), pred); err != nil {
			t.Errorf("RecordPrediction() error: %v", err)
		}
	}

	duration := time.Since(start)
	avgLatency := duration.Milliseconds() / int64(iterations)
	throughput := float64(iterations) / duration.Seconds()

	fmt.Printf("\nPerformance Test Results:\n")
	fmt.Printf("- Total predictions: %d\n", iterations)
	fmt.Printf("- Total time: %dms\n", duration.Milliseconds())
	fmt.Printf("- Average latency: %dms per prediction\n", avgLatency)
	fmt.Printf("- Throughput: %.0f predictions/sec\n", throughput)

	// NFR1 requirement: SDK overhead <50ms (p95 <100ms)
	if avgLatency < 50 {
		fmt.Printf("✓ PASS: Average latency %dms < 50ms target\n", avgLatency)
	} else {
		t.Errorf("✗ FAIL: Average latency %dms exceeds 50ms target", avgLatency)
	}
}

// BenchmarkRecordPrediction measures prediction recording performance.
func BenchmarkRecordPrediction(b *testing.B) {
	cfg := &config.Config{
		ServiceName:       "bench-test",
		ModelID:           "mdl-bench",
		TeamName:          "test",
		CollectorEndpoint: "http://localhost:4318",
		TLSInsecure:       true,
	}

	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	client, err := NewClient(ctx, cfg)
	if err != nil {
		b.Fatalf("NewClient() error: %v", err)
	}
	defer client.Shutdown()

	latency := 0.15
	pred := &Prediction{
		Latency:      &latency,
		PredictionID: "pred-bench",
		Attributes: map[string]string{
			"model_version": "2.0",
		},
	}

	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_ = client.RecordPrediction(context.Background(), pred)
	}
}
```

Run performance tests:

```bash
# Run performance test
go test -v -run TestPerformance

# Expected output (should meet NFR1):
# Performance Test Results:
# - Total predictions: 1000
# - Total time: ~3000ms
# - Average latency: ~3ms per prediction
# - Throughput: ~333 predictions/sec
# ✓ PASS: Average latency 3ms < 50ms target

# Run benchmark
go test -bench=BenchmarkRecordPrediction -benchmem

# Expected output:
# BenchmarkRecordPrediction-8    100000    XXXXX ns/op    XXX B/op    XX allocs/op
```

## Step 9: Defer Pattern Test

Create `test_defer_patterns_test.go`:

```go
package mca

import (
	"context"
	"fmt"
	"testing"
	"time"

	"gitlab.com/bhsf/ai_ml/model-monitoring/sdk/mca-prototype/mca-sdk-go/config"
)

// TestDeferPattern verifies graceful shutdown with defer.
func TestDeferPattern(t *testing.T) {
	fmt.Println("Testing defer pattern (automatic cleanup)...")

	// Test automatic shutdown on success
	func() {
		cfg := &config.Config{
			ServiceName:       "defer-test",
			ModelID:           "mdl-defer",
			ModelVersion:      "1.0",
			TeamName:          "test",
			CollectorEndpoint: "http://localhost:4318",
			TLSInsecure:       true,
		}

		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
		defer cancel()

		client, err := NewClient(ctx, cfg)
		if err != nil {
			t.Fatalf("NewClient() error: %v", err)
		}
		defer client.Shutdown()

		latency := 0.1
		pred := &Prediction{Latency: &latency}
		if err := client.RecordPrediction(context.Background(), pred); err != nil {
			t.Errorf("RecordPrediction() error: %v", err)
		}
		fmt.Println("✓ Prediction recorded inside function")
	}()
	fmt.Println("✓ Client automatically shut down via defer")

	// Test automatic shutdown on error
	func() {
		defer func() {
			if r := recover(); r != nil {
				fmt.Printf("✓ Panic recovered: %v\n", r)
			}
		}()

		cfg := &config.Config{
			ServiceName:       "defer-test-panic",
			ModelID:           "mdl-defer-panic",
			TeamName:          "test",
			CollectorEndpoint: "http://localhost:4318",
			TLSInsecure:       true,
		}

		ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
		defer cancel()

		// Create new instance by resetting singleton
		resetInstance()
		client, err := NewClient(ctx, cfg)
		if err != nil {
			t.Fatalf("NewClient() error: %v", err)
		}
		defer func() {
			if err := client.Shutdown(); err != nil {
				t.Logf("Shutdown error (expected): %v", err)
			}
			fmt.Println("✓ Client still shut down despite panic")
		}()

		latency := 0.1
		pred := &Prediction{Latency: &latency}
		_ = client.RecordPrediction(context.Background(), pred)
		panic("Simulated panic")
	}()

	fmt.Println("\nDefer pattern test completed!")
}
```

## Step 10: Validate All Acceptance Criteria

### AC #1: Basic Prediction Recording
```go
cfg := &config.Config{
	ServiceName:       "test",
	ModelID:           "mdl-001",
	CollectorEndpoint: "http://localhost:4318",
	TLSInsecure:       true,
}

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

client, err := NewClient(ctx, cfg)
if err != nil {
	log.Fatal(err)
}
defer client.Shutdown()

latency := 0.15
pred := &Prediction{Latency: &latency}
_ = client.RecordPrediction(context.Background(), pred)
// ✓ Verify: model.predictions_total counter incremented
```

### AC #2: Custom Attributes
```go
latency := 0.15
pred := &Prediction{
	Latency:      &latency,
	PredictionID: "pred-123",
	Attributes: map[string]string{
		"customAttr":  "value",
		"anotherAttr": "123",
	},
}
_ = client.RecordPrediction(context.Background(), pred)
// ✓ Verify: Attributes included in OTLP payload
```

### AC #3: Shutdown with Flush
```go
err := client.Shutdown()
// ✓ Verify: Returns nil, all telemetry flushed
```

### AC #4: Goroutine Safety
```go
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
	wg.Add(1)
	go func(id int) {
		defer wg.Done()
		latency := 0.1 + float64(id)*0.01
		pred := &Prediction{Latency: &latency}
		_ = client.RecordPrediction(context.Background(), pred)
	}(i)
}
wg.Wait()
// ✓ Verify: No race conditions, all predictions recorded
```

## Troubleshooting

### Issue: "Cannot connect to collector"
```bash
# Check collector is running
docker-compose ps otel-collector

# Check collector endpoint
curl http://localhost:4318/

# Restart collector
docker-compose restart otel-collector
```

### Issue: "No metrics appearing"
```bash
# Check export interval (5 seconds by default)
# Wait at least 10 seconds after recording metrics

# Check collector config
cat mca-prototype/config/otel-collector-config.yaml

# Verify OTLP receiver on port 4318
docker-compose logs otel-collector | grep "4318"
```

### Issue: "go.mod issues"
```bash
# Clean module cache
go clean -modcache

# Re-download dependencies
go mod tidy
go mod download

# Verify go.mod is valid
go mod verify
```

### Issue: "Context deadline exceeded"
```bash
# Increase timeout in client initialization
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()

# Or check if OTel Collector is responsive
curl http://localhost:13133/
```

### Issue: "Race detector warnings"
```bash
# Run tests with race detection to identify issues
go test -race ./...

# Common causes:
# - Unprotected shared state
# - Missing mutex locks
# - Concurrent map access

# The SDK is goroutine-safe, but check your test code
```

## Success Criteria Checklist

- [ ] All unit tests passing (`go test ./...`)
- [ ] Race detector passes (`go test -race ./...`)
- [ ] Integration test connects to OTel Collector on port 4318
- [ ] Metrics visible in collector logs (model.predictions_total, model.latency_seconds)
- [ ] Custom attributes included in telemetry payloads
- [ ] Shutdown flushes all pending data
- [ ] Goroutine safety verified (no race conditions)
- [ ] Performance meets NFR1 (<50ms overhead per prediction)
- [ ] API parity with Python SDK (same metric names, attributes, behavior)
- [ ] Defer pattern automatically shuts down client
- [ ] Context cancellation properly propagated

## Next Steps

1. Run integration tests in Workbench
2. Validate telemetry data in collector logs
3. Compare with Python SDK output
4. Performance test to verify NFR1 compliance
5. Create PR with changes

## Additional Resources

- [Go SDK README](./README.md) - API documentation
- [Example Code](../sdk-examples/internal-model-go/) - Working examples
- [OTel Collector Config](../config/otel-collector-config.yaml) - Collector setup
