# Changelog

All notable changes to the MCA SDK will be documented in this file.

## [0.8.24] - 2026-04-21

### Fixed
- **CRITICAL: Duplicate Label Names Error in Prometheus Export**
  - Fixed "duplicate label names in constant and variable labels for metric" errors that occurred when:
    - OTel Collector's Prometheus exporter has `resource_to_telemetry_conversion: enabled: true`
    - User passes metric attributes (via `record_prediction()` kwargs) that match resource attribute keys
    - Registry's `extra_resource` JSONB field contains keys that users commonly pass (e.g., `batch_size`, `version`, etc.)
  - **Root Cause**: Prometheus exporter converts ALL resource attributes to constant labels. When the same key appears as both a resource attribute (constant label) and a metric data-point attribute (variable label), Prometheus rejects the metric with duplicate label error.
  - **Solution**: Enhanced `_filter_reserved_attributes()` to dynamically extract ALL resource attribute keys from the configured OpenTelemetry Resource and filter them from metric attributes. Previous implementation used a hardcoded list that missed registry `extra_resource` keys and other dynamic attributes.
  - **Impact**: 
    - Prevents collector-level metric drops for `emms_model_predictions_total` and `emms_model_latency_seconds`
    - Protects against arbitrary collisions from registry-supplied resource attributes
    - Ensures predictive models work correctly even when registry configuration changes
  - **Affected Methods**: `record_prediction()`, `record_error()`, `record_metric()`
  - **Backward Compatible**: Filtering is transparent to users; no API changes

### Changed
- **Dynamic Resource Attribute Filtering**: `_filter_reserved_attributes()` now queries actual resource attributes instead of using hardcoded list
  - Automatically protects against duplicates from: SDK resource attrs, registry `extra_resource`, collector-injected attrs (environment, gcp.region), and threshold resource attrs
  - Adds both dotted (`service.name`) and underscored (`service_name`) versions to catch Prometheus normalization
  - Preserves defensive filtering of high-cardinality IDs (`prediction_id`, `user_id`, `session_id`)

## [0.8.23] - 2026-04-21

### Added
- **Status Label for Prediction Metrics**: Added `status` label to `model.predictions_total` counter and `model.latency_seconds` histogram
  - `status="success"` for successful predictions (via `record_prediction()`)
  - `status="error"` for failed predictions (via `record_error()`)
  - **Benefits**:
    - Clean error rate calculations: `rate({status="error"}) / rate(total)`
    - Easy filtering in Grafana dashboards and Cloud Monitoring
    - Simpler Prometheus alerting rules
    - Better observability of prediction outcomes
  - **Backward Compatible**: Additive change only, existing queries continue to work
  - **Visibility**: Status label appears in GCP Cloud Monitoring Metrics Explorer as a filterable dimension
  - **Use Case**: Enables queries like `sum(rate(model_predictions_total{status="error"}[5m])) / sum(rate(model_predictions_total[5m]))` for error rates

### Changed
- **Metric Label Consistency**: All prediction metrics now include explicit status labels for uniform querying

## [0.8.22] - 2026-04-20

### Added
- **Status Label for Prediction Metrics**: Added `status` label to `model.predictions_total` counter and `model.latency_seconds` histogram
  - `status="success"` for successful predictions (via `record_prediction()`)
  - `status="error"` for failed predictions (via `record_error()`)
  - **Benefits**:
    - Clean error rate calculations: `rate({status='error'}) / rate(total)`
    - Easy filtering in Grafana dashboards
    - Simpler Prometheus alerting rules
    - Better observability of prediction outcomes
  - **Backward Compatible**: Additive change only, existing queries continue to work
  - **Use Case**: Enables queries like `sum(rate(model_predictions_total{status="error"}[5m])) / sum(rate(model_predictions_total[5m]))` for error rates

### Fixed
- **Prometheus Metrics Export**: Fixed duplicate label names error in Prometheus metrics export by preventing duplicate label names from causing export failures
- **Test Infrastructure**: Updated test patches to use correct import path for serialize_for_telemetry
- **Code Quality**: Applied black formatting with line-length 100 to match CI configuration
- **Linting**: Removed unused serialize_for_telemetry import to pass ruff linting
- **Test Coverage**: Updated tests to reflect auth error fallback behavior and added missing test parameters
- **Agentic AI**: Fixed prediction endpoint for agentic AI use case
- **Documentation**: Streamlined SDK instrumentation documentation
- **Configuration**: Modified code to use REGISTRY_TOKEN_AUDIENCE environment variable

### Security
- **Dependency Updates**: 
  - Upgraded cryptography from 46.0.6 to 46.0.7 to fix CVE-2026-39892
  - Security upgrade fastapi from 0.103.2 to 0.109.1 (Snyk)
  - Security upgrade axios from 1.14.0 to 1.15.0 (Snyk)
  - Aligned security dependency versions across package configs
  - Reduced vulnerabilities in requirements-security.txt

## [0.8.20] - 2026-04-09

### Added
- **GCP Token Audience Override**: Added `gcp_token_audience` configuration option to `MCAConfig` and `token_audience` parameter to `RegistryClient`. This allows overriding the GCP ID token audience when the registry URL is an IP address but the token audience must match the service DNS name (e.g., `MCA_GCP_TOKEN_AUDIENCE=https://emms-registration-api-dev.internal`). Fixes GCP ID token audience mismatch errors in deployed environments.

### Fixed
- **Environment Variable**: Updated deployed examples to use `MCA_TLS_INSECURE_REGISTRY` environment variable with consistent naming.

## [0.8.19] - 2026-04-08

### Fixed
- **Registry Validation**: Updated `MCAClient` to allow unauthenticated registry connections when `tls_insecure_registry=True` is enabled. This permits connection to internal development endpoints that do not require GCP authentication or bearer tokens.

## [0.8.18] - 2026-04-08

### Added
- **API Consistency**: Updated `RegistryClient` to explicitly accept `tls_insecure_registry` as a keyword argument, matching the naming convention in `MCAConfig`. This prevents `TypeError` when passing configuration properties directly to the registry client.

## [0.8.17] - 2026-04-07

### Added
- **tls_insecure_registry**: Added `tls_insecure_registry=True` (and the `TLS_INSECURE_REGISTRY` environment variable) to `MCAConfig` to allow bypassing strict SSL certificate verification when connecting to HTTPS registry endpoints with self-signed certificates during local development.

## [0.8.13] - 2026-04-06

### Fixed
- **RegistryClient**: Properly restored the `version` parameter across all internal methods and `MCAClient` hydration logic. This ensures that the version parameter provided during SDK initialization (including the default `0.3.0`) is correctly handled and logged, while still maintaining compatibility with legacy registry endpoints that do not yet support explicit version parameters.

## [0.8.12] - 2026-04-04

### Fixed
- **RegistryClient**: Restored the `version: Optional[str] = None` parameter to `fetch_model_config()` and `_fetch_model_config_internal()`. This parameter was mistakenly removed in a previous refactor, causing compatibility issues with `MCAConfig.model_version` defaults and internal `MCAClient` hydration logic, resulting in 404 errors for deployed models trying to resolve the `0.3.0` version string. (Note: This release was missing some file updates due to a commit error, fixed in 0.8.13).

## [0.8.11] - 2026-04-03

### Fixed
- **RegistryClient**: Reverted the unsupported `version` parameter usage for legacy registry API endpoint (`/api/v1/configs/{model_id}`). The version parameter is now ignored and a warning is logged, preventing 404/rejection errors when using legacy registry endpoints.

## [0.8.10] - 2026-04-02

### Fixed
- **Registry API Endpoint**: Reverted to `/api/v1/configs/{model_id}` from `/models/{model_id}`
  - **Root Cause**: The new `/models/{model_id}` endpoint (introduced in v0.8.6) returns 404 for all registered models
  - **Solution**: Reverted to the old `/api/v1/configs/{model_id}` endpoint which still works and has model data
  - **Impact**: Fixes RegistryConfigNotFoundError for all models including patient-readmission-classifier-v1, medical-research-assistant, and clinical-notes-summarizer-v2
  - **Future**: Will re-migrate to new endpoint once Registry team completes data migration to `/models/` endpoint

## [0.8.9] - 2026-04-01

### Fixed
- **Reverted Registry API version=latest behavior**: Reverted the change introduced in v0.8.6 that explicitly sent `version=latest` when fetching model configurations without a specified version.
  - **Root Cause**: The Registry API backend does not support the literal string "latest" and treating it as a semantic version caused `404 Not Found` errors.
  - **Solution**: The SDK now omits the `version` parameter entirely when requesting the latest version, matching pre-0.8.6 behavior.
  - **API Contract Update**: The API contract documentation has been updated to explicitly state that omitting the parameter is the standard way to request the latest version, and "latest" is not a supported literal string.

## [0.8.8] - 2026-03-27

### Fixed
- **CRITICAL: Counter Metric Instrumentation Bug** - Fixed `record_counter()` and `record_gauge()` methods creating incorrect instrument types
  - **Root Cause**: `record_counter()` was calling `record_metric()` without specifying `instrument_type="counter"`, causing counters to be created as Histograms
  - **Impact**: Counter metrics like `emms_genai_requests_total` were recorded as Histograms, making them invisible to aggregation queries that expect counter semantics (e.g., `counter-increase` CTEs in BigQuery)
  - **Affected Metrics**: All metrics recorded via `record_counter()` including:
    - `emms_genai_requests_total` (GenAI request counter)
    - `emms_model_predictions_total` (prediction counter)
    - Any custom counters recorded by applications
  - **Symptoms**: Aggregation ETL queries finding latency metrics but no request counters; traces present but counter data missing
  - **Fix**: Added explicit `instrument_type="counter"` parameter to `record_counter()` and `instrument_type="up_down_counter"` to `record_gauge()`
  - **Verification**: Counter instruments now correctly created as `_Counter` type with `add()` method instead of `_Histogram` with `record()` method
  - **Models Affected**: clinical-notes-summarizer-v2 and any other GenAI models using LiteLLM callback
  - **Action Required**: Redeploy models using SDK v0.8.8+ to populate counter metrics in aggregation tables

## [0.8.7] - 2026-03-27

### Fixed
- **CRITICAL: Counter Metric Instrumentation Bug** - Fixed `record_counter()` and `record_gauge()` methods creating incorrect instrument types
  - **Root Cause**: `record_counter()` was calling `record_metric()` without specifying `instrument_type="counter"`, causing counters to be created as Histograms
  - **Impact**: Counter metrics like `emms_genai_requests_total` were recorded as Histograms, making them invisible to aggregation queries that expect counter semantics (e.g., `counter-increase` CTEs in BigQuery)
  - **Affected Metrics**: All metrics recorded via `record_counter()` including:
    - `emms_genai_requests_total` (GenAI request counter)
    - `emms_model_predictions_total` (prediction counter)
    - Any custom counters recorded by applications
  - **Symptoms**: Aggregation ETL queries finding latency metrics but no request counters; traces present but counter data missing
  - **Fix**: Added explicit `instrument_type="counter"` parameter to `record_counter()` and `instrument_type="up_down_counter"` to `record_gauge()`
  - **Verification**: Counter instruments now correctly created as `_Counter` type with `add()` method instead of `_Histogram` with `record()` method
  - **Models Affected**: clinical-notes-summarizer-v2 and any other GenAI models using LiteLLM callback
  - **Action Required**: Redeploy models using SDK v0.8.7+ to populate counter metrics in aggregation tables

## [0.8.6] - 2026-03-24

### Changed
- Updated registry API endpoint from `/api/v1/configs/{model_id}` to `/models/{model_id}`
- Added support for version parameter in model configuration fetches
- Removed deprecation warning for version parameter in registry client
- Introduced `DEFAULT_MODEL_VERSION` constant to replace magic string "latest"
- Enhanced registry client telemetry with cache hit/miss logging and resolved version tracking

### Added - Backward Compatibility
- **`omit_default_version` parameter** for RegistryClient to support legacy registry APIs
  - Set to `True` to restore pre-0.8.6 behavior (omit version parameter)
  - Defaults to `False` (modern behavior: send `version=latest`)
  - Can be controlled via `MCA_REGISTRY_OMIT_DEFAULT_VERSION` environment variable
  - Logs warning when legacy mode is enabled
  - Allows migration path for systems with non-compliant registry APIs

### Fixed
- **Registry Client Version Resolution** (Story 8-8): Fixed model configuration fetching to explicitly request `version=latest` when no version is specified, resolving "Model config not found" errors for valid registered models
  - Affected models: patient-readmission-classifier-v1, medical-research-assistant, clinical-notes-summarizer-v2
  - Root cause: Mismatch between cache key semantics (using "latest" string) and API request (omitting version parameter)
  - Impact: Registry refresh now works correctly for all registered models
  - Solution: SDK now explicitly sends `version=latest` parameter when no version specified
  - Error messages improved with actionable guidance when model/version not found
- Registry client now properly sends version parameter to API when provided
- Fixed duplicate `capture_output_data` parameter in agentic example causing syntax error

### Added
- Comprehensive integration tests for registry version parameter handling (7 new tests)
- Thread-safety validation tests for concurrent registry operations
- Detailed error messages for 404 responses with troubleshooting guidance
- DEBUG-level logging for cache operations (hits/misses)
- INFO-level logging for successful registry fetches with resolved version information

### Documentation
- Added thread-safety guarantees to RegistryClient class and cache method docstrings
- Updated fetch_model_config docstring to clarify version parameter behavior
- Type hints validated with mypy (zero errors)
- **Async Environment Usage**: Added warnings to RegistryClient docstring and README about synchronous locking and event loop blocking
  - RegistryClient uses `threading.RLock` and is NOT safe to call from async functions
  - Recommended pattern: Initialize during startup, enable background refresh, use @predict() decorator
  - DO NOT call fetch_model_config() from FastAPI endpoints or async request handlers

## [0.8.5] - 2026-03-23

### Added
- **Mid-Lifecycle Telemetry Flush** (flush method)
  - Added `flush()` method to MCAClient for forcing telemetry export without shutdown
  - Allows batch processing scenarios to flush buffers between batches
  - Safe to call mid-lifecycle, does not affect client state
  - Returns boolean indicating success/failure across all providers
  - Configurable timeout (default: 30 seconds)

### Changed
- Removed `capture_input_data` parameter from agentic agent example for cleaner configuration

## [0.8.4] - 2026-03-21

### Added
- **Configurable Metric Filtering** (Story 8.7)
  - SDK now filters unnecessary auto-instrumentation and system metrics before export
  - Dropped metric patterns: `process.*`, `system.*`, `asyncio.*`, `cpython.*`, `http.client.*`, `otel.sdk.*`, `test.*`, `rpc.*`
  - Retained business metrics: `model.*`, `genai.*`, `agent.*`, `vendor.*`, `agentic.*`, `llm_eval/*`
  - Filtering configurable via `filter_system_metrics` parameter (default: True)
  - Reduces Cloud Monitoring costs by eliminating unnecessary custom metric descriptors
  - Implemented using OpenTelemetry Views and DropAggregation

### Fixed
- **Model Version Fallback Bug**
  - Fixed model_version not falling back to SDK version when not provided via config or registry
  - Ensures model.version resource attribute is always populated for telemetry filtering
- **Missing Trace Coverage for ML Errors**
  - Added trace span creation in record_error for predictive ML models
  - Ensures errors are visible in distributed traces alongside metrics and logs

## [0.8.3] - 2026-03-20

### Fixed
- **Metric Cardinality Explosion & Trace Correlation Issues**
  - Defensively pop `prediction_id`, `user_id`, and `session_id` from generic `attributes` in `record_prediction` and `record_error` to prevent metric cardinality explosions.
  - Introduce `span_attributes` parameter in `record_prediction` and `record_error` to allow high-cardinality data safely attached to OpenTelemetry spans instead of metrics.
  - Fix broken trace-to-error correlation in both manual error recording and batch tracking decorators by ensuring `prediction_id` flows through to spans and logging.

- **Trace Span Creation for Predictive & Generative Models**
  - Added `mca_client.trace()` context manager around prediction loops.
  - Generate `prediction_id` for trace correlation and add it as a span attribute.
  - Enables trace visibility in GCP Traces Explorer.

- **Data Generation Tests**
  - Updated data generation tests to match current model configuration.

## [0.8.2] - 2026-03-19

### Changed

- **Registry URL Optional in All Environments (Story 2.7 + Adversarial Review Fixes)**
  - Registry connection is now optional in all environments (dev, prod, staging)
  - Previously: dev/prod required registry_url and model_id, raised ConfigurationError if missing
  - Now: SDK operates normally without registry in any environment, logs informational messages
  - **CRITICAL FIXES** (from adversarial review):
    - `_validate_registry_requirements()` now returns bool to actually control hydration (was theatrical before)
    - Removed dangerous fallback to service_name in `_hydrate_from_registry()`
    - Staging now skips hydration when auth missing (prevents slow 401/timeout on startup)
    - Production defaults to `strict_validation=True` even without registry (prevents security downgrade)
    - CLI template now comments out `model_id` by default (prevents log spam)
  - When both registry_url and model_id are provided, authentication is still required (dev/prod)
  - Staging skips hydration when auth incomplete (prevents startup performance degradation)
  - **Migration**: Prod users without registry should explicitly set `strict_validation=True` if needed
  - **Impact**: Removes artificial barrier for SDK users who don't need registry integration
  - Test coverage: 8 tests (7 updated + 1 new AC2 telemetry test) with network isolation assertions

### Added

- **SDK Version in Resource Attributes (Story 1-21)**
  - SDK version automatically added to all resource attributes
  - Two attributes injected: `mca.sdk.version` (custom) and `telemetry.sdk.version` (OpenTelemetry standard)
  - **Immutable**: Cannot be overridden by `extra_resource`, kwargs, or registry config
  - **Universal**: Present in all telemetry signals (metrics, logs, traces)
  - **Tamper-Proof**: User attempts to override are blocked, real SDK version is always used
  - **Use Cases**: Track deployed SDK versions, correlate telemetry with version changes, backward compatibility analysis
  - **Architecture**: Dedicated `mca_sdk/version.py` module eliminates circular dependencies
  - **Testing**: Comprehensive unit and integration tests validate presence in actual exported payloads
  - **Documentation**: Updated `docs/data-models.md` and `docs/api-contracts.md`

## [0.8.1] - 2026-03-19

### Fixed

- **LLM Completion Tracking Performance**
  - Fixed major performance bug where OpenTelemetry instruments were dynamically created on every LLM call
  - Implemented pre-created instrument caching in `genai.py` with thread-safe global cache
  - Added `_get_instruments()` helper to retrieve or initialize instruments once per client
  - Reduced overhead for high-frequency LLM operations (GenAI workloads)
  - All metric instruments now created once and reused: requests_total, requests_failed_total, tokens_total, latency_seconds, cost_dollars, embedding_*
  - Thread-safe implementation using locks to prevent race conditions
  - Impact: Significant performance improvement for applications with high LLM request rates

### Changed

- **Registry URL Optional in All Environments (Story 2.7 + Adversarial Review Fixes)**
  - Registry connection is now optional in all environments (dev, prod, staging)
  - Previously: dev/prod required registry_url and model_id, raised ConfigurationError if missing
  - Now: SDK operates normally without registry in any environment, logs informational messages
  - **CRITICAL FIXES** (from adversarial review):
    - `_validate_registry_requirements()` now returns bool to actually control hydration (was theatrical before)
    - Removed dangerous fallback to service_name in `_hydrate_from_registry()`
    - Staging now skips hydration when auth missing (prevents slow 401/timeout on startup)
    - Production defaults to `strict_validation=True` even without registry (prevents security downgrade)
    - CLI template now comments out `model_id` by default (prevents log spam)
  - When both registry_url and model_id are provided, authentication is still required (dev/prod)
  - Staging skips hydration when auth incomplete (prevents startup performance degradation)
  - **Migration**: Prod users without registry should explicitly set `strict_validation=True` if needed
  - **Impact**: Removes artificial barrier for SDK users who don't need registry integration
  - Test coverage: 8 tests (7 updated + 1 new AC2 telemetry test) with network isolation assertions

## [0.8.0] - 2026-03-18

### Added

- **SDK Version in Resource Attributes (Story 1-21)**
  - SDK version automatically added to all resource attributes
  - Two attributes injected: `mca.sdk.version` (custom) and `telemetry.sdk.version` (OpenTelemetry standard)
  - **Immutable**: Cannot be overridden by `extra_resource`, kwargs, or registry config
  - **Universal**: Present in all telemetry signals (metrics, logs, traces)
  - **Tamper-Proof**: User attempts to override are blocked, real SDK version is always used
  - **Use Cases**: Track deployed SDK versions, correlate telemetry with version changes, backward compatibility analysis
  - **Architecture**: Dedicated `mca_sdk/version.py` module eliminates circular dependencies
  - **Testing**: Comprehensive unit and integration tests validate presence in actual exported payloads
  - **Documentation**: Updated `docs/data-models.md` and `docs/api-contracts.md`
### Changed (BREAKING)

- **PHI/PII Masking Removed**
  - PHI_PATTERNS regex list removed from client.py
  - _sanitize_phi() function removed from gcp_logging.py
  - All PHI masking tests removed (test_phi_masking.py, test_gcp_logging_phi.py)
  - "Dual-purpose masking" security model removed
  - **KEPT**: CREDENTIAL_PATTERNS and credential masking (unchanged)
  - **KEPT**: phi_utils.py field name extraction (helps apps identify PHI)
  - **Impact**: Applications now have 100% responsibility for PHI sanitization
  - **SDK only masks technical credentials** (AWS/GCP keys, tokens, JWTs)
  - **No breaking API changes** - removal is internal implementation only
  - **Migration**: Review your application code to ensure PHI is sanitized BEFORE calling SDK methods
  - Do not rely on SDK to catch PHI patterns
  - Use `phi_utils.extract_phi_field_names()` to identify PHI fields from registry
  - **Rationale**: SDK cannot know domain-specific PHI context. Regex-based masking is insufficient and creates false positives. Clear responsibility boundaries prevent security assumptions.

### Changed (BREAKING)

- **Default Data Capture Enabled**
  - `capture_input_data` default changed from `False` to `True`
  - `capture_output_data` default changed from `False` to `True`
  - `capture_input` default changed from `False` to `True` (Decorators)
  - `capture_output` default changed from `False` to `True` (Decorators)
  - `record_error_messages` default changed from `False` to `True` (LiteLLM Callback)
  - `persist_queue` default changed from `False` to `True` (Buffering)
  - **Impact**: Prompts, completions, errors, and input/output payload traces are NOW captured and buffered by default.
  - **Migration**: Explicitly set these parameters to `False` if you do not want input/output capture.
  - **Rationale**: PII and PHI handling relies on downstream access controls and sanitization policies.
  - Affects:
    - `MCAConfig` defaults
    - `create_genai_client()` helper function (now defaults to True)
    - `MCALiteLLMCallback` fallback defaults and constructor
    - `@predict()` and `@instrument_model` decorator behavior

### Added

- **Thresholds as Resource Attributes (Production-Ready)**
  - Registry API thresholds automatically added as OpenTelemetry resource attributes
  - Thresholds prefixed with `threshold.` (e.g., `threshold.latency_ms`, `threshold.accuracy_min`)
  - **Preserves native numeric types**: int stays int, float stays float (no forced conversion)
  - Enables proper numeric filtering, sorting, and aggregations in downstream systems
  - **IMPORTANT:** Resources are immutable - thresholds evaluated ONCE at SDK initialization
  - **Comprehensive Safeguards:**
    - Boolean rejection: Explicitly rejects `bool` types (subclass of `int` in Python)
    - Infinity rejection: `math.isinf()` check prevents OpenTelemetry exporter crashes
    - NaN rejection: `math.isnan()` check prevents invalid telemetry data
    - None rejection: Prevents string "None" pollution
    - Maximum 50 thresholds: First 50 kept (dict iteration order), rest dropped with clear warning
    - Key sanitization with collision detection: Invalid chars→`_`, collisions logged
    - Key length accounting: Max 53-char names (63 total with "threshold." prefix)
    - User precedence: User-provided `extra_resource` keys never silently overwritten
    - Module-level imports: `math` and `re` imported at module level (not function level)
    - SDK logger used throughout (not global `logging`)

## [0.7.7] - 2026-03-12

### Fixed

- **Registry Token Validation with GCP Auth**
  - Allow GCP auth without requiring registry token in production environment
  - Fixed validation logic to check for `use_gcp_auth` before requiring `MCA_REGISTRY_TOKEN`
  - When `use_gcp_auth=True`, token is no longer required

## [0.7.2] - 2026-03-10

### Fixed

- **mca-instrument Endpoint Mapping**
  - Map `MCA_COLLECTOR_ENDPOINT` to `OTEL_EXPORTER_OTLP_ENDPOINT` for opentelemetry-instrument
  - Map `MCA_SERVICE_NAME` to `OTEL_SERVICE_NAME` for proper service identification
  - Fixes issue where telemetry was sent to localhost instead of configured collector endpoint

## [0.7.1] - 2026-03-09

### Fixed

- **CLI Entry Points Not Installed**
  - Added `[project.scripts]` section to `pyproject.toml` for CLI tools
  - Fixes `mca-init`, `mca-instrument`, and `mca-run` commands not being available after pip install
  - CLI tools now properly installed to user's PATH

## [0.7.0] - 2026-03-09

### Changed (BREAKING)

- **Default Validation Mode Changed to Relaxed**
  - `strict_validation` default changed from `True` to `False`
  - **Impact**: Users who relied on default strict validation will now get relaxed validation
  - **Migration**: Explicitly set `strict_validation=True` if you need strict validation
  - **Rationale**: Lower barrier to entry for development and prototyping while maintaining production safety

### Added

- **Flexible Validation System**
  - Relaxed validation mode (new default): Only requires `service.name`
  - Strict validation mode: Requires all model-type-specific fields (auto-enabled for registry integration)
  - INFO-level logging for missing optional fields in relaxed mode
  - Auto-detection: Strict validation automatically enabled when `registry_url` is configured

### Fixed

- **Package Configuration Sync**
  - Synced `pyproject.toml` with `setup.py` to include all optional dependencies
  - Added missing extras: `prometheus`, `gcp-auth`, `gcp-secrets`, `gcp-trace`, `gcp-logging`, `autolog`, `instrument`
  - Fixed `[instrument]` extra not being available in published package
  - Users can now install: `pip install "mca-sdk[instrument]"` for zero-code instrumentation

### Migration Guide

**If you need strict validation** (all model-type fields required):
```python
# Option 1: Explicit configuration
client = MCAClient(
    service_name="my-model",
    strict_validation=True,  # Explicitly enable
    # ... other required fields
)

# Option 2: Use registry integration (auto-enables strict validation)
client = MCAClient(
    service_name="my-model",
    registry_url="https://registry.example.com",  # Auto-enables strict validation
    # ... other required fields
)
```

**If you want minimal validation** (service_name only):
```python
# Default behavior in v0.7.0 - no changes needed
client = MCAClient(service_name="my-model")  # Works!
```

## [0.6.7] - 2026-03-05

### Fixed
- Add missing resource attributes for agentic model type

## [0.6.5] - 2026-03-05

### Fixed
- Lower OpenTelemetry requirement to >=1.35.0 for google-adk compatibility

## [0.6.4] - 2026-03-05

### Fixed
- Export print_gcp_auth_status and print_collector_status from shared module
- Add fallback for OpenTelemetry logs import compatibility across versions

## [0.6.1] - 2026-02-26

### Fixed
- **CRITICAL: Context Token Handling in Agentic AI Goal Tracking**
  - Fixed TypeError when detaching context in `record_goal_completed()`
  - **Problem**: `record_goal_started()` was incorrectly using `trace.use_span()` which returns a context manager, not a Token
  - **Error**: `TypeError: expected an instance of Token, got _AgnosticContextManager`
  - **Solution**: Changed to use `context.attach(trace.set_span_in_context(span))` for proper Token handling
  - **Impact**: All users of `record_goal_started()` + `record_goal_completed()` were affected
  - **Severity**: High - causes runtime errors for agentic AI features

### Documentation
- Rebuilt end-to-end demo notebook with comprehensive v0.6.0 feature coverage
  - Added demonstrations of all 11 telemetry features
  - Added model input/output capture examples (Story 1-18)
  - Fixed `record_goal_started()` parameter usage in examples
  - Created notebooks/README.md with usage guide and troubleshooting

## [0.6.0] - 2026-02-26

### Added
- **GCP Authentication Support for Registry Client**
  - Added `use_gcp_auth` parameter to `RegistryClient` initialization
  - Enables automatic GCP credential-based authentication for Registry API
  - Properly passed from `MCAConfig` to `RegistryClient` in client initialization

- **Comprehensive End-to-End Tests** (Story 6-16)
  - Full OTLP JSON validation for metrics, logs, and traces
  - Multi-model telemetry validation across all model types
  - Error scenario testing (network failures, invalid data, collector downtime)
  - Context propagation validation for distributed tracing

- **Integration Tests with OTel Collector** (Story 6-7)
  - Real collector telemetry flow validation
  - High availability features testing (HPA, persistent volumes, DLQ)
  - Collector self-monitoring validation
  - Batch processing and attribute enrichment verification

- **Load Testing Framework** (Story 6-8)
  - Performance testing at 1000 req/s
  - Telemetry completeness validation
  - Collector monitoring utilities
  - Load test report generation

- **Integration Guides** (Story 5-5)
  - Complete guides for Predictive ML, GenAI, Agentic AI, and Vendor Bridge patterns
  - pytest-based testing examples for each model type
  - Context propagation explanations with code examples

- **Input/Output Capture** (Story 1-18)
  - Enhanced `@predict()` decorator with input/output span attachment
  - Configurable via `capture_input` and `capture_output` parameters
  - Sanitization warnings for PHI protection

### Changed
- **OTel Collector Configuration Updates**
  - Enhanced self-monitoring metrics collection
  - Improved batch processor settings for high throughput
  - Updated attributes processor for GCP metadata enrichment

- **Integration Decorator Improvements**
  - Fixed `@predict()` and `@batch_predict()` decorators
  - Better error handling and span lifecycle management
  - Improved attribute validation and length limits

### Fixed
- Registry client GCP authentication parameter propagation from config
- Path validation in `TelemetryQueue` for disk persistence
- `ModelConfig` backward compatibility for older API responses
- Pipeline test failures in CI/CD
- Instrumentation decorator attribute handling

### Documentation
- Updated downstream integration docs for dot notation metric naming
- Added collector metrics guide
- Enhanced Kubernetes deployment README with HA features
- Added Pub/Sub DLQ documentation and replay processor

## [0.5.1] - 2026-02-25

### Fixed
- **CRITICAL: GenAI Metric Double-Prefix Bug** (Production Issue)
  - Fixed incorrect metric names in Google Cloud Monitoring / Managed Prometheus
  - **Problem**: Metrics appeared as `emms_emms_genai_requests_total` instead of `emms_genai_requests_total`
  - **Root Cause**: GenAI integration was prepending `emms_` prefix, but OTel Collector's Prometheus exporter also adds namespace prefix
  - **Solution**: Switched to dot notation (`genai.requests_total`) matching predictive ML pattern
  - **Affected Components**:
    - `track_llm_completion()` in `genai.py`
    - `track_llm_embedding()` in `genai.py`
    - `batch_predict` decorator in `decorators.py`
    - Metric validation schema updated to expect dot notation
  - **Impact**: All GenAI metrics from deployed agents (AI Ops Agent dogfooding) now report correctly
  - **Migration**: If you query Prometheus/GMP directly, update queries from `emms_emms_genai_*` to `emms_genai_*`

### Changed
- Metric naming convention now uses dot notation for all metric types (e.g., `genai.requests_total`, `model.predictions_total`)
- Validation schema updated to reflect dot-notation pattern

## [0.5.0] - 2026-02-25

### Added
- **Registry API GCP Authentication Documentation**
  - New comprehensive guide: [`docs/registry-gcp-auth-guide.md`](docs/registry-gcp-auth-guide.md)
  - Covers all authentication methods: ADC, service account JSON, Workload Identity, gcloud CLI
  - Complete examples for Vertex AI Workbench, Cloud Run, GKE, and local development
  - Token management and automatic refresh documentation
  - Troubleshooting guide with common issues and solutions
  - Security best practices for production deployments
  - Verified with integration tests against real Registry API

- **GCP Cloud Trace Direct Export** (Story 6-3)
  - New `GCPCloudTraceExporter` for exporting traces directly to Google Cloud Trace
  - Configuration fields: `gcp_trace_enabled` (bool), `gcp_project_id` (str)
  - Environment variables: `MCA_GCP_TRACE_ENABLED`, `MCA_GCP_PROJECT_ID`
  - Full span fidelity: status codes, span kinds, links, events, resource attributes
  - Production features: retry logic, timeout protection, batch splitting, validation
  - Internal telemetry metrics for observability (spans exported, failures, retries)
  - Install with: `pip install mca-sdk[gcp-trace]` or `pip install mca-sdk[gcp]`
  - **Note**: Requires GCP credentials with scope `https://www.googleapis.com/auth/trace.append`

- **Registry API Response Structure Updates** (Story 2.2 - API Integration)
  - Updated [`ModelConfig`](mca-prototype/mca_sdk/registry/models.py) dataclass to match actual API response structure
  - Added `registry_id` field (optional, can be null for vendor models)
  - Added `deployment_id` field (optional, can be null for vendor models)
  - Added `association_id_column` field (optional, for time-series models)
  - Added `created_at` and `updated_at` timestamp fields (optional, for debugging)
  - Updated [`RegistryClient`](mca-prototype/mca_sdk/registry/client.py) to correctly extract fields from API response:
    - `model_version` now extracted from top-level (not nested in config)
    - `thresholds` extracted from top-level (not nested in config)
    - `extra_resource` extracted from nested `config.extra_resource` (not top-level)
    - Handles `config: null` case gracefully
    - Handles `phi_fields: {}` empty dict case
  - Added [`extract_phi_field_names()`](mca-prototype/mca_sdk/utils/phi_utils.py) utility function
    - Extracts field names from nested PHI structure: `{"patient_id": {"criticality": "yes"}}`
    - Handles empty dict case: `{}`
    - Used for PHI masking operations
  - **Note**: All `extra_resource` values are strings (JSONB storage), passed as-is to OpenTelemetry

### Changed
- **BREAKING CHANGE: predict() Decorator Defaults** (Story 1.13.6 - Adversarial Review Finding 1)
  - `capture_input` default changed from `True` to **`False`**
  - `capture_output` default changed from `True` to **`False`**
  - **Rationale**: Safer HIPAA compliance defaults - prevents accidental PHI capture
  - **Migration Guide**: If you rely on automatic input/output capture, explicitly set parameters:
    ```python
    # OLD (implicit True - will now be False)
    @client.predict()
    def predict(data):
        return model.predict(data)

    # NEW (explicit opt-in required)
    @client.predict(capture_input=True, capture_output=True)
    def predict(data):
        # IMPORTANT: Ensure data is sanitized before calling
        return model.predict(data)
    ```
  - **Impact**: Code using `@client.predict()` without parameters will no longer capture input/output data
  - **Security**: This change reduces risk of accidental PHI exposure in telemetry

### Added
- **Enhanced PHI Masking Patterns** (Story 1.13.6 - Adversarial Review Finding 2)
  - Added ZIP code pattern (5-digit and ZIP+4 formats)
  - Added common name patterns (First Last, Last First formats)
  - Added street address pattern (basic detection)
  - **Note**: Regex-based PHI masking has inherent limitations. Applications MUST sanitize data before instrumentation.

### Security
- Improved default security posture for data capture in predict() decorator
- Expanded PHI detection patterns for better protection in error messages and logs

### Changed (from Unreleased)
- **BREAKING CHANGE: HTTPS Enforcement** (Story 3.7)
  - Both `registry_url` and `collector_endpoint` now **require HTTPS** for non-localhost endpoints
  - HTTP connections to production endpoints are blocked with `ConfigurationError`
  - **Migration Guide**: Update all production endpoint URLs from `http://` to `https://`
  - Localhost exception: HTTP remains allowed for `localhost`, full `127.0.0.0/8` range, and `::1` for development
  - Enhanced security validation prevents credential exposure over unencrypted connections
  - **Previous Behavior**: HTTP to production endpoints raised a `SecurityWarning` but was allowed
  - **New Behavior**: HTTP to production endpoints raises `ConfigurationError` and blocks initialization

### Security
- Enhanced loopback detection with full IPv4 127.0.0.0/8 range support and IPv6 ::1
- Protection against hostname substring attacks (e.g., `localhost.attacker.com`)
- Case-insensitive localhost matching per RFC 4343

## [0.4.0] - 2026-01-26

### Added
- **Background Registry Refresh Thread** (Story 2.3)
  - Automatic refresh of registry config at configurable intervals
  - Daemon thread with graceful shutdown
  - Thread-safe config updates with locking
  - Comprehensive unit tests for thread safety and lifecycle

- **Attributes Processor Integration** (Story 4.3)
  - OpenTelemetry Collector attributes processor for metadata enrichment
  - Automatic addition of `gcp.region` and `environment` attributes
  - Non-overwriting insertion semantics to preserve existing attributes
  - Validated with functional testing across all telemetry types (metrics, logs, traces)

- **Enhanced Demo Scripts**
  - OTLP endpoint configuration for better local development
  - Improved logging and error handling

### Changed
- **OpenTelemetry Dependencies** - Updated from 1.20.0 to 1.39.0
  - `opentelemetry-api>=1.39.0`
  - `opentelemetry-sdk>=1.39.0`
  - `opentelemetry-exporter-otlp-proto-http>=1.39.0`
  - **Breaking Changes**: OTel 1.39.0 includes significant changes including `LogData` removal (replaced with `ReadableLogRecord`/`ReadWriteLogRecord`), Events API deprecation, and Python 3.8 support dropped. Review [OTel Python CHANGELOG](https://github.com/open-telemetry/opentelemetry-python/blob/main/CHANGELOG.md) for migration guidance.
- **Dependency Reorganization** - Moved `requests` from core dependencies to optional `[vendor]` extras. Install with `pip install mca-sdk[vendor]` if using vendor model integrations.

### Fixed
- CI pipeline refinements for better reliability
- Removed orphaned pyproject.toml scan after merge
- Updated pip-audit command to skip editable packages

### Security
- Added `protobuf>=5.0,<6.0` constraint to mitigate CVE-2026-0994 (DoS via nested Any messages) until patched version is available

## [0.3.0] - 2026-01-22

### Added
- **Custom Exception Hierarchy** (Story 1.14)
  - `MCASDKError` - Base exception class for all SDK errors
  - `ValidationError` - Raised when validation fails (metric names, attributes)
  - `BufferingError` - Raised when buffering operations fail (queue full, disk failure)
  - `ConfigurationError` - Raised when configuration is invalid or incomplete
  - `RegistryError` - Base exception for registry operations
  - `RegistryConnectionError` - Raised when unable to connect to registry
  - `RegistryConfigNotFoundError` - Raised when model config not found in registry
  - `RegistryAuthError` - Raised when registry authentication fails
  - All exceptions exported from main `mca_sdk` package for easy catching
  - Helpful error messages with context for debugging

### Changed
- **Enhanced Configuration Validation** (Story 1.15)
  - `MCAConfig.__post_init__` now raises `ConfigurationError` (was generic `Exception`)
  - `MCAConfig.from_env` raises `ConfigurationError` for invalid integer values
  - `MCAConfig.from_file` raises `ConfigurationError` for file/parsing errors
  - `MCAConfig.load` enforces security: `registry_token` via environment only
  - Better error messages with suggested fixes for missing `service_name`
  - URL validation for `collector_endpoint` and `registry_url`
  - Security warnings for HTTP endpoints (non-localhost) using `SecurityWarning` category

## [0.2.0] - 2026-01-12

### Added
- **Model Registry Integration**: Centralized configuration management system
  - `RegistryClient` for fetching model config from registry API
  - Support for `GET /models/{model_id}` and `GET /deployments/{deployment_id}` endpoints
  - Automatic retry with exponential backoff using existing `RetryPolicy`
  - In-memory cache with configurable TTL (default 10 minutes)
  - Bearer token authentication with HTTPS enforcement

- **Dynamic Configuration**
  - `MCAConfig` extended with 5 new fields: `registry_url`, `registry_token`, `refresh_interval_secs`, `prefer_registry`, `deployment_id`
  - Environment variable support: `MCA_REGISTRY_URL`, `MCA_REGISTRY_TOKEN`, etc.
  - Config precedence: kwargs > registry > env > YAML > defaults

- **Background Refresh**
  - Automatic refresh of thresholds and PHI fields every 10 minutes (configurable)
  - Identity fields (service_name, model_id) are immutable after startup
  - Graceful handling of registry failures with fallback to last-known config

- **PHI Fields Management**
  - Union of local and registry PHI fields for comprehensive masking
  - Dynamic updates via background refresh

- **Registry Telemetry**
  - Self-monitoring metrics: `mca.registry.refresh_success_total`, `mca.registry.refresh_latency_seconds`, `mca.registry.errors_total`

- **Security**
  - HTTPS required for non-localhost registry endpoints
  - Token never logged (even at DEBUG level)
  - Query parameters excluded from logs

- **Testing**
  - 22 unit tests for RegistryClient (fetch, cache, retry, auth)
  - 11 unit tests for hydration flow and config precedence
  - 8 integration tests with FastAPI mock registry

### Changed
- `MCAClient.__init__` now performs registry hydration before provider setup
- `MCAClient.shutdown` now stops background refresh thread and closes registry client
- `setup_all_providers` accepts `extra_resource` parameter for registry attributes
- Provider resource attributes now include registry-provided `extra_resource` fields

### API Additions
- `MCAClient.thresholds` property - Access current thresholds from registry
- `RegistryClient` class exported from main package
- `ModelConfig` and `DeploymentConfig` dataclasses exported from main package

## [0.1.0] - 2026-01-21

### Added
- Core MCA SDK with `MCAClient` for HIPAA-compliant OpenTelemetry instrumentation
- Multi-source configuration system with precedence: kwargs > registry > env > YAML > defaults
- Registry integration with automatic caching and retry mechanisms
- Validation and buffering systems for telemetry data
- Schema validation for metric naming and resource attributes
- Four working SDK examples:
  - Internal model instrumentation
  - GenAI model integration
  - Agentic AI workflows
  - Vendor model integration
- Docker Compose orchestration for local development
- Integration helpers for vendor models, GenAI, and Python decorators
- Comprehensive test suite with unit and integration tests
- Support for Python 3.10, 3.11, and 3.12

### Changed
- Buffering disabled by default for backward compatibility
- Improved demo scripts with better logging and metric handling
- Refactored metric names for consistency and clarity

### Removed
- PHI detection and masking features (architectural decision to simplify initial release)

### Fixed
- Buffering backward compatibility in `from_env()` configuration
- Critical `SecurityWarning` class placement in `MCAConfig`
- Multiple security and reliability issues identified in code review
- Test coverage expanded to meet 85% requirement

### Security
- Security hardening for multi-source configuration system
- Registry token blocked from YAML configuration files to prevent credential exposure
- Localhost validation fixed to prevent substring attacks
- Comprehensive security review addressing CRITICAL and HIGH priority issues

[unreleased]: https://gitlab.com/bhsf/ai_ml/mca-sdk/compare/v0.8.0...HEAD
[0.8.0]: https://gitlab.com/bhsf/ai_ml/mca-sdk/releases/tag/v0.8.0
[0.1.0]: https://gitlab.com/bhsf/ai_ml/mca-sdk/releases/tag/v0.1.0