Metadata-Version: 2.4
Name: uq-metrics
Version: 0.1.0
Summary: Uncertainty quantification metrics for model evaluation
Project-URL: Homepage, https://github.com/debargha/uq-metrics
Author: Debargha
License-Expression: MIT
License-File: LICENSE
Keywords: calibration,machine-learning,metrics,quantification,uncertainty
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.8
Requires-Dist: numpy>=1.20
Provides-Extra: plot
Requires-Dist: matplotlib>=3.0; extra == 'plot'
Description-Content-Type: text/markdown

# uq-metrics

Uncertainty quantification metrics for model evaluation. Pure NumPy implementation.

## Installation

```bash
pip install uq-metrics
```

For plotting support:
```bash
pip install uq-metrics[plot]
```

## Usage

```python
from uq_metrics import auroc, ece, brier_score, aurc
import numpy as np

y_true = np.array([0, 0, 1, 1, 1])
y_scores = np.array([0.2, 0.3, 0.6, 0.8, 0.9])

auroc(y_true, y_scores)        # Area Under ROC Curve
ece(y_true, y_scores)          # Expected Calibration Error
brier_score(y_true, y_scores)  # Brier Score

# With plotting
score, ax = auroc(y_true, y_scores, plot=True)
```

## Available Metrics

- `auroc` - Area Under ROC Curve
- `ece` - Expected Calibration Error
- `brier_score` - Brier Score
- `aurc` - Area Under Risk-Coverage Curve
- `error_vs_abstention` - Error rates at abstention levels
- `optimal_abstention` - Find optimal abstention threshold
