Metadata-Version: 2.4
Name: tadmetric
Version: 0.1.7
Summary: A lightweight evaluation toolkit for time-series anomaly detection
Author-email: ushin20 <yooshin0303@dgist.ac.kr>
License-Expression: MIT
Project-URL: Homepage, https://github.com/ushin20/tadmetric
Project-URL: Documentation, https://github.com/ushin20/tadmetric/tree/main/docs
Project-URL: Repository, https://github.com/ushin20/tadmetric
Project-URL: Issues, https://github.com/ushin20/tadmetric/issues
Keywords: time series,anomaly detection,metrics,evaluation,tsad
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.23
Requires-Dist: scikit-learn>=1.2
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# tadmetric

Lightweight evaluation metrics for time-series anomaly detection.

`tadmetric` gives you a small, practical API for evaluating anomaly detection
results from either binary predictions or raw anomaly scores. The main workflow
is centered on `evaluator(...)`, which lets you reuse one score array across
multiple metrics and threshold-search runs.

## Why tadmetric

- Small API surface for common TSAD evaluation workflows
- Supports both `y_pred` and `y_score` based evaluation
- Fast best-threshold search for repeated experiments
- Includes point-wise, event-wise, point-adjusted, composite, delay, and curve metrics
- Extensible metric registry for custom research metrics

## Installation

```bash
pip install tadmetric
```

Python `3.9+` is supported.

## Quick Start

### 1. Evaluate binary predictions

```python
import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_pred = [0, 0, 1, 1, 0, 0, 0]

result = tm.evaluate(y_true, y_pred, metrics=("point", "point_adjusted", "composite"))

print(result["point"].asdict())
print(result["point_adjusted"].f1)
print(result["composite"].f1)
```

### 2. Evaluate scores with a reusable evaluator

```python
import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_score = [0.1, 0.2, 0.4, 0.9, 0.7, 0.1, 0.0]

e = tm.evaluator(y_true, y_score)

point = e.point_wise(thr=0.5)
adjusted = e.point_adjusted(thr=0.3)
adjusted_k = e.point_adjusted_k(thr=0.3, k=30)
composite = e.composite(thr=0.5)

print(point.asdict())
print(adjusted.f1, adjusted_k.f1, composite.f1)
```

### 3. Search for the best threshold

```python
import tadmetric as tm

y_true = [0, 0, 1, 1, 1, 0, 0]
y_score = [0.1, 0.2, 0.4, 0.9, 0.7, 0.1, 0.0]

e = tm.evaluator(y_true, y_score)
best = e.best(metric="composite", progress=True)

print(best.threshold)
print(best.f1)
print(best.evaluation["composite"].asdict())
```

`progress=True` writes to `stderr`. If you want to see it while running tests,
use `pytest -s`.

## Main API

For most workflows, these are the core entry points:

- `evaluator(y_true, y_score)`
- `evaluate(y_true, y_pred, ...)`
- `evaluate_scores(y_true, y_score, threshold=...)`
- `e.point_wise(thr=...)`
- `e.point_adjusted(thr=...)`
- `e.point_adjusted_k(thr=..., k=...)`
- `e.composite(thr=...)`
- `e.best(metric=...)`
- `available_metrics()`
- `describe_metrics()`
- `api_overview()`
- `register_metric(...)`

## Included Metrics

### Core precision / recall / F1

- Point-wise: `point_precision`, `point_recall`, `point_f1`
- Point-adjusted: `point_adjusted_precision`, `point_adjusted_recall`, `point_adjusted_f1`
- Point-adjusted `%K`: `point_adjusted_k_precision`, `point_adjusted_k_recall`, `point_adjusted_k_f1`
- Composite: `composite_precision`, `composite_recall`, `composite_f1`
- Event-wise: `event_precision`, `event_recall`, `event_f1`

### Delay-aware metrics

- `time_to_detect`
- `mean_time_to_detect`
- `median_time_to_detect`
- `missed_detection_rate`

### Thresholding and utilities

- `threshold_by_quantile`
- `threshold_by_topk`
- `threshold_by_best_f1`
- `search_best_f1_threshold`
- `apply_hysteresis`
- `scores_to_binary`
- `binary_to_intervals`
- `intervals_to_binary`
- `merge_intervals`

### Curve-based metrics

- `precision_recall_curve`
- `roc_curve`
- `auc_pr`
- `auc_roc`

## Metric Semantics

- Point-wise metrics treat each timestamp independently.
- Event-wise metrics treat each contiguous anomaly region as one event.
- Point-adjusted metrics credit a whole event once it is detected.
- Point-adjusted `%K` metrics credit an event when at least `K%` of the event is detected.
- Composite metrics use point-wise precision and event-based recall.
- Interval semantics are half-open: `[start, end)`.
- Event matching defaults to `overlap > 0`.
- `zero_division` defaults to `0.0`.

## Discoverability

If you are opening the package for the first time, these helpers are useful:

```python
import tadmetric as tm

print(tm.api_overview())
print(tm.available_metrics())
print([spec.asdict() for spec in tm.describe_metrics()])
```

## Development

```bash
pip install -e .[dev]
pytest -q
```

## Release Checklist

```bash
pytest -q
python -m build --no-isolation
python -m twine check dist/*
```
