Metadata-Version: 2.4
Name: dsa-metrics
Version: 0.1.0
Summary: Fast, vectorized regression error and directional/trading metrics for forecasts and time series.
Author-email: Patrick Petter <ppetter@datasphere-analytics.com>
License-Expression: MIT
Project-URL: Homepage, https://datasphere-analytics.com
Keywords: metrics,forecasting,time-series,regression,backtesting,sharpe,pnl
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# DSA Metrics

`dsa_metrics` is a small pandas-first package for fast regression error analysis.

It supports:

- one dataframe or one CSV that already contains timestamps, predictions, and true values
- separate prediction and truth dataframes or CSVs with different column names
- global metrics or metrics grouped per timestamp

Supported metrics:

Error metrics:

- `mae`
- `mse`
- `rmse`
- `mape`
- `wape`
- `smape`
- `accuracy`

Directional / trading-style metrics (following the DSA `model_performance` report logic):

- `hit_rate` — directional accuracy: fraction of steps where the predicted move matches the realised move (alias: `directional_accuracy`, `da`)
- `mock_pnl` — total mock PnL: take the predicted direction each step, earn the realised return (alias: `pnl`)
- `pnl_mean` — mean per-step mock PnL (alias: `mean_pnl`)
- `pnl_std` — sample standard deviation (ddof=1) of the per-step mock PnL (alias: `std_pnl`)
- `pnl_sharpe` — Sharpe ratio of the per-step mock PnL: `pnl_mean / pnl_std` (alias: `sharpe`, `sharpe_ratio`)
- `pnl_t_stat` — one-sample t-statistic of the per-step mock PnL against the null hypothesis mean = 0 (alias: `t_stat`, `tstat`)

Notes:

- `mape`, `wape`, and `smape` are returned as fractions, not percentages
- `accuracy` is defined as `1 - smape`
- `hit_rate` is returned as a fraction in `[0, 1]` (multiply by 100 for a percentage)
- The directional / PnL metrics treat `pred_column` and `true_column` as **level** series and
  difference consecutive rows internally:
  `target_return = target - lag(target)`, `predicted_return = pred - lag(pred)`,
  `direction = sign(predicted_return)`, `ts_pnl = direction * target_return`.
  Because they depend on row order, pass the data **time-ordered** and use `mode="global"`
  over a single series (not `mode="per_timestamp"`).

## Install

```bash
pip install -e .
```

## Quick Start

```python
import pandas as pd

from dsa_metrics import calculate_metrics

df = pd.DataFrame(
    {
        "ds": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "y_pred": [101.0, 98.0, 110.0],
        "y": [100.0, 100.0, 105.0],
    }
)

global_metrics = calculate_metrics(
    df,
    timestamp_column="ds",
    pred_column="y_pred",
    true_column="y",
    metrics=["mae", "rmse", "smape", "accuracy"],
    mode="global",
)

per_timestamp_metrics = calculate_metrics(
    df,
    timestamp_column="ds",
    pred_column="y_pred",
    true_column="y",
    metrics=["mae", "smape"],
    mode="per_timestamp",
)
```

## Separate Prediction And Truth Sources

```python
import pandas as pd

from dsa_metrics import calculate_metrics_from_sources

predictions = pd.DataFrame(
    {
        "forecast_date": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "series_id": ["pe", "pe", "pe"],
        "forecast": [101.0, 98.0, 110.0],
    }
)

truth = pd.DataFrame(
    {
        "actual_date": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "product_id": ["pe", "pe", "pe"],
        "actual": [100.0, 100.0, 105.0],
    }
)

result = calculate_metrics_from_sources(
    predictions,
    truth,
    pred_timestamp_column="forecast_date",
    true_timestamp_column="actual_date",
    pred_column="forecast",
    true_column="actual",
    pred_key_columns="series_id",
    true_key_columns="product_id",
    metrics=["mae", "wape", "accuracy"],
    mode="global",
)
```

If your sources are CSV files, you can pass file paths instead of dataframes.

## Public API

```python
from dsa_metrics import available_metrics, calculate_metrics, calculate_metrics_from_sources
```

`calculate_metrics(...)`

- use when predictions and truth already live in the same dataframe or CSV

`calculate_metrics_from_sources(...)`

- use when predictions and truth come from separate dataframes or CSVs
- align them by timestamp and optional key columns before computing metrics

`available_metrics()`

- returns the metric names supported by the package
