Metadata-Version: 2.3
Name: eta-stress
Version: 1.0.6
Summary: Library for conducting global sensitivity analyses and stress tests. Interoperable and model agnostic API. HPC suitability approved.
License: BSD-2-Clause
Keywords: stress-test,global-sensitivity-analysis,optimization,uncertainty,eta-incerto
Author: Technical University of Darmstadt, Institute for Production Management, Technology and Machine Tools (PTW)
Author-email: m.frank@ptw.tu-darmstadt.de
Requires-Python: >=3.11, <3.13
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: eta-incerto (>=1.2.0,<2.0.0)
Project-URL: Bug Tracker, https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/-/issues
Project-URL: Changelog, https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/-/blob/publication-adaptions/CHANGELOG.md
Project-URL: Documentation, https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/-/blob/publication-adaptions/README.md
Project-URL: Homepage, https://www.ptw.tu-darmstadt.de
Project-URL: Repository, https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress
Description-Content-Type: text/markdown

# ETA Stress

[![pipeline](https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/badges/main/pipeline.svg)](https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/-/commits/main)
[![license](https://img.shields.io/badge/License-BSD--2--Clause-blue.svg)](LICENSE)

`eta-stress` is the standalone stress-test orchestration package for ETA workflows.
It integrates with `eta-incerto` through a thin adapter boundary and provides a
clean Tier 1 + Tier 2 + GSA pipeline with explicit contracts and artifact persistence.

## Key Features

- Tier 1 local sensitivity analysis on a solved baseline
- Tier 2 scenario generation, perturbation, and scorecard aggregation
- Tier 3 global sensitivity analysis (PAWN / Sobol) with HDF5 artifacts
- Thin adapter boundary to `eta-incerto` with explicit run contracts
- CLI-driven orchestration and optional plotting summaries

## Overview

```mermaid
flowchart TD
  bootstrap[bootstrap] --> preflight[preflight]
  preflight --> variant_registry[variant_registry]
  variant_registry --> baseline[baseline solve]
  baseline --> tier1[tier1 analyze]
  tier1 --> tier2[tier2 scenarios]
  tier2 --> aggregation[tier2 aggregation]
  aggregation --> gsa[tier3 gsa]
  gsa --> provenance[provenance]
```

## What This Package Runs

The orchestrator executes the full stress-test pipeline:

1. **bootstrap**: create namespace folders under `run_dir`.
2. **preflight**: validate config, mode, scenario files, series files, and variant layout.
3. **variant_registry**: import variant runtime module and validate registry availability.
4. **baseline resolution**: solve baseline through `EtaBackend` or import an existing baseline.
5. **baseline artifacts**: persist `baseline.json` and `incumbent.npz`.
6. **tier1 analyze**: local sensitivity analysis over solved baseline.
7. **tier2 source**: resolve scenarios from `generate` or `existing`.
8. **tier2 runner**: run per-scenario context -> perturb -> incumbent_eval -> metrics -> persist.
9. **tier2 aggregation**: aggregate scenario results into scorecard.
10. **tier3 gsa**: build factor/response data (X/Z), run PAWN/Sobol, persist `gsa.h5`.
11. **provenance**: persist final run provenance and artifact links.

## Installation

```bash
poetry install
```

Compatibility:

- Python: `>=3.11,<3.13`
- `eta-incerto`: `>=1.2.0,<2.0.0` (current package constraint: `^1.2.0`)

See [Solver and third-party requirements](#solver-and-third-party-requirements) for MILP solver notes (via eta-incerto / eta-components).

## CLI Usage

The CLI takes required runtime arguments plus an optional stress-config overrides file.

```bash
poetry run python -m eta_stress.cli.main \
  --run-dir /path/to/run_dir \
  --config-name config \
  --mode stochastic \
  --variant variant_one \
  --variant-import-root examples.conventional \
  --scenario-source generate \
  --stress-config /path/to/stress_test_config.json \
  --plot \
  --plot-heatmap
```

Required CLI arguments:

- `--run-dir`
- `--config-name`
- `--mode`
- `--variant`
- `--variant-import-root`
- `--scenario-source` (`generate|existing`)

Optional:

- `--stress-config` (policy/tolerance overrides)
- `--stress-evaluation-mode` (`fixed_first_stage|full_reopt`) overrides evaluation mode from config
- `--strict-antifragile` / `--no-strict-antifragile` toggle strict antifragile execution (no synthetic fallback when strict)
- `--antifragile-pareto-selection` (`weighted_sum|min_first_objective`) controls antifragile Pareto point selection for stress baseline
- `--antifragile-objective-weights` comma-separated objective weights for `weighted_sum` (default `1/3,1/3,1/3`)
- `--chapter8-price-only-uncertainty` / `--no-chapter8-price-only-uncertainty` enforce Chapter-8 uncertainty scope
- `--plot` (render plotting summaries after pipeline completion)
- `--plot-preset` (style preset for plotting)
- `--plot-heatmap` (enable heatmap panel in per-scenario plotting)

Supported config file formats:

- `.json`
- `.yml`
- `.yaml`
- `.ymal`

## Stress-Test Config Contract

Main model: `eta_stress.orchestrator.stress_test_config.StressTestConfig`.

Required fields:
- none (all fields are optional overrides now)

Important behavior:

- `sensitivity_namespace` is auto-generated by orchestrator using timestamp + incremental index.
- If `generation_policy.scenario_probability_candidates` is provided, `scenario_probability_deltas` is ignored.
- If neither `scenario_probability_candidates` nor `scenario_probability_deltas` is provided, probability deltas are auto-generated.
- `generation_policy.max_scenarios` is applied during scenario generation (early cap), so only up to that many candidates are built.
- Solver option path for MIP gap is normalized to `pyomo.solver_settings.options.mipgap`.
- `stress_evaluation_mode` controls scenario solve semantics:
  - `fixed_first_stage`: evaluate baseline first-stage decisions under stress. This is the default production path.
  - `full_reopt`: re-optimize first- and second-stage decisions under stress. This is kept for compatibility/debugging and explicit re-optimization studies.
- `strict_antifragile` controls antifragile fallback behavior:
  - `true`: fail fast if antifragile runtime/h5/incumbent requirements are not met.
  - `false`: allow synthetic placeholder fallback behavior for compatibility/debugging.
- `antifragile_pareto_selection` controls antifragile Pareto scalarization for stress baseline:
  - `weighted_sum` (default) uses `antifragile_objective_weights` (defaults to equal weights `1/3,1/3,1/3`).
  - `min_first_objective` reproduces legacy behavior.
- `chapter8_price_only_uncertainty` defaults to `true` and enforces uncertainty channels to gas/electricity yearly `unit_price` only.
- `rulebook_preset.allowed_carriers` / `rulebook_preset.allowed_fields` default to `["gas","electricity"]` and `["unit_price"]`.

Complete example config:

- [`eta_stress/stress_test_config.example.json`](eta_stress/stress_test_config.example.json)

## Variant Registry Import Rule

`variant_registry` imports:

```text
<variant_import_root>.variants.<variant>.system
```

Then it validates runtime registration via `eta_incerto.registry.get_system(variant)`.

Example:

- `variant_import_root = "examples.conventional"`
- `variant = "variant_one"`
- imported module = `examples.conventional.variants.variant_one.system`

## Output Layout

For each run, a namespace directory is created under `run_dir`:

```text
run_dir/
  sensitivity_YYYY-MM-DD_HH-MM-SSZ_XXXX/
    baseline/
      baseline.json
      incumbent.npz
      tier1_analysis.json
    scenarios/
      scenarios.json
    results/
      <scenario_id>.h5
    aggregate/
      scorecard.h5
    gsa/
      gsa.h5
    plots/
      conventional_stress_scorecard_summary.png
      conventional_stress_scorecard_summary.pdf
      conventional_stress_scorecard_evaluation.md
      conventional_stress_per_scenario_summary.png
      conventional_stress_per_scenario_summary.pdf
      conventional_stress_per_scenario_evaluation.md
    provenance.json
```

`gsa/gsa.h5` payload contract (`payload_json`) is:

- `gsa.factor_values`
  - `scenario_ids[]`
  - `factor_names[]`
  - `matrix[][]`
- `gsa.validation_z`
  - `metric_key`
  - `values[]`
- `gsa.pawn` (flatten metrics map, keys `gsa.pawn.*`)
- `gsa.sobol` (flatten metrics map, keys `gsa.sobol.*`)

## Runtime Model

`eta-stress` runs one sensitivity namespace per execution:

1. bootstrap
2. preflight
3. variant registry
4. baseline resolution + baseline artifacts
5. Tier 1 analysis when a solved baseline model is available
6. Tier 2 source (generate/existing)
7. Tier 2 scenario runner
8. Tier 2 aggregation
9. GSA (X/Z + PAWN/Sobol payload)
10. optional plotting (`--plot`)
11. provenance persistence

## Chapter 8 Evidence Protocol

- Pilot stress run: `generation_policy.max_scenarios = 200` for ranking/interval stability check.
- Final stress run: `generation_policy.max_scenarios = 500` once pilot is stable.
- Keep Chapter-8 uncertainty scope price-only (`chapter8_price_only_uncertainty=true` and price-only rulebook preset).

## Package Structure (High Level)

- `eta_stress/orchestrator`: top-level pipeline and run config loading.
- `eta_stress/services/tier1`: baseline local sensitivity analysis.
- `eta_stress/services/tier2`: scenario source, runner, and aggregation runners.
- `eta_stress/services/gsa`: Tier 3 GSA encoding/reports/runner.
- `eta_stress/services/artifacts.py`: artifact persistence (idempotent writes).
- `eta_stress/adapters/eta_backend/`: boundary to `eta-incerto` runtime operations.
  - `backend.py`: public `EtaBackend` facade.
  - `baseline.py`: baseline solve dispatch for deterministic, stochastic, robust, regret, and antifragile modes.
  - `runtime.py`: incumbent evaluation and optional full re-optimization runtime calls.
  - `perturbations.py`: local stress perturbation injection.
  - `antifragile.py`: antifragile Pareto H5 extraction and selection.
  - `status.py`: solver/status normalization.
- `eta_stress/io`: JSON/YAML/H5/path utilities.
- `eta_stress/core`: shared domain types, statuses, provenance, and numeric helpers.

## Testing and Validation

See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, tests, and merge request workflow.

Run tests (environment with dev dependencies):

```bash
poetry run pytest
```

Quick syntax check:

```bash
find eta_stress -name '*.py' -print0 | xargs -0 python3 -m py_compile
```

## Notes

- `tools/extract_stress_from_incerto.py` is a migration utility and not part of normal runtime execution.
- The current implementation prioritizes strict contracts and simple stage boundaries over legacy report shape parity.
- A successful end-to-end run typically persists: `baseline.json`, `tier1_analysis.json`, `scenarios/scenarios.json`, `aggregate/scorecard.h5`, `gsa/gsa.h5`, and `provenance.json` (plus `plots/*` if `--plot` is enabled).
- HDF5 writer format tags are `eta_stress.*` (`eta_stress.scenario_result`, `eta_stress.scorecard`, `eta_stress.gsa`).
- HDF5 readers currently accept payloads from both old and new format tags as long as `payload_json` is valid.
- E2E artifact checks are available via `eta_stress.e2e.contract.assert_e2e_artifacts`.
- Current status: Tier 1–2 (including GSA) is integrated in `eta-stress`; remaining epic closure items are mainly cross-repo cutover and real-config E2E/harness alignment.
- Detailed Slurm/venv split patterns (optimization stage vs stress stage, array indexing, worker/thread tuning) are intentionally tracked in a separate follow-up issue and are not finalized in this README yet.

## Recent Backend and Tier 2 Changes

### Eta Backend Adapter

`eta_stress.adapters.eta_backend` is now a package instead of a single adapter file. The public import remains stable:

```python
from eta_stress.adapters.eta_backend import BackendRequest, EtaBackend, RuntimeVersions
```

The public `EtaBackend` facade intentionally stays small:

- `solve_baseline(request)`
- `evaluate_incumbent(request)`
- `solve_scenario(request)`
- `get_runtime_versions()`

The implementation is split by responsibility:

- `baseline.py` handles baseline solves for `deterministic`, `stochastic`, `robust`, `regret`, and `antifragile`.
- `runtime.py` handles per-scenario runtime evaluation.
- `perturbations.py` applies local model, solver, scenario probability, and scenario CSV perturbations.
- `antifragile.py` reads antifragile Pareto H5 outputs and selects a point via `weighted_sum` or `min_first_objective`.
- `incumbent.py` extracts first-stage incumbent maps from solved frameworks, bundles, or antifragile Pareto H5 files.
- `status.py` normalizes solver status and feasibility flags.

The new adapter no longer imports `eta_incerto.stress_test.*`. It may still import normal `eta_incerto` runtime modules, evaluators, config builders, and model utilities.

### Baseline Resolution

There are two baseline paths:

- `baseline_resolution=solve`: eta-stress solves the baseline, persists `baseline.json` and `incumbent.npz`, and keeps the live solved system object for Tier 1.
- `baseline_resolution=import`: eta-stress imports the baseline objective and incumbent from an existing optimization result H5, persists `baseline.json` and `incumbent.npz`, and skips Tier 1 because no live solved Linopy model is available.

Tier 1 currently requires the solved model object, specifically access to `solved_system._backend.model`, model constraints with `lhs.solution`, `rhs`, `sign`, and variables with `solution`, `lower`, `upper`. Existing H5 outputs are not a uniform replacement for that object across all modes.

Observed H5 shape by mode:

- `stochastic`, `robust`, `regret`: expose conventional RODMO-style expected groups such as `__expected__/investment`, `__expected__/unit_dispatch`, and `__expected__/carriers`.
- `deterministic`: writes scenario-level result groups and dispatch data rather than the same expected-result contract.
- `antifragile`: exposes Pareto data such as `pareto/X`, `pareto/F`, `pareto/variable_names`, and `pareto/objective_names`.

Because of that, import mode is currently Tier-2-oriented: it imports the incumbent design and evaluates that fixed design under stress.

### Incumbent Evaluation Flow

`evaluate_incumbent` evaluates the existing first-stage design under one stress scenario:

1. Read `incumbent.npz` from the namespace baseline artifacts.
2. Load the eta-incerto experiment and the scenario/system collection for the requested variant.
3. Apply the scenario perturbations to model parameters, solver options, scenario probabilities, and/or scenario CSV values.
4. Convert the incumbent vector into first-stage investment decisions.
5. Fix those first-stage investment variables on each perturbed scenario system.
6. Run eta-incerto's `DeterministicEvaluator` on the perturbed scenario system with the first-stage variables already fixed. At this point the scenario is concrete and the investment design is fixed, so the remaining problem is a deterministic operational evaluation rather than a stochastic/robust/regret investment optimization.
7. Return `objective_value`, solver status, feasibility, fixed-design metadata, and diagnostics for scenario metrics.

The important distinction is that `incumbent_eval` does not choose a new investment design. It fixes the baseline/imported first-stage design first, then evaluates how that design performs under the perturbed scenario. This matches the production Tier 2 goal: post-hoc stress evaluation of an existing design.

`full_reopt` remains available only when explicitly requested:

- `fixed_first_stage`: `scenario_runner` calls `evaluate_incumbent` and persists `resolve` as skipped for compatibility.
- `full_reopt`: `scenario_runner` calls both `evaluate_incumbent` and `solve_scenario`.

The decision to run or skip resolve lives in `scenario_runner`, not in the backend facade.

### Scenario Metrics

Scenario-level metrics always include incumbent metrics:

- `objective.incumbent_eval.cost`
- `loss.incumbent_vs_baseline.abs`
- `loss.incumbent_vs_baseline.rel`
- `flag.incumbent_feasible`
- `decision.hamming_rate`
- `fragility.F_s`

`fragility.F_s` is based on incumbent stress loss and incumbent feasibility. It does not depend on resolve metrics in the default production path.

When `stress_evaluation_mode=full_reopt` produces a real resolve result, scenario metrics may also include resolve extras:

- `objective.resolve.cost`
- `loss.resolve_vs_baseline.abs`
- `loss.resolve_vs_baseline.rel`
- `loss.resolve_vs_incumbent.abs`
- `loss.resolve_vs_incumbent.rel`
- `flag.resolve_feasible`

### Aggregation Metrics

Aggregation always computes incumbent-based summaries:

- core objective spread uses `objective.incumbent_eval.*`;
- threshold and tail metrics use `objective.incumbent_eval.cost`;
- pair/asymmetry metrics use `loss.incumbent_vs_baseline.abs`;
- fragility CVaR uses `loss.incumbent_vs_baseline.rel`.

When resolve values exist, aggregation also emits resolve summaries:

- `objective.resolve.*`
- resolve-vs-baseline loss summaries;
- resolve tail/CVaR summaries;
- resolve-enabled flatness/count diagnostics.

Those resolve summaries are optional and are not required for fixed-first-stage production runs.

### Plotting

Plotting now prefers incumbent metrics:

- scorecard summaries use `objective.incumbent_eval.*` as the primary objective family;
- per-scenario summaries use incumbent stress cost and incumbent-vs-baseline loss;
- all-method sensitivity plots use incumbent objective and feasibility signals.

When `full_reopt` is used and resolve metrics exist, plotting can include resolve information as an additional comparison layer. Fixed-first-stage plots do not require resolve metrics.

### Provenance and Summaries

Run summaries now report incumbent objective flatness/count diagnostics independently of resolve:

- `incumbent_objective_count`
- `incumbent_objective_unique_count`
- `incumbent_objective_missing_count`
- `incumbent_objective_flat_detected`

Resolve diagnostics are only emitted when resolve is enabled:

- `resolve_enabled`
- `resolve_skipped_count`
- `resolve_objective_count`
- `resolve_objective_unique_count`
- `resolve_objective_missing_count`
- `resolve_objective_flat_detected`

In `fixed_first_stage`, scenario payloads keep a compatibility block:

```json
{
  "resolve": {
    "status": "skipped",
    "skipped": true,
    "reason": "fixed_first_stage"
  }
}
```

This keeps older readers stable while making the production semantics explicit.

## Documentation

Project overview and usage: this README on GitLab ([README.md](https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-stress/-/blob/publication-adaptions/README.md)).

## Related software

- [eta-incerto](https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-incerto) — required dependency; provides optimization runtime and variant registry

## Solver and third-party requirements

Stress tests solve MILP models through **eta-incerto** and **eta-components** (Linopy). Baseline and scenario runs inherit solver requirements from those packages:

- **Gurobi** (`gurobipy`) — required for many default eta-incerto paths; commercial license required
- **CPLEX** — optional; see [eta-incerto scripts/README_CPLEX.md](https://git.ptw.maschinenbau.tu-darmstadt.de/eta-fabrik/public/eta-incerto/-/blob/publication-adaptions/scripts/README_CPLEX.md)
- **HiGHS** — open-source alternative where supported

This package is licensed under **BSD-2-Clause**. Solver products are third-party software with their own licenses and terms.

## Citation

For academic use, cite this repository using [CITATION.cff](CITATION.cff). See [AUTHORS.rst](AUTHORS.rst)
for further contributors.

## License

BSD-2-Clause — see [LICENSE](LICENSE). See also [CHANGELOG.md](CHANGELOG.md) and [SECURITY.md](SECURITY.md).

