# inewave Benchmark Suite

This directory contains tooling to measure inewave's runtime performance across
three dimensions: module import time, file read time, and DataFrame aggregation
time. Results are written as Markdown tables to
`benchmarks/benchmark_results.md` so they can be diffed in version control and
compared across runs.

---

## Files

| File                   | Purpose                                                                                                                                                                                          |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `run_benchmarks.py`    | Entry point. Runs all benchmarks and writes `benchmark_results.md`.                                                                                                                              |
| `bench_read.py`        | Benchmark functions for read operations. Uses `mock_open` to avoid I/O variance.                                                                                                                 |
| `bench_import.py`      | Benchmark functions for module import operations. Clears only `inewave.*` from `sys.modules` between iterations so pandas/numpy/cfinterface startup cost is not re-paid.                         |
| `profile_read.py`      | Phase-level profiler. Monkey-patches internal helpers to record time spent in parsing, DataFrame construction, formatting, and concatenation. Also runs cProfile and writes `profile_report.md`. |
| `benchmark_results.md` | Latest benchmark output. Regenerated by `run_benchmarks.py`. Do not edit by hand.                                                                                                                |
| `profile_report.md`    | Latest profiler output. Regenerated by `profile_read.py`. Do not edit by hand.                                                                                                                   |
| `__init__.py`          | Makes `benchmarks/` a package so its modules can import each other.                                                                                                                              |

---

## How to Run

All commands must be run from the **repository root** so that `benchmarks/` is
resolvable as a package.

### Full benchmark suite

```bash
python benchmarks/run_benchmarks.py
```

Runs import, read, and aggregation benchmarks with the default of 10 iterations
each and overwrites `benchmarks/benchmark_results.md`. To change the iteration
count:

```bash
python benchmarks/run_benchmarks.py --iterations 20
```

On the second and subsequent runs a Delta column is populated by comparing the
new means against the values stored in the previous results file.

### Phase-level profiler

```bash
python benchmarks/profile_read.py
```

Measures wall-clock time broken down by internal phase (parsing, DataFrame
construction, formatting, concatenation) for each representative file type, and
runs cProfile to produce per-function call statistics. Output is written to
`benchmarks/profile_report.md`.

---

## Interpreting Results

### benchmark_results.md tables

The file contains three tables, each sharing the same column schema.

| Column               | Meaning                                                                                                                                                                                          |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Benchmark**        | Name of the operation being measured.                                                                                                                                                            |
| **Mean (s)**         | Arithmetic mean of all iteration wall-clock times in seconds. The primary comparison metric.                                                                                                     |
| **Delta**            | Percentage change of Mean vs the previous run's Mean for the same benchmark. Only present when a previous results file existed before the run. A positive Delta means the current run is slower. |
| **Median (s)**       | Median of iteration times. Less sensitive to outliers than mean.                                                                                                                                 |
| **Min (s)**          | Fastest single iteration. Approximates best-case performance.                                                                                                                                    |
| **Max (s)**          | Slowest single iteration.                                                                                                                                                                        |
| **Std Dev (s)**      | Sample standard deviation across iterations. High values indicate noisy measurements.                                                                                                            |
| **Peak Memory (MB)** | Peak Python-level heap delta measured by `tracemalloc`. This captures only allocations made by Python objects; C-level buffers (numpy, pandas internals) are not fully reflected.                |

**Import Benchmarks** measure the time to `importlib.import_module` a module
from a partially warm cache (only `inewave.*` entries are cleared; pandas,
numpy, and cfinterface remain loaded).

**Read Benchmarks** measure the time for a handler's `read()` call against
in-memory mock data (same mock files used by the test suite). I/O is
eliminated; the measurement reflects parsing and DataFrame construction cost
only.

**Aggregation Benchmarks** measure the time for the public `.valores()`
property accessor, which triggers `__monta_tabela` to concatenate per-block
DataFrames into the final result.

### profile_report.md tables

The Summary Table shows what fraction of total read time each phase consumed:

| Column             | Meaning                                                                    |
| ------------------ | -------------------------------------------------------------------------- |
| **Total Time (s)** | Best-of-5 wall-clock time for the full `read()` call.                      |
| **Parsing %**      | Time spent in `TabularParser.parse_lines` as a percentage of total.        |
| **DataFrame %**    | Time spent in `_build_dataframe` + `formata_df_meses_para_datas_nwlistop`. |
| **Aggregation %**  | Time spent in the `__monta_tabela` concat loop.                            |
| **Other %**        | Remainder: cfinterface block dispatch, section detection, Python overhead. |

The Raw Phase Times table gives absolute seconds for each phase. The
Per-File cProfile Top-20 sections list the 20 functions with the highest
cumulative time for each file type.

---

## Key Performance Characteristics

Numbers below are from the most recent run recorded in
`benchmarks/benchmark_results.md` and `benchmarks/profile_report.md`.

### Import times

Importing the top-level `inewave` package takes roughly **0.36 s** in a
partially-warm process (pandas/numpy already loaded). Sub-package imports
(`inewave.nwlistop`, `inewave.newave`) are much cheaper at roughly **4.6 ms**
each because the top-level package has already resolved shared dependencies.
A single-file import (`inewave.nwlistop.earmf`) costs roughly **7.2 ms**.

The large top-level cost reflects Python's module-graph traversal for a package
with many sub-modules on first access in each process.

### Read times

| File type               | Representative handler        | Mean read time |
| ----------------------- | ----------------------------- | -------------- |
| Non-patamar (REE / SIN) | `Earmf`, `Earmfsin`, `Earmfp` | ~67–74 ms      |
| Patamar Submercado      | `Cmarg`                       | ~210 ms        |
| Newave SectionFile      | `Pmo`                         | ~644 ms        |
| Newave binary           | `Hidr` (aggregation)          | ~84 ms         |

Parsing (`TabularParser.parse_lines`) accounts for roughly 28–32 % of read
time for the nwlistop file types. DataFrame construction adds another 21 %.
The remaining ~50 % is cfinterface block dispatch and Python overhead.

`Cmarg` is ~3x slower than `Earmf` because patamar files have 3 patamares per
time step, tripling the number of `Line.read` calls through the
`FloatField._textual_read` hot path.

`Pmo` is the slowest reader (644 ms) because its `converte_tabela_em_df` method
uses `DataFrame.apply` + `DataFrame.melt` on every section block. cProfile
attributes roughly 171 ms of the total to these two pandas operations.

### Aggregation times

Calling `.valores()` on already-loaded nwlistop objects (Earmf, Earmfsin,
Cmarg, Earmfp) takes roughly **0.1 ms** with the mock test data. This appears
negligible but the underlying `__monta_tabela` implementation uses an
incremental `pd.concat` pattern that scales as O(n^2) copies in the number
of year-blocks. On real production files with 20+ year-blocks the cost is
significantly higher than what the benchmarks show, because the mock data
contains only a single year-block.

---

## Architecture: How the Profiler Works

`profile_read.py` measures phases using **targeted monkey-patching** rather
than modifying production code:

1. Before each `read()` call it wraps the internal helpers with timing
   decorators using `unittest.mock.patch`. The patched functions record
   `time.perf_counter_ns()` at entry and exit and accumulate elapsed time in a
   `PhaseTimings` dataclass.
2. The four instrumented phases are:
   - **Parse**: `cfinterface.components.tabular.Tabular.parse_lines`
   - **Build-DF**: `TabelaSerieAnual._build_dataframe`
   - **Format**: `_utils.formatacao.formata_df_meses_para_datas_nwlistop`
   - **Concat**: `__monta_tabela` (the per-block `pd.concat` loop inside file
     aggregation classes)
3. Each measurement is the best of 5 iterations using `time.perf_counter_ns()`.
4. cProfile is run in a separate pass for each file type with
   `cProfile.Profile().enable()` / `.disable()` wrapping the same
   `profiled_read` function. The top-20 functions by cumulative time are
   captured via `pstats.Stats`.

Because the patches are applied and removed around each call, production code
is never modified and the profiler has no effect on the test suite.

---

## Notes on Result Reliability

- **Delta noise on low-iteration runs.** With few samples, the Delta column
  reflects run-to-run variation more than real performance change.
  Use `--iterations 20` or higher for stable Delta values.

- **`__monta_tabela` is invisible in mock benchmarks.** The aggregation
  benchmarks use single-block mock files, so `__monta_tabela` runs only one
  `pd.concat` call. The O(n^2) scaling problem only manifests on real files
  with many year-blocks. Do not conclude from the ~0.1 ms aggregation times
  that this path is fast in production.

- **Memory measurements are Python-level only.** `tracemalloc` tracks
  Python-object allocations. numpy and pandas back their arrays with C-level
  memory that is not captured, so Peak Memory (MB) values undercount true
  resident-set growth.

- **Import benchmark cache state.** `bench_import.py` clears only
  `inewave.*` entries from `sys.modules`. cfinterface, pandas, and numpy
  remain loaded. The measured import times therefore represent the marginal
  cost of loading inewave's own module graph, not a fully cold-cache scenario.

- **Phase percentages in profile_report.md do not sum to 100 %.** The phases
  are not mutually exclusive: `_build_dataframe` calls
  `formata_df_meses_para_datas_nwlistop` internally, so the DataFrame % and
  Parsing % columns can overlap.
