# inewave Read Performance Profile

> Generated by `benchmarks/profile_read.py`.  Times are best-of-5 wall-clock measurements using `time.perf_counter_ns()`.  Phases are measured via targeted monkey-patching of internal helpers; overlaps between phases (build_df includes format) mean columns may not sum to exactly 100 %.

## Summary Table

| File Type | Handler | Total Time (s) | Parsing % | DataFrame % | Aggregation % | Other % |
|---|---|---|---|---|---|---|
| Non-patamar REE (Earmf) | `Earmf` | 0.024282 | 27.6% | 21.2% | 0.0% | 51.2% |
| Non-patamar SIN (Earmfsin) | `Earmfsin` | 0.022534 | 29.0% | 21.6% | 0.0% | 49.4% |
| Non-patamar Submercado (Cdef) | `Cdef` | 0.002508 | 0.3% | 105.3% | 0.1% | 0.0% |
| Patamar Submercado (Cmarg) | `Cmarg` | 0.078079 | 31.6% | 13.7% | 0.0% | 54.7% |
| Patamar REE (Earmfp) | `Earmfp` | 0.023125 | 28.6% | 20.7% | 0.0% | 50.7% |
| Newave SectionFile (Pmo) | `Pmo` | 0.236452 | 0.0% | 0.0% | 0.0% | 100.0% |

## Raw Phase Times (seconds)

| File Type | Total (s) | Parse (s) | Build-DF (s) | Format (s) | Concat (s) |
|---|---|---|---|---|---|
| Non-patamar REE (Earmf) | 0.024282 | 0.006707 | 0.004038 | 0.001104 | 0.000004 |
| Non-patamar SIN (Earmfsin) | 0.022534 | 0.006546 | 0.003831 | 0.001032 | 0.000004 |
| Non-patamar Submercado (Cdef) | 0.002508 | 0.000009 | 0.001972 | 0.000667 | 0.000003 |
| Patamar Submercado (Cmarg) | 0.078079 | 0.024681 | 0.009083 | 0.001607 | 0.000004 |
| Patamar REE (Earmfp) | 0.023125 | 0.006602 | 0.003752 | 0.001036 | 0.000004 |
| Newave SectionFile (Pmo) | 0.236452 | 0.000000 | 0.000000 | 0.000000 | 0.000000 |

## Import Time

| Module | Time (s) |
|---|---|
| `inewave` | 0.165178 |
| `inewave.nwlistop` | 0.017776 |
| `inewave.newave` | 0.013677 |

## Per-File cProfile Top-20

### Non-patamar REE (Earmf)

```
384417 function calls (384043 primitive calls) in 0.105 seconds

   Ordered by: cumulative time
   List reduced from 769 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.105    0.105 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.103    0.103 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.103    0.103 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.000    0.000    0.103    0.103 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
        1    0.002    0.002    0.102    0.102 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:60(read)
     2046    0.002    0.000    0.047    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
        1    0.004    0.004    0.039    0.039 /home/rogerio/git/cfinterface/cfinterface/components/tabular.py:58(parse_lines)
     2046    0.009    0.000    0.037    0.000 /usr/lib64/python3.14/unittest/mock.py:1181(_increment_mock_call)
     2002    0.000    0.000    0.033    0.000 /home/rogerio/git/cfinterface/cfinterface/components/line.py:49(read)
     2002    0.001    0.000    0.033    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:66(read)
     2002    0.003    0.000    0.032    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:54(__positional_reading)
    26002    0.007    0.000    0.025    0.000 /home/rogerio/git/cfinterface/cfinterface/components/field.py:41(read)
    10227    0.017    0.000    0.020    0.000 /usr/lib64/python3.14/unittest/mock.py:2572(__new__)
    24000    0.011    0.000    0.016    0.000 /home/rogerio/git/cfinterface/cfinterface/components/floatfield.py:51(_textual_read)
        1    0.000    0.000    0.008    0.008 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:90(_build_dataframe)
     2046    0.001    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1178(_mock_call)
     2046    0.002    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1227(_execute_mock_call)
    72001    0.005    0.000    0.005    0.000 {method 'replace' of 'str' objects}
2093/2086    0.003    0.000    0.005    0.000 /usr/lib64/python3.14/unittest/mock.py:691(__getattr__)
47424/47351    0.004    0.000    0.004    0.000 {built-in method builtins.isinstance}
```

### Non-patamar SIN (Earmfsin)

```
384333 function calls (383959 primitive calls) in 0.094 seconds

   Ordered by: cumulative time
   List reduced from 763 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.094    0.094 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.092    0.092 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.092    0.092 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.000    0.000    0.092    0.092 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
        1    0.002    0.002    0.091    0.091 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:60(read)
        1    0.004    0.004    0.039    0.039 /home/rogerio/git/cfinterface/cfinterface/components/tabular.py:58(parse_lines)
     2046    0.002    0.000    0.035    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
     2001    0.000    0.000    0.034    0.000 /home/rogerio/git/cfinterface/cfinterface/components/line.py:49(read)
     2001    0.001    0.000    0.033    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:66(read)
     2001    0.003    0.000    0.033    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:54(__positional_reading)
     2046    0.010    0.000    0.026    0.000 /usr/lib64/python3.14/unittest/mock.py:1181(_increment_mock_call)
    26001    0.007    0.000    0.025    0.000 /home/rogerio/git/cfinterface/cfinterface/components/field.py:41(read)
    24000    0.011    0.000    0.016    0.000 /home/rogerio/git/cfinterface/cfinterface/components/floatfield.py:51(_textual_read)
    10227    0.006    0.000    0.008    0.000 /usr/lib64/python3.14/unittest/mock.py:2572(__new__)
        1    0.000    0.000    0.007    0.007 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:90(_build_dataframe)
     2046    0.001    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1178(_mock_call)
     2046    0.002    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1227(_execute_mock_call)
    72001    0.005    0.000    0.005    0.000 {method 'replace' of 'str' objects}
2093/2086    0.003    0.000    0.005    0.000 /usr/lib64/python3.14/unittest/mock.py:691(__getattr__)
47422/47349    0.004    0.000    0.005    0.000 {built-in method builtins.isinstance}
```

### Non-patamar Submercado (Cdef)

```
20332 function calls (19943 primitive calls) in 0.009 seconds

   Ordered by: cumulative time
   List reduced from 768 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.009    0.009 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.007    0.007 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.007    0.007 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.000    0.000    0.006    0.006 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
        1    0.000    0.000    0.005    0.005 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:60(read)
        1    0.000    0.000    0.005    0.005 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:90(_build_dataframe)
      3/1    0.000    0.000    0.002    0.002 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/generic.py:6360(astype)
        1    0.000    0.000    0.002    0.002 /home/rogerio/git/inewave/tests/mocks/mock_open.py:17(mock_open)
       13    0.000    0.000    0.002    0.000 /usr/lib64/python3.14/unittest/mock.py:2174(__init__)
        1    0.000    0.000    0.002    0.002 /home/rogerio/git/inewave/inewave/_utils/formatacao.py:148(formata_df_meses_para_datas_nwlistop)
        1    0.000    0.000    0.001    0.001 /home/rogerio/git/inewave/inewave/_utils/formatacao.py:10(__formata_df_meses_para_datas_serie)
        4    0.000    0.000    0.001    0.000 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/frame.py:702(__init__)
        3    0.000    0.000    0.001    0.000 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/internals/construction.py:375(dict_to_mgr)
        1    0.000    0.000    0.001    0.001 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/reshape/concat.py:157(concat)
       43    0.000    0.000    0.001    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
        1    0.000    0.000    0.001    0.001 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/reshape/concat.py:553(_get_result)
        3    0.000    0.000    0.001    0.000 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/internals/construction.py:96(arrays_to_mgr)
       26    0.000    0.000    0.001    0.000 /usr/lib64/python3.14/unittest/mock.py:2180(_mock_set_magics)
       11    0.000    0.000    0.001    0.000 /usr/lib64/python3.14/unittest/mock.py:1061(_get_child_mock)
      4/2    0.000    0.000    0.001    0.000 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/indexing.py:1192(__getitem__)
```

### Patamar Submercado (Cmarg)

```
1142246 function calls (1141882 primitive calls) in 0.308 seconds

   Ordered by: cumulative time
   List reduced from 723 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.308    0.308 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.306    0.306 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.306    0.306 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.001    0.001    0.305    0.305 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
        1    0.006    0.006    0.304    0.304 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_patamar_anual.py:68(read)
     6046    0.005    0.000    0.134    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
        1    0.012    0.012    0.129    0.129 /home/rogerio/git/cfinterface/cfinterface/components/tabular.py:58(parse_lines)
     6002    0.001    0.000    0.111    0.000 /home/rogerio/git/cfinterface/cfinterface/components/line.py:49(read)
     6002    0.002    0.000    0.109    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:66(read)
     6002    0.011    0.000    0.107    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:54(__positional_reading)
     6046    0.028    0.000    0.107    0.000 /usr/lib64/python3.14/unittest/mock.py:1181(_increment_mock_call)
    84002    0.024    0.000    0.082    0.000 /home/rogerio/git/cfinterface/cfinterface/components/field.py:41(read)
    30227    0.047    0.000    0.054    0.000 /usr/lib64/python3.14/unittest/mock.py:2572(__new__)
    72000    0.033    0.000    0.049    0.000 /home/rogerio/git/cfinterface/cfinterface/components/floatfield.py:51(_textual_read)
     6046    0.002    0.000    0.022    0.000 /usr/lib64/python3.14/unittest/mock.py:1178(_mock_call)
     6046    0.005    0.000    0.020    0.000 /usr/lib64/python3.14/unittest/mock.py:1227(_execute_mock_call)
   216001    0.015    0.000    0.015    0.000 {method 'replace' of 'str' objects}
     6002    0.009    0.000    0.014    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:38(values)
6093/6086    0.008    0.000    0.013    0.000 /usr/lib64/python3.14/unittest/mock.py:691(__getattr__)
141353/141278    0.012    0.000    0.013    0.000 {built-in method builtins.isinstance}
```

### Patamar REE (Earmfp)

```
384416 function calls (384042 primitive calls) in 0.096 seconds

   Ordered by: cumulative time
   List reduced from 769 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.096    0.096 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.094    0.094 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.094    0.094 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.000    0.000    0.094    0.094 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
        1    0.002    0.002    0.093    0.093 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:60(read)
        1    0.004    0.004    0.040    0.040 /home/rogerio/git/cfinterface/cfinterface/components/tabular.py:58(parse_lines)
     2046    0.002    0.000    0.036    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
     2002    0.000    0.000    0.034    0.000 /home/rogerio/git/cfinterface/cfinterface/components/line.py:49(read)
     2002    0.001    0.000    0.034    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:66(read)
     2002    0.003    0.000    0.033    0.000 /home/rogerio/git/cfinterface/cfinterface/adapters/components/line/repository.py:54(__positional_reading)
     2046    0.010    0.000    0.027    0.000 /usr/lib64/python3.14/unittest/mock.py:1181(_increment_mock_call)
    26002    0.007    0.000    0.026    0.000 /home/rogerio/git/cfinterface/cfinterface/components/field.py:41(read)
    24000    0.011    0.000    0.016    0.000 /home/rogerio/git/cfinterface/cfinterface/components/floatfield.py:51(_textual_read)
    10227    0.006    0.000    0.008    0.000 /usr/lib64/python3.14/unittest/mock.py:2572(__new__)
        1    0.000    0.000    0.007    0.007 /home/rogerio/git/inewave/inewave/nwlistop/modelos/blocos/tabela_serie_anual.py:90(_build_dataframe)
     2046    0.001    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1178(_mock_call)
     2046    0.002    0.000    0.007    0.000 /usr/lib64/python3.14/unittest/mock.py:1227(_execute_mock_call)
    72001    0.005    0.000    0.005    0.000 {method 'replace' of 'str' objects}
2093/2086    0.003    0.000    0.005    0.000 /usr/lib64/python3.14/unittest/mock.py:691(__getattr__)
47424/47351    0.004    0.000    0.005    0.000 {built-in method builtins.isinstance}
```

### Newave SectionFile (Pmo)

```
2275724 function calls (2262915 primitive calls) in 0.753 seconds

   Ordered by: cumulative time
   List reduced from 988 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.753    0.753 /home/rogerio/git/inewave/benchmarks/profile_read.py:222(profiled_read)
        1    0.000    0.000    0.751    0.751 /home/rogerio/git/cfinterface/cfinterface/files/blockfile.py:47(read)
        1    0.000    0.000    0.751    0.751 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:66(read)
        1    0.001    0.001    0.751    0.751 /home/rogerio/git/cfinterface/cfinterface/reading/blockreading.py:54(__read_file)
       17    0.002    0.000    0.226    0.013 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:504(read)
       16    0.002    0.000    0.222    0.014 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:420(read)
       16    0.000    0.000    0.192    0.012 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:505(converte_tabela_em_df)
        1    0.001    0.001    0.192    0.192 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:1093(read)
       15    0.000    0.000    0.184    0.012 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:421(converte_tabela_em_df)
       31    0.000    0.000    0.172    0.006 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/frame.py:12184(apply)
       31    0.000    0.000    0.172    0.006 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/apply.py:963(apply)
       31    0.000    0.000    0.172    0.006 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/apply.py:1165(apply_standard)
       31    0.009    0.000    0.161    0.005 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/apply.py:1174(apply_series_generator)
     7423    0.007    0.000    0.143    0.000 /usr/lib64/python3.14/unittest/mock.py:1170(__call__)
       31    0.000    0.000    0.119    0.004 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/frame.py:11567(melt)
       31    0.002    0.000    0.119    0.004 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/reshape/melt.py:44(melt)
     7423    0.038    0.000    0.107    0.000 /usr/lib64/python3.14/unittest/mock.py:1181(_increment_mock_call)
    30720    0.023    0.000    0.101    0.000 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/series.py:945(__getitem__)
        8    0.001    0.000    0.097    0.012 /home/rogerio/git/inewave/inewave/newave/modelos/pmo.py:1054(__fecha_configuracao)
      112    0.000    0.000    0.083    0.001 /home/rogerio/.local/lib/python3.14/site-packages/pandas/core/indexing.py:917(__setitem__)
```

## Key Findings

1. `Pmo` read time (236.5 ms) is dominated by `DataFrame.apply` + `DataFrame.melt` calls inside `converte_tabela_em_df` (cProfile shows ~171 ms / 31 calls for apply+melt vs ~126 ms for mock readline). The newave block-file path uses a different DataFrame construction strategy than the nwlistop TabularParser path.
2. `TabularParser.parse_lines` accounts for 31.6% of read time in `Patamar Submercado (Cmarg)` (24.7 ms out of 78.1 ms). The hot path is `Line.read` -> `__positional_reading` -> `FloatField._textual_read` called once per data cell.
3. `_build_dataframe` + `formata_df_meses_para_datas_nwlistop` accounts for 21.6% of read time in `Non-patamar SIN (Earmfsin)` (4.9 ms). Internally, `np.tile` / `np.repeat` / `np.concatenate` are called once per year-block to construct the tidy DataFrame.

## Recommendations

1. **Parsing bottleneck (ticket 020):** `TabularParser.parse_lines` iterates over every raw line and calls `Line.read()` once per line via Python-level field extraction. Consider batching field reads with `numpy.frompyfunc` or replacing the per-line Python loop with a vectorised `numpy.genfromtxt` / `pandas.read_fwf` approach on the collected raw text.
2. **DataFrame construction bottleneck (ticket 020/021):** `_build_dataframe` and `formata_df_meses_para_datas_nwlistop` perform repeated `np.tile` / `np.repeat` / `np.concatenate` calls for every year-block. Pre-allocating a single output array across all year-blocks (known from `ANO:` header scanning) before populating it would eliminate repeated intermediate allocations.
3. **Aggregation bottleneck (ticket 021):** `__monta_tabela` in `ArquivoREE` / `ArquivoSubmercado` / etc. calls `pd.concat([df, b.data], ignore_index=True)` in a loop, which allocates a new DataFrame on every iteration (O(n^2) copies). Replace with `pd.concat(list_of_all_blocks)` called once after collecting all block DataFrames.
