Metadata-Version: 2.4
Name: scope-profiler
Version: 0.2.1
Summary: Profile code regions in python, optionally with LIKWID markers.
Author: Max
Project-URL: Source, https://github.com/max-models/scope-profiler
Keywords: python
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: h5py
Requires-Dist: numpy
Requires-Dist: line-profiler
Provides-Extra: mpi
Requires-Dist: mpi4py; extra == "mpi"
Provides-Extra: pproc
Requires-Dist: ipykernel; extra == "pproc"
Requires-Dist: jupyterlab; extra == "pproc"
Requires-Dist: matplotlib; extra == "pproc"
Provides-Extra: dev
Requires-Dist: black[jupyter]; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: scope-profiler[docs,mpi,pproc,test]; extra == "dev"
Provides-Extra: docs
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: nbconvert; extra == "docs"
Requires-Dist: nbsphinx; extra == "docs"
Requires-Dist: pre-commit; extra == "docs"
Requires-Dist: pyproject-fmt; extra == "docs"
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: sphinx-book-theme; extra == "docs"
Requires-Dist: scope-profiler[pproc]; extra == "docs"
Provides-Extra: test
Requires-Dist: coverage; extra == "test"
Requires-Dist: pytest; extra == "test"

# scope-profiler

This module provides a unified profiling system for Python applications, with optional integration of [LIKWID](https://github.com/RRZE-HPC/likwid) markers using the [pylikwid](https://github.com/RRZE-HPC/pylikwid) marker API for hardware performance counters.

It allows you to:

- Configure profiling globally via a singleton ProfilingConfig.
- Collect timing data via context-managed profiling regions.
- Use a clean decorator syntax to profile functions.
- Optionally record time traces in HDF5 files.
- Automatically initialize and close LIKWID markers only when needed.
- Print aggregated summaries of all profiling regions.

## Install

Install from [PyPI](https://pypi.org/project/scope-profiler/):

```
pip install scope-profiler
```

## Usage

To set up the configuration, create an instance of `ProfilingConfig` and add it to the `ProfileManager`, this should be done once at application startup and will persist until the program exits or is explicitly finalized (see below). Note that the config applies to any profiling contexts created (even in other files) after it has been initialized.

```python
from scope_profiler import ProfileManager

# Setup global profiling configuration
ProfileManager.setup(
    use_likwid=False,
    recursive_profile=False,
    time_trace=True,
    flush_to_disk=True,
)

# Profile the main() function with a decorator
@ProfileManager.profile("main")
def main():
    x = 0
    for i in range(10):
        # Profile each iteration with a context manager
        with ProfileManager.profile_region(region_name="iteration"):
            x += 1

# Call main
main()

# Finalize profiler
ProfileManager.finalize()
```

Execution:

```bash
❯ python test.py
Region: main
  Total Calls : 1
  Total Time  : 0.001503709 s
  Avg Time    : 0.001503709 s
  Min Time    : 0.001503709 s
  Max Time    : 0.001503709 s
  Std Dev     : 0.0 s
----------------------------------------
Region: iteration
  Total Calls : 10
  Total Time  : 3.832e-06 s
  Avg Time    : 3.832e-07 s
  Min Time    : 2.08e-07 s
  Max Time    : 8.75e-07 s
  Std Dev     : 2.2431888016838885e-07 s
----------------------------------------
```

## Example plots

`scope-profiler pproc` turns an HDF5 profiling file into Gantt, flame,
duration, and speedup charts (see [Flame graphs](#flame-graphs) below for
details). The plots here come from `examples/generate_readme_figures.py`, a
small mock timestep loop with nested and self-recursive regions, and are
saved to `figures/`:

```bash
python examples/generate_readme_figures.py
```

![Gantt chart of a mock timestep loop](https://raw.githubusercontent.com/max-models/scope-profiler/refs/heads/devel/figures/gantt_plot.png)

![Average duration per region](https://raw.githubusercontent.com/max-models/scope-profiler/refs/heads/devel/figures/durations_plot.png)

The flame graph for the same run is shown in [Flame graphs](#flame-graphs) below.

## Overhead

The profiling overhead per call depends on the region type.
The benchmark below (`examples/benchmark_overhead.py`) measures each mode
against a bare function call:

![Profiling overhead by region type](https://raw.githubusercontent.com/max-models/scope-profiler/refs/heads/devel/figures/benchmark_overhead.png)

The two modes most relevant to HPC — **NCallsOnly** and **TimeOnly** — add
roughly **0.09 µs** and **0.75 µs** per instrumented call respectively.

Profiling can also be fully deactivated at setup time
(`profiling_activated=False`) to reduce the overhead to ~0.03 µs — barely
above a bare function call — making it safe to leave instrumentation in
production code and toggle it on only when needed.

The **LineProfiler** mode is intentionally heavier (~41 µs/call) because
`line_profiler` traces every source line. It is designed for targeted
debugging of individual functions, not for always-on use in hot loops.

## Recursive profiling of nested calls

You can profile nested Python calls from one decorated entrypoint:

```python
from scope_profiler import ProfileManager

ProfileManager.setup(recursive_profile=True)


def leaf(x):
    return x + 1


def inner(x):
    return leaf(x) * 2


@ProfileManager.profile("entry")
def entry():
    return sum(inner(i) for i in range(3))


entry()
ProfileManager.finalize()
```

When enabled, the profiler records regions for nested calls using fully
qualified names (for example, `my_module.inner`), in addition to the main
decorated region.

## Zero-instrumentation CLI profiling

You can profile a whole script without touching its source, similar to
`python -m cProfile`:

```bash
scope-profiler run my_script.py [script args...]
# equivalently: python -m scope_profiler run my_script.py [script args...]
```

Every Python function call the script makes is recorded as its own region
under a name derived from its module and qualified name, using the same
recursive tracer as `recursive_profile=True` above. By default only the
script's own code is instrumented (the standard library and installed
packages are skipped) to keep overhead low; pass `--all` to trace
everything. Results are written to `profiling_data.h5` by default
(`-o`/`--outfile` to change it), and a per-region summary is printed unless
`-q`/`--quiet` is given.

See `examples/ex_cli_profiling.py` for a script with no scope-profiler
imports at all, run with:

```bash
scope-profiler run examples/ex_cli_profiling.py
```

## Profiling self-recursive functions

A single region can also be safely re-entered by a recursive function -
each call gets its own slot in the region's buffer, so nested calls don't
overwrite each other's timing data. This works with both the decorator and
context-manager forms:

```python
from scope_profiler import ProfileManager

ProfileManager.setup()


@ProfileManager.profile("fibonacci")
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)


def fibonacci_context_manager(n):
    with ProfileManager.profile_region("fibonacci_ctx"):
        if n < 2:
            return n
        return fibonacci_context_manager(n - 1) + fibonacci_context_manager(n - 2)


fibonacci(10)
fibonacci_context_manager(10)
ProfileManager.finalize()
```

Both `fibonacci` and `fibonacci_ctx` will report one call per recursive
invocation, each with correct, non-overlapping timing data.

## Flame graphs

Because each call - including recursive re-entries of the same region -
now has its own correctly nested (start, end) interval, the call stack can
be reconstructed straight from the timing data and rendered as a flame
graph, with recursion showing up as a narrowing tower of frames - as with
`refine_mesh` below, from the same run shown in [Example plots](#example-plots):

![Flame graph of a mock timestep loop](https://raw.githubusercontent.com/max-models/scope-profiler/refs/heads/devel/figures/flame_plot.png)

`scope-profiler pproc` generates `flame_plot.png` alongside the Gantt chart
for every run:

```bash
scope-profiler pproc profiling_data.h5 --show -o figures
```

Or programmatically:

```python
from scope_profiler.h5reader import ProfilingH5Reader
from scope_profiler.plotting_scripts import plot_flame

reader = ProfilingH5Reader("profiling_data.h5")
plot_flame(reader, filepath="flame_plot.png")
```

Gantt and flame charts (and `plot_speedup`) always color the same region the
same way. Pass `--cmap` (or `cmap=` on the `plot_*` functions) to use a
different [matplotlib colormap](https://matplotlib.org/stable/users/explain/colors/colormaps.html)
than the default `tab20`:

```bash
scope-profiler pproc profiling_data.h5 --cmap viridis -o figures
```

By default the flame graph covers rank 0, since it represents a single
execution's call stack; pass `ranks=[...]` to render one flame graph per
requested rank.

## Exporting plot data

Every `plot_*` function accepts a `data_filepath` argument that writes the
exact data behind the chart to a file, so it can be re-parsed and re-plotted
later without the original HDF5 file. `data_format` selects `"csv"` (default)
or `"json"`:

```python
plot_gantt(reader, filepath="gantt_plot.png", data_filepath="gantt_data.csv")
plot_gantt(
    reader,
    filepath="gantt_plot.png",
    data_filepath="gantt_data.json",
    data_format="json",
)
```

The JSON payload additionally includes a `colors` map (region or file label
to `#rrggbb`) matching the colors used in the matplotlib plot, so a
JavaScript charting library like Plotly can reproduce the same look.

`scope-profiler pproc --export-data` does the same for every plot in one
run, writing `gantt_data`, `flame_data`, `durations_data`, and (for multiple
input files) `speedup_data` alongside the PNGs. Pass `--export-data-format
json` to get `.json` files instead of the default `.csv`:

```bash
scope-profiler pproc profiling_data.h5 -o figures --export-data
scope-profiler pproc profiling_data.h5 -o figures --export-data --export-data-format json
```

Pass `--skip-plot-images` (requires `--export-data`) to skip rendering the
PNGs entirely and only write the exported data plus `region_statistics.json`
— useful when a website renders charts client-side (e.g. with Plotly)
straight from the JSON:

```bash
scope-profiler pproc profiling_data.h5 -o figures \
  --export-data --export-data-format json --skip-plot-images
```
