simplebench.decorators module🔗

Decorators for simplifying benchmark case creation.

simplebench.decorators.benchmark(
group: str | Callable[[...], Any] = 'default',
/,
*,
title: str | None = None,
description: str | None = None,
iterations: int = 20,
warmup_iterations: int = 10,
rounds: int = 1,
min_time: float = 5.0,
max_time: float = 20.0,
variation_cols: dict[str, str] | None = None,
kwargs_variations: dict[str, list[Any]] | None = None,
options: list[ReporterOptions] | None = None,
n: int = 1,
use_field_for_n: str | None = None,
) → Callable[[Callable[[P], R]], Callable[[P], R]][source]🔗

A decorator to register a function as a benchmark case.

This module uses a global registry to store benchmark cases created via the @benchmark decorator. This enables a streamlined workflow where users simply decorate functions and call main().

Note

Importing a module that uses @benchmark will register its cases globally. For testing, use clear_registered_cases() to reset state between tests.

This simplifies creating a Case by wrapping the decorated function. The decorated function should contain the code to be benchmarked.

It is important to note that the decorated function will be called within the context of a SimpleRunner.run() call, which means it should not handle its own timing or iterations.

The args provided to the decorator are used to create a Case instance, which is then added to a global registry. The original function is returned unmodified, allowing it to be called directly if needed.

The arguments to the decorator are largely the same as those for Case, with the exception of action, which is replaced by the decorated function.

n is included to allow n-weighting the complexity of the benchmark case when using runners that support it.

A minimal example:

from simplebench import benchmark, main


@benchmark
def addition_benchmark():
    '''A simple addition benchmark.'''
    sum(range(1000))


if __name__ == '__main__':
    extra_args = None if len(sys.argv) > 1 else ['--progress', '--rich-table.console']
main(extra_args=extra_args)

You should read the documentation for Case for full details on the parameters and their meanings.

Parameters:

  • group (str, positional-only) – The benchmark reporting group to which the benchmark case belongs for selection and reporting purposes. It is used to categorize and filter benchmark cases. Cannot be blank. The group parameter is positional-only. All other parameters must be passed as keyword arguments. When the decorator is used without parameters, the group defaults to ‘default’. This has special handling to allow the decorator to be used easily without any parameters.

  • title (Optional[str]) – The title of the benchmark case. Uses the function name if None. Cannot be blank.

  • description (Optional[str]) – A description for the case. Uses the function’s docstring if None or ‘(no description)’ if there is no docstring. Cannot be blank.

  • iterations (int) – The minimum number of iterations to run for the benchmark.

  • warmup_iterations (int) – The number of warmup iterations to run before the benchmark.

  • rounds (int) – The number of rounds to run the benchmark within each iteration.

  • min_time (int | float) – The minimum time in seconds to run the benchmark. Must be a positive number.

  • max_time (int | float) – The maximum time in seconds to run the benchmark. Must be a positive number greater than min_time.

  • variation_cols (Optional[dict[str, str]]) – kwargs to be used for cols to denote kwarg variations. Each key is a keyword argument name, and the value is the column label to use for that argument. Only keywords that are also in kwargs_variations can be used here. These fields will be added to the output of reporters that support them as columns of data with the specified labels. If None, an empty dict is used.

  • kwargs_variations (Optional[dict[str, list[Any]]]) – A mapping of keyword argument key names to a list of possible values for that argument. Default is {}. When tests are run, the benchmark will be executed for each combination of the specified keyword argument variations. The action function will be called with a bench parameter that is an instance of the runner and the keyword arguments for the current variation. If None, an empty dict is used.

  • options (Optional[list[ReporterOptions]]) – A list of additional options for the benchmark case. Each option is an instance of ReporterOptions or a subclass of ReporterOptions. Reporter options can be used to customize the output of the benchmark reports for specific reporters. Reporters are responsible for extracting applicable ReporterOptionss from the list of options themselves.

  • n (int) – The ‘n’ weighting of the benchmark case. Must be a positive integer.

  • use_field_for_n (Optional[str]) – If provided, use the value of this field from kwargs_variations to set ‘n’ dynamically for each variation.

Returns:

A decorator that registers the function for benchmarking and returns it unmodified.

Return type:

Callable[[Callable[P, R]], Callable[P, R]]

Raises:
simplebench.decorators.clear_registered_cases() → None[source]🔗

Clear all benchmark cases registered via the @benchmark decorator.

This can be useful in testing scenarios to reset the state.

simplebench.decorators.get_registered_cases() → list[Case][source]🔗

Retrieve all benchmark cases registered via the @benchmark decorator.

Returns:

A list of Case objects.

Return type:

list[Case]