simplebench.reporters.reporter_manager package🔗

Reporter manager for benchmark results.

class simplebench.reporters.reporter_manager.ReporterManager(load_defaults: bool = True)[source]🔗

Bases: object

Manager for all available Reporter classes.

This class maintains a registry of Reporter instances and their associated CLI arguments. It allows for the registration, retrieval, and unregistration of Reporter instances.

Initially, it registers a set of built-in Reporter instances: CSVReporter, ScatterPlotReporter, and RichTableReporter.

New Reporter instances can be added using the register() method, and existing ones can be removed using the unregister() or unregister_by_name() methods.

Reporter instances are required to have unique names and CLI arguments in their Choices to avoid conflicts and the manager will raise exceptions if duplicates are detected during registration.

reporter_manager = ReporterManager()
my_custom_reporter = CustomReporter()
reporter_manager.register(my_custom_reporter)
add_reporters_to_argparse(
parser: ArgumentParser,
) → None[source]🔗

Add all registered reporter choices to an ArgumentParser.

This method adds the CLI arguments for all registered reporters to the provided ArgumentParser instance using the add_flags_to_argparse() method of each Reporter.

Parameters:

parser (ArgumentParser) – The ArgumentParser to add the flags to.

all_reporters() → dict[str, Reporter][source]🔗

Return all available Reporter instances as a dictionary keyed by their names.

Returns:

A dictionary of all Reporter instances with their names as keys.

Return type:

dict[str, Reporter]

choice_for_arg(
arg: str,
) → Choice | None[source]🔗

Return the Choice instance for a given CLI argument.

Parameters:

arg (str) – The CLI argument to look up.

Returns:

The corresponding Choice instance, or None if not found.

Return type:

Choice | None

property choices: Choices🔗

Return a Choices instance containing all registered reporter choices.

Returns:

A Choices instance with all registered reporter choices.

Return type:

Choices

register(
reporter: Reporter,
) → None[source]🔗

Register a new Reporter.

This method adds a new Reporter to the registry. Unlike the predefined reporters and the register_reporter() decorator which both register by class, this method requires an instance of the Reporter.

This allows for more flexibility, such as registering Reporter instances with custom initialization parameters.

Parameters:

reporter (Reporter) – The Reporter to register.

Raises:
unregister(
reporter: Reporter,
) → None[source]🔗

Unregister a Reporter by an instance exemplar.

It looks up the reporter by its name and removes it from the registry.

reporter_manager.unregister(MyCustomReporter())
Parameters:

reporter (Reporter) – Reporter instance exemplar to unregister.

Raises:
unregister_all() → None[source]🔗

Unregister all Reporter instances.

This clears the entire registry of Reporter instances.

unregister_by_name(
name: str,
) → None[source]🔗

Unregister a Reporter by its name.

Parameters:

name (str) – The name of the Reporter to unregister.

Raises:

SimpleBenchKeyError – If no reporter with the given name is registered.

Subpackages🔗

Submodules🔗