API reference¶

Experiment¶

class experimentator.Experiment(tree, filename=None)¶

Bases: experimentator.section.ExperimentSection

An ExperimentSection subclass that represents the largest ‘section’ of the experiment; that is, the entire experiment. Functionality added on top of ExperimentSection includes various constructors, saving to disk, and management of callbacks.

Parameters:	tree : DesignTree A DesignTree instance defining the experiment hierarchy. filename : str, optional A file location where the Experiment will be saved.

Attributes

tree	(DesignTree) The DesignTree instance defining the experiment’s hierarchy.
filename	(str) The file location where the Experiment will be pickled.
run_callback	(func) The function to be run when the bottom of the tree is reached (i.e., the trial function).
context_managers	(dict) A dictionary, mapping level names to With Statement Context Managers (e.g., generator functions decorated with contextlib.contextmanager). Defines behavior to run before and/or after each section at the associated level.
session_data	(dict) A dictionary where temporary data can be stored, persistent only within one session of the Python interpreter. This is a good place to store external resources that aren’t picklable; external resources, for example, can be loaded in a context-manager callback and stored here. In addition, anything returned by the __exit__ method of a context-manager callback will be stored here, with the callback’s level name as the key. This dictionary is emptied before saving the Experiment to disk.
experiment_data	(dict) A dictionary where data can be stored that is persistent across Python sessions. Everything stored here must be picklable.

Methods

add_data
all_subsections
append_child
append_design_tree
basic
blocked
breadth_first_search
depth_first_search
export_data
find_first_not_run
find_first_partially_run
from_dict
from_yaml_file
get_next_tree
load
parent
parents
resume_section
run_section
save
set_context_manager
set_run_callback
subsection
walk
within_subjects

add_data(data)¶

Update the ExperimentSection.data ChainMap. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.

Parameters:	data : dict Elements to be added to ExperimentSection.data.

all_subsections(**section_numbers)¶

Find all subsections in the experiment matching the given section numbers.

Yields specified ExperimentSection instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.

Parameters:	**section_numbers Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.

See also

ExperimentSection.subsection

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'):

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')

Get the first session of each participant:

>>> all_first_sessions = list(exp.all_subsections(session=1))

Get the second trial of the first block in each session:

>>> trials = list(exp.all_subsections(block=1, trial=2))

Get the first three trials of each block in the first session of each participant:

>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))

append_child(data, tree=None, to_start=False, _renumber=True)¶

Create a new ExperimentSection (and its descendants) and append it as a child of the current ExperimentSection.

Parameters:

data : dict Data to be included in the new section’s ExperimentSection.data ChainMap. Should include values of IVs at the section’s level, for example. tree : DesignTree, optional If given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data. to_start : bool, optional If True, the new ExperimentSection will be appended to the beginning of the current section. If False (the default), it will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

append_design_tree(tree, to_start=False, _renumber=True)¶

Append all sections associated with the top level of a DesignTree (and therefore also create descendant sections) to the ExperimentSection.

Parameters:	tree : DesignTree The tree to append. to_start : bool, optional If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

classmethod basic(levels, ivs_by_level, design_matrices_by_level=None, ordering_by_level=None, filename=None)¶

Construct a homogeneously-organized Experiment, with arbitrary levels but only one Design at each level, and the same structure throughout its hierarchy.

Parameters:

levels : sequence of str Names of the levels of the experiment ivs_by_level : dict Dictionary specifying the IVs and their possible values at every level. The keys are be the level names, and the values are lists of the IVs at that level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See IV docs for more on specifying IVs. design_matrices_by_level : dict, optional Specify the design matrix for any levels. Keys are level names; values are design matrices. Any levels without a design matrix will be fully crossed. See design matrix docs for details. ordering_by_level : dict, optional Specify the ordering for each level. Keys are level names; values are instance objects from experimentator.order. For any levels without an order specified, Shuffle will be used. filename : str, optional File location to save the experiment.

Returns:

Experiment

classmethod blocked(trial_ivs, n_participants, design_matrices=None, orderings=None, block_ivs=None, filename=None)¶

Create a blocked within-subjects Experiment, in which all the IVs are at either the trial level or the block level.

Parameters:

trial_ivs : list or dict A list of the IVs to define at the trial level, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs. n_participants : int Number of participants to initialize. If a NonAtomicOrdering is used, this is the number of participants per order. design_matrices : dict, optional Design matrices for the experiment. Keys are 'trial' and 'block'; values are the respective design matrices (if any). If not specified, IVs will be fully crossed. See the design matrix docs for details. orderings : dict, optional Dictionary with keys of 'trial' and 'block'. Each value should be an instance of the class Ordering or one of its subclasses, specifying how the trials will be ordered If not specified, Shuffle will be used. block_ivs : list or dict, optional IVs to define at the block level. See IV docs for more on specifying IVs. filename : str, optional File location to save the experiment.

Returns:

Experiment

Notes

For blocks to have any effect, you should either define at least one IV at the block level or use the ordering Ordering(n) to create n blocks for every participant.

breadth_first_search(key)¶

Breadth-first search starting from here. Returns the entire search path.

Parameters:	key : func Function that returns True or False when passed an ExperimentSection.
Returns:	list of ExperimentSection

depth_first_search(key, path_key=None, _path=None)¶

Depth-first search starting from here. Returns the entire search path.

Parameters:	key : func Function that returns True or False when passed an ExperimentSection. path_key : func, optional Function that returns True or False when passed an ExperimentSection. If given, the search will proceed only via sections for which path_key returns True.
Returns:	list of ExperimentSection

export_data(filename, skip_columns=None, **kwargs)¶

Export Experiment.dataframe in .csv format.

Parameters:	filename : str A file location where the data should be saved. skip_columns : list of str, optional Columns to skip. **kwargs Arbitrary keyword arguments to pass to pandas.DataFrame.to_csv.

Notes

This method is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In those cases it is recommended to write a custom script that parses the Experiment.dataframe attribute as desired, or use the skip_columns option to skip any compound columns.

find_first_not_run(at_level, by_started=True)¶

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has not yet been run.

Parameters:	at_level : str Which level to search. by_started : bool, optional If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:	ExperimentSection

find_first_partially_run(at_level)¶

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has been started but not finished.

Parameters:	at_level : str Which level to search.
Returns:	ExperimentSection

classmethod from_dict(spec)¶

Construct an Experiment based on a dictionary specification.

Parameters:	spec : dict spec should have, at minimum, a key named 'design'. The value of this key specifies the DesignTree. See DesignTree.from_spec for details. The value of the key 'filename' or 'file', if one exists,is saved in Experiment.filename. All other fields are saved in Experiment.experiment_data.
Returns:	Experiment

See also

Experiment.from_yaml_file

classmethod from_yaml_file(filename)¶

Construct an Experiment based on specification in a YAML file. Requires PyYAML.

Parameters:	filename : str YAML file location. The YAML should specify a dictionary matching the specification of Experiment.from_dict.
Returns:	Experiment

get_next_tree()¶

Get a tree to use for creating child ExperimentSection instances.

Returns:	DesignTree

static load(filename)¶

Load an experiment from disk.

Parameters:	filename : str Path to a file generated by Experiment.save.
Returns:	Experiment

parent(section)¶

Find the parent of a section.

Parameters:	section : ExperimentSection The section to find the parent of.
Returns:	ExperimentSection

parents(section)¶

Find all parents of a section, in top-to-bottom order.

Parameters:	section : ExperimentSection The section to find the parents of.
Returns:	list of ExperimentSection

resume_section(section, **kwargs)¶

Rerun a section that has been started but not finished, starting where running last left off.

Parameters:	section : ExperimentSection The section to resume. **kwargs Keyword arguments to pass to Experiment.run_section.

Notes

The wrapper function run_experiment_section should be used instead of this method, if possible.

run_section(section, demo=False, parent_callbacks=True, from_section=None)¶

Run a section and all its descendant sections. Saves the results in the data attribute of each lowest-level ExperimentSection.

Parameters:

section : ExperimentSection The section to be run. demo : bool, optional Data will only be saved if demo is False (the default). parent_callbacks : bool, optional If True (the default), all parent callbacks will be called. from_section : int or list of int, optional Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. For example, assuming the experiment hierarchy is ('participant', 'session', 'block', 'trial'), this would start from the fifth trial of the second block (of the first participant’s second session): >>> exp = Experiment.load('example.exp') >>> exp.run_section(exp.subsection(participant=1, session=2), from_section=[2, 5])

Notes

The wrapper function run_experiment_section should be used instead of this method, if possible.

save(filename=None)¶

Save the Experiment to disk.

Parameters:	filename : str, optional If specified, overrides Experiment.filename.

set_context_manager(level, manager, *args, func_module=None, func_name=None, **kwargs)¶

Set a context manager to run at a certain level.

Define a context-manager manager to run at every section at a particular level. A context manager is a convenient way to define behavior to occur at the beginning and end of every section at a particular level. See contextlib for various ways to create context managers, and experimentator’s context-manager docs for more details.

Any value returned by the __exit__ method of manager (equivalently, yielded by a function decorated with contextlib.contextmanager) will be stored in Experiment.session_data under the key level.

Parameters:

level : str Which level of the hierarchy to manage. func : func The context manager function. func should have the signature func(experiment, section, *args, **kwargs) where experiment and section are the current Experiment and ExperimentSection instances, respectively, and args and kwargs are arbitrary arguments passed to this method. *args Any arbitrary positional arguments to be passed to func. func_module : str, optional func_name : str, optional These two arguments specify where the given function should be imported from in future Python sessions (i.e., from <func_module> import <func_name>). Usually, this is figured out automatically by introspection; these arguments are provided for the rare situation where introspection fails. **kwargs Any arbitrary keyword arguments to be passed to func.

set_run_callback(func, *args, func_module=None, func_name=None, **kwargs)¶

Define a function to run at the lowest level of the experiment (i.e., the trial function). See the callback docs for more details.

Parameters:

func : func The function to be set as the run callback. func should have the signature func(experiment, section, *args, **kwargs) where experiment and section are the current Experiment and ExperimentSection instances, respectively, and args and kwargs are arbitrary arguments passed to this method. *args Any arbitrary positional arguments to be passed to func. func_module : str, optional func_name : str, optional These two arguments specify where the given function should be imported from in future Python sessions (i.e., from <func_module> import <func_name>). Usually, this is figured out automatically by introspection; these arguments are provided for the rare situation where introspection fails. **kwargs Any arbitrary keyword arguments to be passed to func.

subsection(**section_numbers)¶

Find a single, descendant ExperimentSection based on section numbers.

Parameters:	**section_numbers Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns:	ExperimentSection

See also

ExperimentSection.all_subsections

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'), this will return the third block of the second participant’s first session:

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')
>>> some_block = exp.subsection(participant=2, session=1, block=3)

walk()¶

Walk the tree depth-first, starting from here. Yields this section and every descendant section.

classmethod within_subjects(ivs, n_participants, design_matrix=None, ordering=None, filename=None)¶

Create a within-subjects Experiment, with all the IVs at the 'trial' level.

Parameters:

ivs : list or dict A list of the experiment’s IVs, specified in the form of tuples with the first element being the IV name and the second element a list of its possible values. Alternatively, the IVs at each level can be specified in a dictionary. See the IV docs more on specifying IVs. n_participants : int Number of participants to initialize. design_matrix : array-like, optional Design matrix for the experiment. If not specified, IVs will be fully crossed. See the design matrix docs for more details. ordering : Ordering, optional An instance of the class Ordering or one of its subclasses, specifying how the trials will be ordered. If not specified, Shuffle will be used. filename : str, optional File location to save the experiment.

Returns:

Experiment

Helper functions¶

experimentator.run_experiment_section(experiment, section_obj=None, demo=False, resume=False, parent_callbacks=True, from_section=1, session_options='', **section_numbers)¶

Run an experiment from a file or an Experiment instance, and save it. If an exception is encountered, the Experiment will be backed up and saved.

Parameters:

experiment : str or Experiment File location where an Experiment instance is pickled, or an Experiment instance. demo : bool, optional If True, data will not be saved and sections will not be marked as run. resume: bool, optional If True, the specified section will be resumed (started automatically where it left off). parent_callbacks : bool, optional If True (the default), all parent callbacks will be called. section_obj : ExperimentSection, optional The section of the experiment to run. Alternatively, the section can be specified using **section_numbers. from_section : int or list of int, optional Which section to start running from. If a list is passed, it specifies where to start running on multiple levels. See the example below. session_options : str, optional Pass an experiment-specific options string to be stored in Experiment.session_data under the key 'options'. **section_numbers Keyword arguments describing how to descend the experiment hierarchy to find the section to run. See the example below.

Examples

A simple example:

>>> exp = Experiment.load('example.exp')
>>> run_experiment_section(exp, exp.subsection(participant=1, session=2))

Equivalently:

>>> run_experiment_section(exp, participant=1, session=2)

To demonstrate from_section, assuming the experiment hierarchy is ('participant', 'session', 'block', 'trial'), this would start from the second block:

>>> run_experiment_section(exp, participant=1, session=2, from_section=2)

To start from the fifth trial of the second block:

>>> run_experiment_section(exp, participant=1, session=2, from_section=[2, 5])

experimentator.export_experiment_data(exp_filename, data_filename, **kwargs)¶

Reads a pickled Experiment instance and saves its data in .csv format.

Parameters:	exp_filename : str The file location where an Experiment instance is pickled. data_filename : str The file location where the data will be written. skip_columns : list of str, optional Data columns to skip. **kwargs Arbitrary keyword arguments passed through to pandas.DataFrame.to_csv.

Notes

This shortcut function is not recommended for experiments with compound data types, for example an experiment which stores a time series for every trial. In such cases it is recommended to write a custom script that parses Experiment.dataframe as desired (or use the skip_columns option to ignore the compound data).

ExperimentSection¶

class experimentator.section.ExperimentSection(tree, data)[source]¶

A section of the experiment, at any level of the hierarchy. Single trials and groups of trials (blocks, sessions, participants, etc.) are represented as ExperimentSection instances. A complete experiment consists of ExperimentSection instances arranged in a tree. The root element should be an Experiment (a subclass of ExperimentSection); the rest of the sections can be reached via its descendants (see below on the sequence protocol). A new ExperimentSection instance is automatically populated with ExperimentSection descendants according to the DesignTree passed to its constructor.

ExperimentSection implements Python’s sequence protocol; its contents are ExperimentSection instances at the level below. In other words, children can be accessed using the [index] notation, as well as with slices ([3:6]) or iteration (for section in experiment_section). However, ExperimentSection breaks the Python convention of 0-based indexing, using 1-based indexing to match the convention in experimental science.

Parameters:

tree : DesignTree Describes the design of the experiment hierarchy. data : ChainMap All data to be associated with the ExperimentSection, including the values of independent variables, the section numbers indicating the section’s location in the experiment, and any results associated with this section, arising from either the run callback of the Experiment or from the method ExperimentSection.add_data. data should be a collections.ChainMap, which behaves like a dictionary but has a hierarchical organization such that children can access values from the parent but not vice-versa.

Notes

Use 1-based indexing to refer to ExperimentSection children, both when when using indexing or slicing with an ExperimentSection, and when identifying sections in keyword arguments to methods such as ExperimentSection.subsection. This better corresponds to the language commonly used by scientists to identify participants, trials, etc.

Attributes

tree	(DesignTree)
data	(ChainMap)
dataframe	(DataFrame) All data associated with the ExperimentSection and its descendants.
heterogeneous_design_iv_name	(str) IV name determining which branch of the DesignTree to follow.
level	(str) The level of the hierarchy at which this section lives.
levels	(list of str) Level names below this section.
local_levels	(set) Level names of this section’s children. Usually a single-element set.
is_bottom_level	(bool) If true, this is the lowest level of the hierarchy.
has_start	(bool) Whether this section has started to be run.
has_finished	(bool) Whether this section has finished running.

Methods

add_data
all_subsections
append_child
append_design_tree
breadth_first_search
depth_first_search
find_first_not_run
find_first_partially_run
get_next_tree
parent
parents
subsection
walk

add_data(data)[source]¶

Update the ExperimentSection.data ChainMap. This data will apply to this section and all child sections. This can be used, for example, to manually record a participant’s age.

Parameters:	data : dict Elements to be added to ExperimentSection.data.

all_subsections(**section_numbers)[source]¶

Find all subsections in the experiment matching the given section numbers.

Yields specified ExperimentSection instances. The yielded sections will be at the lowest level given in section_numbers. If levels not in section_numbers are encountered before reaching its lowest level, all sections will be descended into.

Parameters:	**section_numbers Keyword arguments describing what subsections to find. Keys are level names, values are ints or sequences of ints.

See also

ExperimentSection.subsection

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'):

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')

Get the first session of each participant:

>>> all_first_sessions = list(exp.all_subsections(session=1))

Get the second trial of the first block in each session:

>>> trials = list(exp.all_subsections(block=1, trial=2))

Get the first three trials of each block in the first session of each participant:

>>> more_trials = list(exp.all_subsections(session=1, trial=[1, 2, 3]))

append_child(data, tree=None, to_start=False, _renumber=True)[source]¶

Create a new ExperimentSection (and its descendants) and append it as a child of the current ExperimentSection.

Parameters:

data : dict Data to be included in the new section’s ExperimentSection.data ChainMap. Should include values of IVs at the section’s level, for example. tree : DesignTree, optional If given, the section will be appended from the top level of tree. If not passed, the tree of the current section will be used. Note that this does not affect IV values; IV values must still be included in data. to_start : bool, optional If True, the new ExperimentSection will be appended to the beginning of the current section. If False (the default), it will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

append_design_tree(tree, to_start=False, _renumber=True)[source]¶

Append all sections associated with the top level of a DesignTree (and therefore also create descendant sections) to the ExperimentSection.

Parameters:	tree : DesignTree The tree to append. to_start : bool, optional If True, the sections will be inserted at the beginning of the section. If False (the default), they will be appended to the end.

Notes

After calling this method, the section numbers in the children’s ExperimentSection.data attributes will be automatically replaced with the correct numbers.

breadth_first_search(key)[source]¶

Breadth-first search starting from here. Returns the entire search path.

Parameters:	key : func Function that returns True or False when passed an ExperimentSection.
Returns:	list of ExperimentSection

depth_first_search(key, path_key=None, _path=None)[source]¶

Depth-first search starting from here. Returns the entire search path.

Parameters:	key : func Function that returns True or False when passed an ExperimentSection. path_key : func, optional Function that returns True or False when passed an ExperimentSection. If given, the search will proceed only via sections for which path_key returns True.
Returns:	list of ExperimentSection

find_first_not_run(at_level, by_started=True)[source]¶

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has not yet been run.

Parameters:	at_level : str Which level to search. by_started : bool, optional If True (default), returns the first section that has not been started. Otherwise, finds the first section that has not finished.
Returns:	ExperimentSection

find_first_partially_run(at_level)[source]¶

Search the experimental hierarchy, and return the first descendant ExperimentSection at at_level that has been started but not finished.

Parameters:	at_level : str Which level to search.
Returns:	ExperimentSection

get_next_tree()[source]¶

Get a tree to use for creating child ExperimentSection instances.

Returns:	DesignTree

parent(section)[source]¶

Find the parent of a section.

Parameters:	section : ExperimentSection The section to find the parent of.
Returns:	ExperimentSection

parents(section)[source]¶

Find all parents of a section, in top-to-bottom order.

Parameters:	section : ExperimentSection The section to find the parents of.
Returns:	list of ExperimentSection

subsection(**section_numbers)[source]¶

Find a single, descendant ExperimentSection based on section numbers.

Parameters:	**section_numbers Keyword arguments describing which subsection to find Must include every level higher than the desired section.
Returns:	ExperimentSection

See also

ExperimentSection.all_subsections

Examples

Assuming the levels of the experiment saved in 'example.exp' are ('participant', 'session', 'block', 'trial'), this will return the third block of the second participant’s first session:

>>> from experimentator import Experiment
>>> exp = Experiment.load('example.exp')
>>> some_block = exp.subsection(participant=2, session=1, block=3)

walk()[source]¶

Walk the tree depth-first, starting from here. Yields this section and every descendant section.

Design¶

class experimentator.Design(ivs=None, design_matrix=None, ordering=None, extra_data=None)¶

Design instances specify the experimental design at one level of the experimental hierarchy. They guide the creation of ExperimentSection instances by parsing design matrices or crossing independent variables (IVs).

Parameters:

ivs : dict or list of tuple, optional Independent variables can be specified in two ways: A dictionary mapping strings (IV names) to lists (levels of the IV). Note: if you use this format, you cannot rely on the order of the IVs (unless you use an OrderedDict). A list of (IV name, IV levels) tuples. If an IV takes continuous values, use None for its levels. This only works when specifying values using design_matrix. See the IV docs for more information. design_matrix : array-like, optional A numpy array (or convertible, e.g. a list-of-lists) representing a design matrix specifying how IV values should be grouped to form conditions. Design matrices generated by pyDOE are compatible. Each column represents one IV, and each row represents one condition. Values of elements in this matrix are interpreted in two ways: If the levels of the IV were passed as None in ivs, then the values in the array are interpreted “at face value”, i.e. as IV values. Otherwise, each element from the list of IV values is associated with a unique value in a column (in order of value). For example, if an IV has two levels (e.g. [True, False]) then the corresponding column of design_matrix should have two unique values. If they are 0 and 1, they are associated with True and False, respectively. Similarly, they could be 1 and 2, or any two non-equal values. Note that a design matrix also specifies the order of the conditions. For this reason, the default ordering changes from Shuffle to Ordering, preserving the order of the conditions. When no design_matrix is passed, IVs are fully crossed. See the design matrix docs for more details. ordering : Ordering, optional An instance of Ordering or one of its subclasses defining the behavior for duplicating and ordering the conditions of the Design. The default is Shuffle unless a design_matrix is passed. extra_data : dict, optional Items from this dictionary will be included in the data attribute of any ExperimentSection instances created with this Design.

See also

experimentator.order, DesignTree

Examples

>>> from experimentator.order import Shuffle
>>> design = Design(ivs={'side': ['left', 'right'], 'difficulty': ['easy', 'hard']}, ordering=Shuffle(2))
>>> design.first_pass()
IndependentValue(name=(), values=())
>>> design.get_order()
[{'difficulty': 'easy', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'left'},
 {'difficulty': 'easy', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'right'},
 {'difficulty': 'easy', 'side': 'right'},
 {'difficulty': 'easy', 'side': 'right'},
 {'difficulty': 'hard', 'side': 'left'},
 {'difficulty': 'hard', 'side': 'right'}]

Attributes

iv_names	(list of str)
iv_values	(list of tuple)
design_matrix	(array-like)
extra_data	(dict)
ordering	(Ordering)
heterogeneous_design_iv_name	(str) The IV name that triggers a heterogeneous (i.e., branching) tree structure when it is encountered. 'design' by default.
is_heterogeneous	(bool) True if this Design is the lowest level before the tree structure diverges.
branches	(dict) The IV values corresponding to named heterogeneous branches in the tree structure following this Design.

Methods

first_pass
from_dict
full_cross
get_order
update

first_pass()¶

Initialize design.

Initializes the design by parsing the design matrix or crossing the IVs If a NonAtomicOrdering is used, an additional IV will be returned which should be incorporated into the design one level up in the experimental hierarchy. For this reason, the first_pass methods in a hierarchy of Design instances should be called in reverse order, from bottom up. Use a DesignTree to ensure this occurs properly.

Returns:	iv_name : str or tuple The name of the IV, for non-atomic orderings. Otherwise, an empty tuple. iv_values : tuple The possible values of the IV. Empty for atomic orderings.

classmethod from_dict(spec)¶

Construct a Design instance from a specification based on dictionaries (e.g., parsed from a YAML file).

Parameters:

spec : dict A dictionary containing some of the following keys (all optional): 'name', the name of the level. 'ivs', 'design_matrix', 'extra_data', interpreted as keyword arguments to the Design constructor. 'order' or 'ordering', determining the ordering keyword argument. If the value is a string, it is interpreted as a class name in experimentator.order. If the value is a dictionary, the key 'class' should contain the class name, and the rest of the items are interpreted as keyword arguments to its constructor. If the value is a sequence, then the first element is interpreted as the class name and the following as positional arguments. 'n' or 'number', interpreted as the number argument to the specified Ordering. A dictionary containing any fields not otherwise used is passed to the Design constructor as the extra_data argument.

Returns:

name : str Only returned if spec contains a field 'name'. design : Design

See also

DesignTree.from_spec

Examples

>>> design_spec = {
...'name': 'block',
...'ivs': {'speed': [1, 2, 3], 'size': [15, 30]},
...'ordering': 'Shuffle',
...'n': 3}
>>> Design.from_dict(design_spec)
Level(name='block', design=Design(ivs=[('speed', [1, 2, 3]), ('size', [15, 30])], design_matrix=None, ordering=Shuffle(number=3, avoid_repeats=False), extra_data={}))

static full_cross(iv_names, iv_values)¶

Perform a full factorial cross of the independent variables. Yields dictionaries, each describing one condition, a mapping from IV names to IV values. One dictionary is yielded for every possible combination of IV values.

Parameters:	iv_names : list of str Names of IVs. iv_values : list of list Each element defines the possible values of an IV. Must be the same length as iv_names. Its elements must be hashable.

get_order(data=None)¶

Order the conditions.

Returns:	list of dict A list of dictionaries, each specifying a condition (a mapping from IV names to values).

update(names, values)¶

Add additional independent variables to the Design. This will have no effect after Design.first_pass has been called.

Parameters:	names : list of str Names of IVs to add. values : list of list For each IV, a list of possible values.

DesignTree¶

class experimentator.DesignTree(levels_and_designs, **other_designs)¶

A container for Design instances, describing the entire hierarchy of a basic Experiment. DesignTree instances are iterators; calling next on one will return another DesignTree with the top level removed. In this way, the entire experimental hierarchy can be created by recursively calling next.

Parameters:

levels_and_designs : OrderedDict or list of tuple This input defines the structure of the tree, and is either an OrderedDict or a list of 2-tuples. Keys (or first element of each tuple) are level names. Values (or second element of each tuple) are design specifications, in the form of either a Design instance, or a list of Design instances to occur in sequence. **other_designs Named design trees, can be other DesignTree instances or suitable levels_and_designs inputs (i.e., OrderedDict or list of tuples). These designs allow for heterogeneous design structures (i.e. not every section at the same level has the same Design). To make a heterogeneous DesignTree, use an IV named 'design' at the level where the heterogeneity should occur. Values of this IV should be strings, each corresponding to the name of a DesignTree from` other_designs`. The value of the IV 'design' at each section determines which DesignTree is used for children of that section.

Notes

Calling next on the last level of a heterogeneous DesignTree will return a dictionary of named DesignTree instances (rather than a single DesignTree instance). The keys are the possible values of the IV 'design' and the values are the corresponding DesignTree instances.

Attributes

levels_and_designs	(list of tuple)
other_designs	(dict)
branches	(dict) Only those items from other_designs that follow directly from this tree.

Methods

add_base_level
first_pass
from_spec

add_base_level()¶

Adds a section to the top of the tree called '_base'. This makes the DesignTree suitable for constructing an Experiment.

Notes

The Experiment constructor calls this automatically, and this shouldn’t be called when appending a tree to an existing Experiment, so there is no use case for manually calling this method.

static first_pass(levels_and_designs)¶

Make a first pass of all designs in a DesignTree, from bottom to top. This calls Design.first_pass on every Design instance in the tree in the proper order, updating designs when a new IV is returned. This is necessary for non-atomic orderings because they modify the parent Design.

classmethod from_spec(spec)¶

Constructs a DesignTree instance from a specification (e.g., parsed from a YAML file).

spec : dict or list of dict

The DesignTree specification. A dictionary with keys as tree names and values as lists of dictionaries. Each sub-dictionary should specify a Design according to Design.from_dict. The main tree should be named 'main'. Other names are used for generating heterogeneous trees (see DesignTree docs). A homogeneous tree can be specified as a dictionary with only a single key 'main', or directly as a list of dictionaries

Returns:	DesignTree

experimentator.order¶

This module contains the class Ordering and its descendants. These classes handle how unique conditions at a particular experimental level are ordered and duplicated. Ordering instances should be passed directly to the Design constructor; there is no reason to otherwise interact with them in normal use.

Of special note are non-atomic orderings: the class NonAtomicOrdering and its descendants. ‘’Non-atomic’’ here means that the orderings between sections are not independent. A Shuffle ordering is atomic; the order in one section is independent of the order in another. However, if one wants to ensure, for example, that possible block orders are evenly distributed among participants (a counterbalanced design), the block orders within each participants are not independent. Each participate can decide its order of blocks only in the context of the other participants’ block orders. This means that the parent section must handle orderings (in the example of counterbalanced blocks, the Experiment.base_section–the experiment itself, essentially–must tell each participant what block order to use).

class experimentator.order.CompleteCounterbalance(number=1)[source]¶

Bases: experimentator.order.NonAtomicOrdering

In a complete counterbalance design, every unique ordering of the conditions appears the same numbers of times.

Parameters:	number : int, optional The number of times each condition should be duplicated. Note that conditions are duplicated before determining the possible orderings.

Notes

The number of possible orderings can get very large very quickly. Therefore, a complete counterbalance is not recommended for more than 3 conditions. The number of unique orderings can be determined by factorial(number * k) // number**k, where k is the number of conditions (assuming all conditions are unique). For example, with 5 conditions there are 120 possible orders; with 3 conditions and number==2, there are 90 unique orders.

Attributes

iv	The IV associated with the non-atomic ordering.

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

first_pass(conditions)[source]¶

Handle operations that should only be performed once, initializing the object before ordering conditions. For CompleteCounterbalance, all possible orders are determined. This method should not be called manually.

Parameters:	conditions : sequence of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:	iv_name : str or tuple The name of the IV to be created one level up, 'counterbalance_order'. iv_values : tuple Values of the IV to be created one level up, integers each associated with an order of the conditions.

experimentator.order.IndependentVariable¶

alias of IndependentValue

class experimentator.order.LatinSquare(number=1, balanced=True, uniform=False)[source]¶

Bases: experimentator.order.NonAtomicOrdering

Orders the conditions by constructing an NxN Latin square, where N is the number of unique conditions. A Latin square is a matrix with each element appearing exactly once in each row and column. Each row represents a different potential ordering of the conditions. This allows for balanced counterbalancing, in designs too large to accommodate a complete counterbalance.

Parameters:

number : int, optional The number of times the Latin square should be repeated (default=1). Duplication occurs after constructing the square. balanced : bool, optional If True (the default), first-order order effects will be balanced Each condition will appear the same number of times immediately before and immediately after every other condition. Balanced latin squares can only be constructed with an even number of conditions. uniform : bool, optional If True (default is False), the Latin square will be randomly sampled from a uniform distribution of Latin squares of size NxN. Otherwise, the sampling will be biased. The construction of balanced, uniform Latin squares is not implemented.

Notes

The algorithm for computing unbalanced Latin squares is not very efficient. It is not recommended to construct unbalanced, uniform Latin squares of order above 5; for non-uniform, unbalanced Latin squares it is safe to go up to an order of 10 Higher than that, computation times increase rapidly.

The algorithm for computing balanced Latin squares is fast only because it is not robust; it is very biased and only samples from the same limited set of balanced Latin squares. However, this is usually not an issue. For more implementation details, see latin_square and balanced_latin_square.

Attributes

iv	The IV associated with the non-atomic ordering.

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

first_pass(conditions)[source]¶

Handle operations that should only be performed once, initializing the object before ordering conditions. For LatinSquare, the square is constructed.

Parameters:	conditions : sequence of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:	iv_name : str or tuple The name of the IV to be created one level up, 'latin_square_row'. iv_values : tuple Values of the IV to be created one level up, integers each corresponding to one row of the Latin square.

class experimentator.order.NonAtomicOrdering(number=1)[source]¶

Bases: experimentator.order.Ordering

This is a base class for non-atomic orderings, and is not meant to be directly instantiated. Non-atomic orderings work by creating a new independent variable one level up. The IV name will start with an underscore, a convention to avoid name clashes.

Attributes

iv	The IV associated with the non-atomic ordering.

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

get_order(data=None)[source]¶

Get an order of conditions.

Parameters:	data : dict, optional A dictionary describing the data of the parent section.
Returns:	list of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.

iv[source]¶

The IV associated with the non-atomic ordering. It will be added to the design one level up in the experiment hierarchy.

Returns:	iv_name : str or tuple The name of the IV, for non-atomic orderings. Otherwise, an empty tuple. iv_values : seq The possible values of the IV. Empty for atomic orderings.

class experimentator.order.Ordering(number=1)[source]¶

Bases: builtins.object

The base ordering class. It will keep conditions in the order they are defined by the Design instance (either the order of rows in the design matrix, or a full factorial cross–the output of calling itertools.product on the IV levels). Remember not to rely on the order of dictionary items. Therefore, if a specific order is desired, it is recommended to use a design matrix or an OrderedDict to define the IVs. See the IV docs for more information.

Parameters:	number : int, optional The number of times each unique condition should appear. The default is 1. If number > 1, the entire order will be cycled (as opposed to repeating each condition within the order).

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

first_pass(conditions)[source]¶

Handle operations that should only be performed once, initializing the object before ordering conditions. For Ordering, the only operation is duplication of the list of conditions (if Ordering.number > 1). This methods should not be called manually.

Parameters:	conditions : sequence of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:	iv_name : str or tuple The name of the IV, for non-atomic orderings. Otherwise, an empty tuple. iv_values : tuple The possible values of the IV. Empty for atomic orderings.

get_order(data=None)[source]¶

Get an order of conditions. For Ordering, always returns the same order.

Parameters:	data : dict, optional A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns:	list of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.

static possible_orders(conditions, unique=True)[source]¶

Yield all possible orders of the conditions. Each order is a list of dictionaries, with each dictionary representing a condition.

Parameters:	conditions : sequence of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values. unique : bool, optional If true (the default), will only return unique orders. If false, some identical orders will be generated if conditions contains identical elements.

class experimentator.order.Shuffle(number=1, avoid_repeats=False)[source]¶

Bases: experimentator.order.Ordering

This ordering randomly shuffles the conditions.

Parameters:	number : int, optional Number of times each condition should appear (default=1). Conditions are duplicated before shuffling. avoid_repeats : bool, optional If True (default is False), no identical conditions will appear back-to-back.

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

get_order(data=None)[source]¶

Get an order of conditions. For Shuffle, returns the conditions in a random order.

Parameters:	data : dict, optional A dictionary describing the data of the parent section. Unused for atomic orderings.
Returns:	list of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.

class experimentator.order.Sorted(number=1, order='both')[source]¶

Bases: experimentator.order.NonAtomicOrdering

Sorts the conditions based on the value of the IV defined at its level. This ordering can be non-atomic (if order == 'both'), creating an IV with levels of 'ascending' and 'descending'. If order is 'ascending' or 'descending', the ordering will be atomic (each section will be ordered the same way).

Parameters:	order : {‘both’, ‘ascending’, ‘descending’}, optional The order to sort the sections. If 'both' (the default), half the sections will be created in ascending order, and half in descending order, depending on the value of the new IV 'sorted_order'. number : int, optional The number of times each condition should appear.

Notes

To avoid ambiguity, Sorted can only be used at levels containing only one IV.

Attributes

iv	The IV associated with the non-atomic ordering.

Methods

first_pass(conditions)	Handle operations that should only be performed once, initializing the object before ordering conditions.
get_order([data])	Get an order of conditions.
possible_orders(conditions[, unique])	Yield all possible orders of the conditions.

first_pass(conditions)[source]¶

Handle operations that should only be performed once, initializing the object before ordering conditions. For Sorted, the conditions are sorted on IV values.

Parameters:	conditions : sequence of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.
Returns:	iv_name : str or tuple If order is 'both', the name of the IV to be created one level up, 'sorted_order'. Otherwise, an empty tuple (denoting that no IV will be created). iv_values : tuple If order is 'both', values of the IV to be created one level up, integers each associated with an order of the conditions. Otherwise, empty tuple.

get_order(data=None)[source]¶

Get an order of conditions.

Parameters:	data : dict, optional A dictionary describing the data of the parent section.
Returns:	list of dict A list of conditions, where each condition is a dictionary mapping IV names to IV values.

experimentator.order.balanced_latin_square(order)[source]¶

Constructs a row-balanced latin square of order order. In a row-balanced Latin square, immediate order effects are accounted for. Every two-element, back-to-back sequence occurs the same number of times. This is in addition to the standard Latin square constraint with every row and column containing each element exactly once.

Parameters:	order : int Order of the Latin square to construct. Must be even.
Returns:	array-like A balanced Latin square of size order x order.

See also

latin_square, LatinSquare

Notes

The algorithm constructs a stereotypical balanced Latin square, then shuffles the rows and elements (but not the columns). This algorithm is much faster than the algorithm used by latin_square. However, it does not sample from a uniform distribution of balanced Latin squares (i.e. it is biased) and it cannot create Latin squares that are both reduced and balanced.

Examples

>>> balanced_latin_square(6)
[[0, 2, 1, 5, 4, 3],
 [5, 3, 2, 4, 0, 1],
 [1, 0, 4, 2, 3, 5],
 [2, 5, 0, 3, 1, 4],
 [4, 1, 3, 0, 5, 2],
 [3, 4, 5, 1, 2, 0]]  # random

experimentator.order.latin_square(order, reduced=False, uniform=True, shuffle=False)[source]¶

Constructs a Latin square of size order x order. Each row and column will contain every element of range(order) exactly once.

Parameters:

order : int Size of Latin square to construct. reduced : bool, optional If True (default is False), the first row and first column of the square will be the list(range(order)), unless shuffle is also True. uniform : bool, optional If True (the default), the Latin square will be sampled from a uniform distribution of Latin squares. Set to False to relax this constraint and allow for a faster run time. shuffle : bool, optional If True (default is False), after construction of the Latin square its rows will be shuffled randomly, then its columns, and then the elements will be randomly permuted. Shuffling is irrelevant when uniform is True. Otherwise, it adds some randomness, though the resulting Latin square will still be biased.

Returns:

array-like A Latin square of size order x order.

See also

balanced_latin_square, LatinSquare

Notes

This function uses a naive algorithm to construct latin squares, randomly generating elements and starting over whenever a collision is encountered. It will take a long time to construct Latin squares of order 5, when sampling from a uniform distribution. However, if a uniform distribution is not required, it is recommended to also set reduced and shuffle to True for fastest run times. In this case, latin squares up to an order of about 10 can be constructed in a reasonable amount of time.

Examples

>>> latin_square(5)
[[4, 2, 0, 1, 3],
 [0, 3, 1, 4, 2],
 [3, 1, 2, 0, 4],
 [2, 0, 4, 3, 1],
 [1, 4, 3, 2, 0]]  #random

>>> latin_square(5, reduced=True, uniform=False)
[[0, 1, 2, 3, 4],
 [1, 2, 4, 0, 3],
 [2, 0, 3, 4, 1],
 [3, 4, 1, 2, 0],
 [4, 3, 0, 1, 2]]  #random