pulse2percept.stimuli.videos

VideoStimulus, BostonTrain, GirlPool

Classes

BostonTrain([resize, electrodes, as_gray, ...])

Boston Train sequence

GirlPool([resize, electrodes, as_gray, metadata])

A girl jumping into a swimming pool

VideoStimulus(source[, format, resize, ...])

A stimulus made from a movie file, where each pixel gets assigned to an electrode, and grayscale values in the range [0, 255] get assigned to activation values in the range [0, 1].
class pulse2percept.stimuli.videos.VideoStimulus(source, format=None, resize=None, as_gray=False, electrodes=None, time=None, metadata=None, compress=False)[source]

A stimulus made from a movie file, where each pixel gets assigned to an electrode, and grayscale values in the range [0, 255] get assigned to activation values in the range [0, 1].

The frame rate of the movie is used to infer the time points at which to stimulate.

See also

  • Basic Concepts > Electrical Stimuli <topics-stimuli>

  • ImageStimulus

Added in version 0.7.

Parameters:

  • source (str) –

    Path to video file. Supported file types include MP4, AVI, MOV, and GIF; and are inferred from the file ending. If the file does not have a proper file ending, specify the file type via format.

    Alternatively, pass a <rows x columns x channels x frames> NumPy array or another VideoStimulus object.

  • format (str) – A video format string supported by imageio, such as ‘MP4’, ‘AVI’, or ‘MOV’. Use if the file type cannot be inferred from source. For a full list of supported formats, see https://imageio.readthedocs.io/en/stable/formats.html.

  • resize ((height, width) or None, optional, default: None) – A tuple specifying the desired height and the width of each video frame

  • as_gray (bool, optional) – Flag whether to convert the image to grayscale. A four-channel image is interpreted as RGBA (e.g., a PNG), and the alpha channel will be blended with the color black.

  • electrodes (int, string or list thereof; optional, default: None) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image: a letter for the row, a number for the column, and a suffix for the color channel (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the (resized) image.

  • metadata (dict, optional, default: None) – Additional stimulus metadata can be stored in a dictionary.

  • compress (bool, optional, default: False) – If True, will compress the source data in two ways: * Remove electrodes with all-zero activation. * Retain only the time points at which the stimulus changes.

compress()[source]

Compress the source data

Also brings vid_shape back in line with the compressed data: compression drops the time points at which the video does not change, so the frame count of the source is no longer the frame count of the stimulus. Every data.reshape(vid_shape) in this module relies on that invariant. (Compression can also drop all-zero pixels, in which case no shape describes the data any more; see _frames.)

Returns:

compressed

Return type:

VideoStimulus

apply(func, *args, electrodes=None, **kwargs)[source]

Apply a function to each frame of the video

Changed in version 0.10.0: func may now change the shape of a frame, and electrodes can name the result.

Parameters:

  • func (function) – The function to apply to each frame in the video. Must accept a 2D or 3D image and return a 2D or 3D image. The returned frames need not have the same shape as the originals (but must all have the same shape as each other); see electrodes.

  • *args – Additional positional arguments passed to the function

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, the original names are carried over whenever func leaves the shape of a frame alone, and the result is named after its place in the new frame otherwise (e.g. for skimage.transform.resize). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in a returned frame.

  • **kwargs – Additional keyword arguments passed to the function

Returns:

stim – A copy of the stimulus object with the new video

Return type:

VideoStimulus

invert()[source]

Invert the gray levels of the video

Returns:

stim – A copy of the stimulus object with all grayscale values inverted in the range [0, 1].

Return type:

VideoStimulus

rgb2gray(electrodes=None)[source]

Convert the video to grayscale

Parameters:

electrodes (int, string or list thereof; optional) –

Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

Note

The number of electrode names provided must match the number of pixels in the grayscale video.

Returns:

stim – A copy of the stimulus object with all RGB values converted to grayscale in the range [0, 1].

Return type:

VideoStimulus

resize(shape, electrodes=None, **kwargs)[source]

Resize the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • shape ((rows, cols)) – Shape of each frame in the resized video. If one of the dimensions is set to -1, its value will be inferred by keeping a constant aspect ratio.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the resized video.

  • **kwargs – Additional keyword arguments passed to skimage.transform.resize, such as order=0 for nearest-neighbor interpolation (which keeps a binary video binary).

Returns:

stim – A copy of the stimulus object containing the resized video

Return type:

VideoStimulus

crop(idx_space=None, idx_time=None, left=0, right=0, top=0, bottom=0, front=0, back=0, electrodes=None)[source]

Crop the video

This method maps a rectangle (defined by two corners) from each video frame to a rectangle of the given size. Similarly, the video can be shortened to a specified range of frames.

Alternatively, this method can be used to crop a number of columns either from the left or the right of the video frame, or a number of rows either from the top or the bottom, or a number of frames from the front (beginning) or back (end) of the video.

Added in version 0.8.

Parameters:

  • idx_space (4-tuple (y0, x0, y1, x1)) – Image indices of the top-left corner [y0, x0] and bottom-right corner [y1, x1] (exclusive) of the rectangle to crop.

  • idx_time (tuple (t0, t1)) – Frame indices defining the start t0 and end t1 of the cropped video.

  • left (int) – Number of columns to crop from the left of each video frame

  • right (int) – Number of columns to crop from the right of each video frame

  • top (int) – Number of rows to crop from the top of each video frame

  • bottom (int) – Number of rows to crop from the bottom of each video frame

  • front (int) – Number of frames to crop from the front (beginning) of the video

  • back (int) – Number of frames to crop from the back (end) of the video

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the cropped image.

Returns:

stim – A copy of the stimulus object containing the video

Return type:

VideoStimulus

trim(tol=0, electrodes=None)[source]

Remove any black border around the video

Added in version 0.7.

Parameters:

  • tol (float) – Any pixels with gray levels > tol will be trimmed.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in each frame of the trimmed video.

Returns:

stim – A copy of the stimulus object with trimmed borders.

Return type:

VideoStimulus

rotate(angle, mode='constant', electrodes=None, **kwargs)[source]

Rotate each frame of the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • angle (float) – Angle by which to rotate each video frame (degrees). Positive: counter-clockwise, negative: clockwise

  • mode (str, optional) – How to fill in the corners the rotation leaves empty; see skimage.transform.rotate.

  • electrodes (int, string or list thereof; optional) – Optionally, you can provide your own electrode names. If none are given, each pixel keeps the name it had before the rotation, unless resize=True grew the frame, in which case the enlarged video is named after its own pixel grid. See ElectrodeNames.

  • **kwargs – Additional keyword arguments passed to skimage.transform.rotate, such as order, cval, or resize=True to grow each frame so that it contains every rotated pixel.

Returns:

stim – A copy of the stimulus object containing the rotated video

Return type:

VideoStimulus

shift(shift_cols, shift_rows)[source]

Shift the image foreground

This function shifts the center of mass (CoM) of the image by the specified number of rows and columns.

Parameters:

  • shift_cols (float) – Number of columns by which to shift the CoM. Positive: to the right, negative: to the left

  • shift_rows (float) – Number of rows by which to shift the CoM. Positive: downward, negative: upward

Returns:

stim – A copy of the stimulus object containing the shifted image

Return type:

ImageStimulus

center(loc=None)[source]

Center the image foreground

This function shifts the center of mass (CoM) to the image center.

Parameters:

loc ((col, row), optional) – The pixel location at which to center the CoM. By default, shifts the CoM to the image center.

Returns:

stim – A copy of the stimulus object containing the centered image

Return type:

ImageStimulus

scale(scaling_factor)[source]

Scale the image foreground

This function scales the image foreground (excluding black pixels) by a factor.

Parameters:

scaling_factor (float) – Factory by which to scale the image

Returns:

stim – A copy of the stimulus object containing the scaled image

Return type:

ImageStimulus

filter(filt, **kwargs)[source]

Filter each frame of the video

Parameters:

  • filt (str) –

    Image filter that will be applied to every frame of the video. Additional parameters can be passed as keyword arguments. The following filters are supported:

    • ’sobel’: Edge filter the image using the Sobel filter.

    • ’scharr’: Edge filter the image using the Scarr filter.

    • ’canny’: Edge filter the image using the Canny algorithm. You can also specify sigma, low_threshold, high_threshold, mask, and use_quantiles.

    • ’median’: Return local median of the image.

  • **kwargs – Additional parameters passed to the filter

Returns:

stim – A copy of the stimulus object with the filtered image

Return type:

VideoStimulus

encode(amp_range=(0, 50), freq=20, implant=None, **kwargs)[source]

Encode the video using amplitude modulation

Encodes every frame of the video as a train of biphasic pulses, where the gray level of a pixel sets the amplitude of its pulses. Each train lasts one frame period.

This is a shorthand for AmplitudeEncoder; use that directly for the full set of options.

Changed in version 0.10.0: Gray levels now map onto amp_range absolutely rather than being stretched to fill it (pass stretch=True for the old behavior), each frame receives a pulse train rather than a single pulse, and implant encodes at electrode rather than pixel resolution.

Parameters:

  • amp_range ((min_amp, max_amp), optional) – Range of pulse amplitudes (uA). A gray level of 0 maps onto min_amp, a gray level of 1 onto max_amp.

  • freq (float, optional) – Pulse train frequency (Hz).

  • implant (ProsthesisSystem, optional) – If given, the video is first sampled at the implant’s electrode locations, so that the pulse trains are built at electrode rather than pixel resolution. Strongly recommended: a video has orders of magnitude more pixels than an implant has electrodes.

  • **kwargs – Additional arguments passed to AmplitudeEncoder.

Returns:

stim – Encoded stimulus

Return type:

Stimulus

rewind()[source]

Rewind the iterator

play(fps=None, repeat=True, annotate_time=True, ax=None, fmt='jpg')[source]

Animate the video as HTML with JavaScript

The video will be played in an interactive player in IPython or Jupyter Notebook.

Parameters:

  • fps (float or None) – If None, uses the video’s time axis. Not supported for non-homogeneous time axis.

  • repeat (bool, optional) – Whether the animation should repeat when the sequence of frames is completed.

  • annotate_time (bool, optional) – If True, the time of the frame will be shown as t = X ms in the title of the panel.

  • ax (matplotlib.axes.AxesSubplot, optional) – A Matplotlib axes object. If None, will create a new Axes object

  • fmt ({'jpg', 'png'}, optional) –

    The image format used to embed the frames. ‘jpg’ keeps notebooks and doc pages an order of magnitude smaller; use ‘png’ if you need the frames to be pixel-exact.

    Added in version 0.10.0.

Returns:

ani – A Matplotlib animation object that will play the video frame-by-frame.

Return type:

pulse2percept.utils.HTMLAnimation

Notes

Changed in version 0.10.0: The HTML player is now generated by HTMLAnimation, which renders the figure once and ships all frames as a single sprite sheet. This is roughly two orders of magnitude faster than Matplotlib’s to_jshtml and produces much smaller notebooks and doc pages.

append(other)[source]

Append another stimulus

This method appends another stimulus (with matching electrodes) in time. The combined stimulus duration will be the sum of the two individual stimuli.

Added in version 0.7.

Parameters:

other (Stimulus) – Another stimulus with matching electrodes.

Returns:

comb – A combined stimulus with the same number of electrodes and new stimulus duration equal to the sum of the two individual stimuli.

Return type:

Stimulus

property data

Stimulus data container A 2-D NumPy array that contains the stimulus data, where the rows denote electrodes and the columns denote points in time.

property dt

Sampling time step (ms)

Defines the duration of the signal edge transitions.

Added in version 0.7.

property duration

Stimulus duration (ms)

property electrodes

Electrode names A list of electrode names, corresponding to the rows in the data container.

property is_charge_balanced

Flag indicating whether the stimulus is charge-balanced

A stimulus with a time component is considered charge-balanced if its net current is smaller than 10 pico Amps. For the whole stimulus to be charge-balanced, every electrode must be charge-balanced as well.

Returns None if the stimulus is not a current at all: the gray levels of an ImageStimulus integrate to a number like any others, but that number is not a charge and asking whether it is zero answers nothing. Note that this is “not applicable”, not “unbalanced” – it is safe_mode that turns the question into an error, since a safety system genuinely cannot do its job on a stimulus that is not electrical.

Changed in version 0.10.0: Returns None for a stimulus that is not measured in units of current (was: integrated the values anyway).

property is_compressed

Flag indicating whether the stimulus has been compressed

Read-only: the flag is maintained by compress. Assigning to it raises an AttributeError.

plot(electrodes=None, time=None, fmt='k-', ax=None)[source]

Plot the stimulus

Added in version 0.7.

Parameters:

  • electrodes (int, string, or list thereof; optional, default: None) – The electrodes for which to plot the stimulus. If None, all electrodes are plotted.

  • time ((t_min, t_max) tuple, slice, or list of exact time points) – The time points at which to plot the stimulus. Specify a range of time points with a tuple or a slice, or specify the exact time points to interpolate. If None, all time points are plotted.

  • fmt (str, optional, default: 'k-') – A Matplotlib format string; e.g., ‘ro’ for red circles.

  • ax (matplotlib.axes.Axes or list thereof; optional, default: None) – A Matplotlib Axes object or a list thereof (one per electrode to plot). If None, a new Axes object will be created.

Returns:

axes – Returns one matplotlib.axes.Axes per electrode

Return type:

matplotlib.axes.Axes or np.ndarray of them

property quantity

The stimulus data, with its unit attached

Added in version 0.10.0.

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA
>>> Stimulus([500, 1000] * uA).quantity
[[ 500.]
 [1000.]] uA
remove(electrodes)[source]

Remove electrode(s)

Removes the stimulus of a certain electrode or list of electrodes.

Added in version 0.8.

Parameters:

electrodes (int, string, or list of int/str) – The item(s) to remove from the stimulus. Can either be an electrode index, electrode name, or a list thereof.

property shape

Data container shape

property time

Time steps A list of time steps, corresponding to the columns in the data container.

property time_quantity

The stimulus time axis with its unit attached, or None

Added in version 0.10.0.

property time_unit

The unit time is expressed in (milliseconds)

Added in version 0.10.0.

times(unit=None)[source]

The stimulus time axis, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the time axis in. If None, time is returned as it is stored (milliseconds).

Returns:

times – An ordinary NumPy array, or None if the stimulus has no time component.

Return type:

np.ndarray or None

property unit

The unit data is expressed in

Microamps for an electrical stimulus, dimensionless for the gray levels of an ImageStimulus or VideoStimulus.

Read-only. The canonical storage unit is fixed so that models, safety checks and Cython kernels can rely on it; ask for another unit with values().

Added in version 0.10.0.

values(unit=None)[source]

The stimulus data, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the data in. Must be compatible with unit. If None, the stimulus’ own unit is used and data is returned as it is stored.

Returns:

values – An ordinary NumPy array, never a Quantity. This is the boundary a numerical implementation should take its data across.

Return type:

np.ndarray

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA, mA
>>> Stimulus([500, 1000] * uA).values(mA)
array([[0.5],
       [1. ]], dtype=float32)
class pulse2percept.stimuli.videos.BostonTrain(resize=None, electrodes=None, as_gray=False, metadata=None)[source]

Boston Train sequence

Load the Boston subway sequence, consisting of 94 frames of 240x426x3 pixels each.

Added in version 0.7.

Parameters:

  • resize ((height, width) or None) – A tuple specifying the desired height and the width of the video stimulus.

  • electrodes (int, string or list thereof; optional, default: None) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image: a letter for the row, a number for the column, and a suffix for the color channel (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the (resized) video frame.

  • as_gray (bool, optional) – Flag whether to convert the image to grayscale. A four-channel image is interpreted as RGBA (e.g., a PNG), and the alpha channel will be blended with the color black.

  • metadata (dict, optional, default: None) – Additional stimulus metadata can be stored in a dictionary.

append(other)[source]

Append another stimulus

This method appends another stimulus (with matching electrodes) in time. The combined stimulus duration will be the sum of the two individual stimuli.

Added in version 0.7.

Parameters:

other (Stimulus) – Another stimulus with matching electrodes.

Returns:

comb – A combined stimulus with the same number of electrodes and new stimulus duration equal to the sum of the two individual stimuli.

Return type:

Stimulus

apply(func, *args, electrodes=None, **kwargs)[source]

Apply a function to each frame of the video

Changed in version 0.10.0: func may now change the shape of a frame, and electrodes can name the result.

Parameters:

  • func (function) – The function to apply to each frame in the video. Must accept a 2D or 3D image and return a 2D or 3D image. The returned frames need not have the same shape as the originals (but must all have the same shape as each other); see electrodes.

  • *args – Additional positional arguments passed to the function

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, the original names are carried over whenever func leaves the shape of a frame alone, and the result is named after its place in the new frame otherwise (e.g. for skimage.transform.resize). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in a returned frame.

  • **kwargs – Additional keyword arguments passed to the function

Returns:

stim – A copy of the stimulus object with the new video

Return type:

VideoStimulus

center(loc=None)[source]

Center the image foreground

This function shifts the center of mass (CoM) to the image center.

Parameters:

loc ((col, row), optional) – The pixel location at which to center the CoM. By default, shifts the CoM to the image center.

Returns:

stim – A copy of the stimulus object containing the centered image

Return type:

ImageStimulus

compress()[source]

Compress the source data

Also brings vid_shape back in line with the compressed data: compression drops the time points at which the video does not change, so the frame count of the source is no longer the frame count of the stimulus. Every data.reshape(vid_shape) in this module relies on that invariant. (Compression can also drop all-zero pixels, in which case no shape describes the data any more; see _frames.)

Returns:

compressed

Return type:

VideoStimulus

crop(idx_space=None, idx_time=None, left=0, right=0, top=0, bottom=0, front=0, back=0, electrodes=None)[source]

Crop the video

This method maps a rectangle (defined by two corners) from each video frame to a rectangle of the given size. Similarly, the video can be shortened to a specified range of frames.

Alternatively, this method can be used to crop a number of columns either from the left or the right of the video frame, or a number of rows either from the top or the bottom, or a number of frames from the front (beginning) or back (end) of the video.

Added in version 0.8.

Parameters:

  • idx_space (4-tuple (y0, x0, y1, x1)) – Image indices of the top-left corner [y0, x0] and bottom-right corner [y1, x1] (exclusive) of the rectangle to crop.

  • idx_time (tuple (t0, t1)) – Frame indices defining the start t0 and end t1 of the cropped video.

  • left (int) – Number of columns to crop from the left of each video frame

  • right (int) – Number of columns to crop from the right of each video frame

  • top (int) – Number of rows to crop from the top of each video frame

  • bottom (int) – Number of rows to crop from the bottom of each video frame

  • front (int) – Number of frames to crop from the front (beginning) of the video

  • back (int) – Number of frames to crop from the back (end) of the video

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the cropped image.

Returns:

stim – A copy of the stimulus object containing the video

Return type:

VideoStimulus

property data

Stimulus data container A 2-D NumPy array that contains the stimulus data, where the rows denote electrodes and the columns denote points in time.

property dt

Sampling time step (ms)

Defines the duration of the signal edge transitions.

Added in version 0.7.

property duration

Stimulus duration (ms)

property electrodes

Electrode names A list of electrode names, corresponding to the rows in the data container.

encode(amp_range=(0, 50), freq=20, implant=None, **kwargs)[source]

Encode the video using amplitude modulation

Encodes every frame of the video as a train of biphasic pulses, where the gray level of a pixel sets the amplitude of its pulses. Each train lasts one frame period.

This is a shorthand for AmplitudeEncoder; use that directly for the full set of options.

Changed in version 0.10.0: Gray levels now map onto amp_range absolutely rather than being stretched to fill it (pass stretch=True for the old behavior), each frame receives a pulse train rather than a single pulse, and implant encodes at electrode rather than pixel resolution.

Parameters:

  • amp_range ((min_amp, max_amp), optional) – Range of pulse amplitudes (uA). A gray level of 0 maps onto min_amp, a gray level of 1 onto max_amp.

  • freq (float, optional) – Pulse train frequency (Hz).

  • implant (ProsthesisSystem, optional) – If given, the video is first sampled at the implant’s electrode locations, so that the pulse trains are built at electrode rather than pixel resolution. Strongly recommended: a video has orders of magnitude more pixels than an implant has electrodes.

  • **kwargs – Additional arguments passed to AmplitudeEncoder.

Returns:

stim – Encoded stimulus

Return type:

Stimulus

filter(filt, **kwargs)[source]

Filter each frame of the video

Parameters:

  • filt (str) –

    Image filter that will be applied to every frame of the video. Additional parameters can be passed as keyword arguments. The following filters are supported:

    • ’sobel’: Edge filter the image using the Sobel filter.

    • ’scharr’: Edge filter the image using the Scarr filter.

    • ’canny’: Edge filter the image using the Canny algorithm. You can also specify sigma, low_threshold, high_threshold, mask, and use_quantiles.

    • ’median’: Return local median of the image.

  • **kwargs – Additional parameters passed to the filter

Returns:

stim – A copy of the stimulus object with the filtered image

Return type:

VideoStimulus

invert()[source]

Invert the gray levels of the video

Returns:

stim – A copy of the stimulus object with all grayscale values inverted in the range [0, 1].

Return type:

VideoStimulus

property is_charge_balanced

Flag indicating whether the stimulus is charge-balanced

A stimulus with a time component is considered charge-balanced if its net current is smaller than 10 pico Amps. For the whole stimulus to be charge-balanced, every electrode must be charge-balanced as well.

Returns None if the stimulus is not a current at all: the gray levels of an ImageStimulus integrate to a number like any others, but that number is not a charge and asking whether it is zero answers nothing. Note that this is “not applicable”, not “unbalanced” – it is safe_mode that turns the question into an error, since a safety system genuinely cannot do its job on a stimulus that is not electrical.

Changed in version 0.10.0: Returns None for a stimulus that is not measured in units of current (was: integrated the values anyway).

property is_compressed

Flag indicating whether the stimulus has been compressed

Read-only: the flag is maintained by compress. Assigning to it raises an AttributeError.

play(fps=None, repeat=True, annotate_time=True, ax=None, fmt='jpg')[source]

Animate the video as HTML with JavaScript

The video will be played in an interactive player in IPython or Jupyter Notebook.

Parameters:

  • fps (float or None) – If None, uses the video’s time axis. Not supported for non-homogeneous time axis.

  • repeat (bool, optional) – Whether the animation should repeat when the sequence of frames is completed.

  • annotate_time (bool, optional) – If True, the time of the frame will be shown as t = X ms in the title of the panel.

  • ax (matplotlib.axes.AxesSubplot, optional) – A Matplotlib axes object. If None, will create a new Axes object

  • fmt ({'jpg', 'png'}, optional) –

    The image format used to embed the frames. ‘jpg’ keeps notebooks and doc pages an order of magnitude smaller; use ‘png’ if you need the frames to be pixel-exact.

    Added in version 0.10.0.

Returns:

ani – A Matplotlib animation object that will play the video frame-by-frame.

Return type:

pulse2percept.utils.HTMLAnimation

Notes

Changed in version 0.10.0: The HTML player is now generated by HTMLAnimation, which renders the figure once and ships all frames as a single sprite sheet. This is roughly two orders of magnitude faster than Matplotlib’s to_jshtml and produces much smaller notebooks and doc pages.

plot(electrodes=None, time=None, fmt='k-', ax=None)[source]

Plot the stimulus

Added in version 0.7.

Parameters:

  • electrodes (int, string, or list thereof; optional, default: None) – The electrodes for which to plot the stimulus. If None, all electrodes are plotted.

  • time ((t_min, t_max) tuple, slice, or list of exact time points) – The time points at which to plot the stimulus. Specify a range of time points with a tuple or a slice, or specify the exact time points to interpolate. If None, all time points are plotted.

  • fmt (str, optional, default: 'k-') – A Matplotlib format string; e.g., ‘ro’ for red circles.

  • ax (matplotlib.axes.Axes or list thereof; optional, default: None) – A Matplotlib Axes object or a list thereof (one per electrode to plot). If None, a new Axes object will be created.

Returns:

axes – Returns one matplotlib.axes.Axes per electrode

Return type:

matplotlib.axes.Axes or np.ndarray of them

property quantity

The stimulus data, with its unit attached

Added in version 0.10.0.

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA
>>> Stimulus([500, 1000] * uA).quantity
[[ 500.]
 [1000.]] uA
remove(electrodes)[source]

Remove electrode(s)

Removes the stimulus of a certain electrode or list of electrodes.

Added in version 0.8.

Parameters:

electrodes (int, string, or list of int/str) – The item(s) to remove from the stimulus. Can either be an electrode index, electrode name, or a list thereof.

resize(shape, electrodes=None, **kwargs)[source]

Resize the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • shape ((rows, cols)) – Shape of each frame in the resized video. If one of the dimensions is set to -1, its value will be inferred by keeping a constant aspect ratio.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the resized video.

  • **kwargs – Additional keyword arguments passed to skimage.transform.resize, such as order=0 for nearest-neighbor interpolation (which keeps a binary video binary).

Returns:

stim – A copy of the stimulus object containing the resized video

Return type:

VideoStimulus

rewind()[source]

Rewind the iterator

rgb2gray(electrodes=None)[source]

Convert the video to grayscale

Parameters:

electrodes (int, string or list thereof; optional) –

Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

Note

The number of electrode names provided must match the number of pixels in the grayscale video.

Returns:

stim – A copy of the stimulus object with all RGB values converted to grayscale in the range [0, 1].

Return type:

VideoStimulus

rotate(angle, mode='constant', electrodes=None, **kwargs)[source]

Rotate each frame of the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • angle (float) – Angle by which to rotate each video frame (degrees). Positive: counter-clockwise, negative: clockwise

  • mode (str, optional) – How to fill in the corners the rotation leaves empty; see skimage.transform.rotate.

  • electrodes (int, string or list thereof; optional) – Optionally, you can provide your own electrode names. If none are given, each pixel keeps the name it had before the rotation, unless resize=True grew the frame, in which case the enlarged video is named after its own pixel grid. See ElectrodeNames.

  • **kwargs – Additional keyword arguments passed to skimage.transform.rotate, such as order, cval, or resize=True to grow each frame so that it contains every rotated pixel.

Returns:

stim – A copy of the stimulus object containing the rotated video

Return type:

VideoStimulus

scale(scaling_factor)[source]

Scale the image foreground

This function scales the image foreground (excluding black pixels) by a factor.

Parameters:

scaling_factor (float) – Factory by which to scale the image

Returns:

stim – A copy of the stimulus object containing the scaled image

Return type:

ImageStimulus

property shape

Data container shape

shift(shift_cols, shift_rows)[source]

Shift the image foreground

This function shifts the center of mass (CoM) of the image by the specified number of rows and columns.

Parameters:

  • shift_cols (float) – Number of columns by which to shift the CoM. Positive: to the right, negative: to the left

  • shift_rows (float) – Number of rows by which to shift the CoM. Positive: downward, negative: upward

Returns:

stim – A copy of the stimulus object containing the shifted image

Return type:

ImageStimulus

property time

Time steps A list of time steps, corresponding to the columns in the data container.

property time_quantity

The stimulus time axis with its unit attached, or None

Added in version 0.10.0.

property time_unit

The unit time is expressed in (milliseconds)

Added in version 0.10.0.

times(unit=None)[source]

The stimulus time axis, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the time axis in. If None, time is returned as it is stored (milliseconds).

Returns:

times – An ordinary NumPy array, or None if the stimulus has no time component.

Return type:

np.ndarray or None

trim(tol=0, electrodes=None)[source]

Remove any black border around the video

Added in version 0.7.

Parameters:

  • tol (float) – Any pixels with gray levels > tol will be trimmed.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in each frame of the trimmed video.

Returns:

stim – A copy of the stimulus object with trimmed borders.

Return type:

VideoStimulus

property unit

The unit data is expressed in

Microamps for an electrical stimulus, dimensionless for the gray levels of an ImageStimulus or VideoStimulus.

Read-only. The canonical storage unit is fixed so that models, safety checks and Cython kernels can rely on it; ask for another unit with values().

Added in version 0.10.0.

values(unit=None)[source]

The stimulus data, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the data in. Must be compatible with unit. If None, the stimulus’ own unit is used and data is returned as it is stored.

Returns:

values – An ordinary NumPy array, never a Quantity. This is the boundary a numerical implementation should take its data across.

Return type:

np.ndarray

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA, mA
>>> Stimulus([500, 1000] * uA).values(mA)
array([[0.5],
       [1. ]], dtype=float32)
class pulse2percept.stimuli.videos.GirlPool(resize=None, electrodes=None, as_gray=False, metadata=None)[source]

A girl jumping into a swimming pool

Load the “girl jumping in a pool” sequence, consisting of 91 frames of 240x426x3 pixels each.

Added in version 0.9.

Parameters:

  • resize ((height, width) or None) – A tuple specifying the desired height and the width of the video stimulus.

  • electrodes (int, string or list thereof; optional, default: None) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image: a letter for the row, a number for the column, and a suffix for the color channel (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the (resized) video frame.

  • as_gray (bool, optional) – Flag whether to convert the image to grayscale. A four-channel image is interpreted as RGBA (e.g., a PNG), and the alpha channel will be blended with the color black.

  • metadata (dict, optional, default: None) – Additional stimulus metadata can be stored in a dictionary.

append(other)[source]

Append another stimulus

This method appends another stimulus (with matching electrodes) in time. The combined stimulus duration will be the sum of the two individual stimuli.

Added in version 0.7.

Parameters:

other (Stimulus) – Another stimulus with matching electrodes.

Returns:

comb – A combined stimulus with the same number of electrodes and new stimulus duration equal to the sum of the two individual stimuli.

Return type:

Stimulus

apply(func, *args, electrodes=None, **kwargs)[source]

Apply a function to each frame of the video

Changed in version 0.10.0: func may now change the shape of a frame, and electrodes can name the result.

Parameters:

  • func (function) – The function to apply to each frame in the video. Must accept a 2D or 3D image and return a 2D or 3D image. The returned frames need not have the same shape as the originals (but must all have the same shape as each other); see electrodes.

  • *args – Additional positional arguments passed to the function

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, the original names are carried over whenever func leaves the shape of a frame alone, and the result is named after its place in the new frame otherwise (e.g. for skimage.transform.resize). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in a returned frame.

  • **kwargs – Additional keyword arguments passed to the function

Returns:

stim – A copy of the stimulus object with the new video

Return type:

VideoStimulus

center(loc=None)[source]

Center the image foreground

This function shifts the center of mass (CoM) to the image center.

Parameters:

loc ((col, row), optional) – The pixel location at which to center the CoM. By default, shifts the CoM to the image center.

Returns:

stim – A copy of the stimulus object containing the centered image

Return type:

ImageStimulus

compress()[source]

Compress the source data

Also brings vid_shape back in line with the compressed data: compression drops the time points at which the video does not change, so the frame count of the source is no longer the frame count of the stimulus. Every data.reshape(vid_shape) in this module relies on that invariant. (Compression can also drop all-zero pixels, in which case no shape describes the data any more; see _frames.)

Returns:

compressed

Return type:

VideoStimulus

crop(idx_space=None, idx_time=None, left=0, right=0, top=0, bottom=0, front=0, back=0, electrodes=None)[source]

Crop the video

This method maps a rectangle (defined by two corners) from each video frame to a rectangle of the given size. Similarly, the video can be shortened to a specified range of frames.

Alternatively, this method can be used to crop a number of columns either from the left or the right of the video frame, or a number of rows either from the top or the bottom, or a number of frames from the front (beginning) or back (end) of the video.

Added in version 0.8.

Parameters:

  • idx_space (4-tuple (y0, x0, y1, x1)) – Image indices of the top-left corner [y0, x0] and bottom-right corner [y1, x1] (exclusive) of the rectangle to crop.

  • idx_time (tuple (t0, t1)) – Frame indices defining the start t0 and end t1 of the cropped video.

  • left (int) – Number of columns to crop from the left of each video frame

  • right (int) – Number of columns to crop from the right of each video frame

  • top (int) – Number of rows to crop from the top of each video frame

  • bottom (int) – Number of rows to crop from the bottom of each video frame

  • front (int) – Number of frames to crop from the front (beginning) of the video

  • back (int) – Number of frames to crop from the back (end) of the video

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the cropped image.

Returns:

stim – A copy of the stimulus object containing the video

Return type:

VideoStimulus

property data

Stimulus data container A 2-D NumPy array that contains the stimulus data, where the rows denote electrodes and the columns denote points in time.

property dt

Sampling time step (ms)

Defines the duration of the signal edge transitions.

Added in version 0.7.

property duration

Stimulus duration (ms)

property electrodes

Electrode names A list of electrode names, corresponding to the rows in the data container.

encode(amp_range=(0, 50), freq=20, implant=None, **kwargs)[source]

Encode the video using amplitude modulation

Encodes every frame of the video as a train of biphasic pulses, where the gray level of a pixel sets the amplitude of its pulses. Each train lasts one frame period.

This is a shorthand for AmplitudeEncoder; use that directly for the full set of options.

Changed in version 0.10.0: Gray levels now map onto amp_range absolutely rather than being stretched to fill it (pass stretch=True for the old behavior), each frame receives a pulse train rather than a single pulse, and implant encodes at electrode rather than pixel resolution.

Parameters:

  • amp_range ((min_amp, max_amp), optional) – Range of pulse amplitudes (uA). A gray level of 0 maps onto min_amp, a gray level of 1 onto max_amp.

  • freq (float, optional) – Pulse train frequency (Hz).

  • implant (ProsthesisSystem, optional) – If given, the video is first sampled at the implant’s electrode locations, so that the pulse trains are built at electrode rather than pixel resolution. Strongly recommended: a video has orders of magnitude more pixels than an implant has electrodes.

  • **kwargs – Additional arguments passed to AmplitudeEncoder.

Returns:

stim – Encoded stimulus

Return type:

Stimulus

filter(filt, **kwargs)[source]

Filter each frame of the video

Parameters:

  • filt (str) –

    Image filter that will be applied to every frame of the video. Additional parameters can be passed as keyword arguments. The following filters are supported:

    • ’sobel’: Edge filter the image using the Sobel filter.

    • ’scharr’: Edge filter the image using the Scarr filter.

    • ’canny’: Edge filter the image using the Canny algorithm. You can also specify sigma, low_threshold, high_threshold, mask, and use_quantiles.

    • ’median’: Return local median of the image.

  • **kwargs – Additional parameters passed to the filter

Returns:

stim – A copy of the stimulus object with the filtered image

Return type:

VideoStimulus

invert()[source]

Invert the gray levels of the video

Returns:

stim – A copy of the stimulus object with all grayscale values inverted in the range [0, 1].

Return type:

VideoStimulus

property is_charge_balanced

Flag indicating whether the stimulus is charge-balanced

A stimulus with a time component is considered charge-balanced if its net current is smaller than 10 pico Amps. For the whole stimulus to be charge-balanced, every electrode must be charge-balanced as well.

Returns None if the stimulus is not a current at all: the gray levels of an ImageStimulus integrate to a number like any others, but that number is not a charge and asking whether it is zero answers nothing. Note that this is “not applicable”, not “unbalanced” – it is safe_mode that turns the question into an error, since a safety system genuinely cannot do its job on a stimulus that is not electrical.

Changed in version 0.10.0: Returns None for a stimulus that is not measured in units of current (was: integrated the values anyway).

property is_compressed

Flag indicating whether the stimulus has been compressed

Read-only: the flag is maintained by compress. Assigning to it raises an AttributeError.

play(fps=None, repeat=True, annotate_time=True, ax=None, fmt='jpg')[source]

Animate the video as HTML with JavaScript

The video will be played in an interactive player in IPython or Jupyter Notebook.

Parameters:

  • fps (float or None) – If None, uses the video’s time axis. Not supported for non-homogeneous time axis.

  • repeat (bool, optional) – Whether the animation should repeat when the sequence of frames is completed.

  • annotate_time (bool, optional) – If True, the time of the frame will be shown as t = X ms in the title of the panel.

  • ax (matplotlib.axes.AxesSubplot, optional) – A Matplotlib axes object. If None, will create a new Axes object

  • fmt ({'jpg', 'png'}, optional) –

    The image format used to embed the frames. ‘jpg’ keeps notebooks and doc pages an order of magnitude smaller; use ‘png’ if you need the frames to be pixel-exact.

    Added in version 0.10.0.

Returns:

ani – A Matplotlib animation object that will play the video frame-by-frame.

Return type:

pulse2percept.utils.HTMLAnimation

Notes

Changed in version 0.10.0: The HTML player is now generated by HTMLAnimation, which renders the figure once and ships all frames as a single sprite sheet. This is roughly two orders of magnitude faster than Matplotlib’s to_jshtml and produces much smaller notebooks and doc pages.

plot(electrodes=None, time=None, fmt='k-', ax=None)[source]

Plot the stimulus

Added in version 0.7.

Parameters:

  • electrodes (int, string, or list thereof; optional, default: None) – The electrodes for which to plot the stimulus. If None, all electrodes are plotted.

  • time ((t_min, t_max) tuple, slice, or list of exact time points) – The time points at which to plot the stimulus. Specify a range of time points with a tuple or a slice, or specify the exact time points to interpolate. If None, all time points are plotted.

  • fmt (str, optional, default: 'k-') – A Matplotlib format string; e.g., ‘ro’ for red circles.

  • ax (matplotlib.axes.Axes or list thereof; optional, default: None) – A Matplotlib Axes object or a list thereof (one per electrode to plot). If None, a new Axes object will be created.

Returns:

axes – Returns one matplotlib.axes.Axes per electrode

Return type:

matplotlib.axes.Axes or np.ndarray of them

property quantity

The stimulus data, with its unit attached

Added in version 0.10.0.

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA
>>> Stimulus([500, 1000] * uA).quantity
[[ 500.]
 [1000.]] uA
remove(electrodes)[source]

Remove electrode(s)

Removes the stimulus of a certain electrode or list of electrodes.

Added in version 0.8.

Parameters:

electrodes (int, string, or list of int/str) – The item(s) to remove from the stimulus. Can either be an electrode index, electrode name, or a list thereof.

resize(shape, electrodes=None, **kwargs)[source]

Resize the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • shape ((rows, cols)) – Shape of each frame in the resized video. If one of the dimensions is set to -1, its value will be inferred by keeping a constant aspect ratio.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in the resized video.

  • **kwargs – Additional keyword arguments passed to skimage.transform.resize, such as order=0 for nearest-neighbor interpolation (which keeps a binary video binary).

Returns:

stim – A copy of the stimulus object containing the resized video

Return type:

VideoStimulus

rewind()[source]

Rewind the iterator

rgb2gray(electrodes=None)[source]

Convert the video to grayscale

Parameters:

electrodes (int, string or list thereof; optional) –

Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

Note

The number of electrode names provided must match the number of pixels in the grayscale video.

Returns:

stim – A copy of the stimulus object with all RGB values converted to grayscale in the range [0, 1].

Return type:

VideoStimulus

rotate(angle, mode='constant', electrodes=None, **kwargs)[source]

Rotate each frame of the video

Changed in version 0.10.0: Keyword arguments are passed on to scikit-image.

Parameters:

  • angle (float) – Angle by which to rotate each video frame (degrees). Positive: counter-clockwise, negative: clockwise

  • mode (str, optional) – How to fill in the corners the rotation leaves empty; see skimage.transform.rotate.

  • electrodes (int, string or list thereof; optional) – Optionally, you can provide your own electrode names. If none are given, each pixel keeps the name it had before the rotation, unless resize=True grew the frame, in which case the enlarged video is named after its own pixel grid. See ElectrodeNames.

  • **kwargs – Additional keyword arguments passed to skimage.transform.rotate, such as order, cval, or resize=True to grow each frame so that it contains every rotated pixel.

Returns:

stim – A copy of the stimulus object containing the rotated video

Return type:

VideoStimulus

scale(scaling_factor)[source]

Scale the image foreground

This function scales the image foreground (excluding black pixels) by a factor.

Parameters:

scaling_factor (float) – Factory by which to scale the image

Returns:

stim – A copy of the stimulus object containing the scaled image

Return type:

ImageStimulus

property shape

Data container shape

shift(shift_cols, shift_rows)[source]

Shift the image foreground

This function shifts the center of mass (CoM) of the image by the specified number of rows and columns.

Parameters:

  • shift_cols (float) – Number of columns by which to shift the CoM. Positive: to the right, negative: to the left

  • shift_rows (float) – Number of rows by which to shift the CoM. Positive: downward, negative: upward

Returns:

stim – A copy of the stimulus object containing the shifted image

Return type:

ImageStimulus

property time

Time steps A list of time steps, corresponding to the columns in the data container.

property time_quantity

The stimulus time axis with its unit attached, or None

Added in version 0.10.0.

property time_unit

The unit time is expressed in (milliseconds)

Added in version 0.10.0.

times(unit=None)[source]

The stimulus time axis, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the time axis in. If None, time is returned as it is stored (milliseconds).

Returns:

times – An ordinary NumPy array, or None if the stimulus has no time component.

Return type:

np.ndarray or None

trim(tol=0, electrodes=None)[source]

Remove any black border around the video

Added in version 0.7.

Parameters:

  • tol (float) – Any pixels with gray levels > tol will be trimmed.

  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, each pixel is named after its place in the image (e.g. ‘A1’, ‘C12’, ‘A1_R’). See ElectrodeNames.

    Note

    The number of electrode names provided must match the number of pixels in each frame of the trimmed video.

Returns:

stim – A copy of the stimulus object with trimmed borders.

Return type:

VideoStimulus

property unit

The unit data is expressed in

Microamps for an electrical stimulus, dimensionless for the gray levels of an ImageStimulus or VideoStimulus.

Read-only. The canonical storage unit is fixed so that models, safety checks and Cython kernels can rely on it; ask for another unit with values().

Added in version 0.10.0.

values(unit=None)[source]

The stimulus data, expressed in unit

Added in version 0.10.0.

Parameters:

unit (Unit, optional) – The unit to express the data in. Must be compatible with unit. If None, the stimulus’ own unit is used and data is returned as it is stored.

Returns:

values – An ordinary NumPy array, never a Quantity. This is the boundary a numerical implementation should take its data across.

Return type:

np.ndarray

Examples

>>> from pulse2percept.stimuli import Stimulus
>>> from pulse2percept.units import uA, mA
>>> Stimulus([500, 1000] * uA).values(mA)
array([[0.5],
       [1. ]], dtype=float32)