Coordinates and Visual Field Maps

A simulation uses four coordinate systems, so device geometry, anatomy, and the visual world can be specified independently.

Frame	Unit	Origin and meaning
Device	um	Device-local. The array’s own origin; identical for every copy of a device, wherever it is implanted.
Tissue	um	Retinal or cortical position, measured from the fovea or its cortical representation.
Visual field	dva	Eye-centered. The fovea is `(0, 0)`; this is where phosphenes and a `Scotoma` live.
Scene	dva	Fixed to the world in front of the eye; related to the visual field by gaze.

dva and um convert only through a VisualFieldMap, never directly (see Physical Units).

Device to tissue: placement

Electrode coordinates are device-local. The model places the device:

import pulse2percept as p2p
from pulse2percept.units import dva, mm

model = p2p.models.retina.AxonMapModel(
    p2p.implants.retina.ArgusII(),
    implant_position=(2, -1) * dva,
    implant_rotation=15,
)

implant_position: Position of the device-local origin, in tissue coordinates or dva.
implant_rotation: In-plane rotation (deg), positive counter-clockwise.
implant_depth: Signed offset (um) along the normal of a 2D tissue map.

Retinal positions are measured from the fovea. Cortical positions are measured from the foveal representation of the right hemisphere; the left hemisphere is offset by left_offset (default -20 mm) along x.

Tissue to visual field: the maps

A VisualFieldMap converts between visual field and tissue: dva_to_ret / ret_to_dva for retinal maps, dva_to_v1 / v1_to_dva (and V2, V3) for cortical maps. The inverse exists only where the mapping is invertible.

from pulse2percept.topography.retina import Watson2014Map

x_um, y_um = Watson2014Map().dva_to_ret(2 * dva, 3 * dva)

A model holds one map (visual_field_map), samples the visual field on a Grid2D, and maps that grid onto tissue during build.

Retinal maps

Retinal maps derive from RetinalMap.

Map	Reference	Retinal magnification
`Curcio1990Map`	[Curcio1990]	Linear, 280 um per dva
`Watson2014Map`	[Watson2014]	Nonlinear; the default for retinal models
`Montesano2020Map`	[Montesano2020]	Watson2014 plus a 2D, meridian-dependent RGC displacement field

A regular grid in the visual field therefore lands differently on the retina under each map:

Montesano2020Map also separates the position of a ganglion cell body from that of its receptive field, which differ within about 15 dva of the fovea:

The displacement zone reaches 14.1 dva temporally and superiorly, 10.5 dva inferiorly, and 9.5 dva nasally.
Displacement is reconstructed from [Montesano2020] in dva, then converted to microns with Watson2014Map for consistency with the other retinal maps.
It is population-average anatomy ([Montesano2020], [Curcio1990]), not subject-specific.
The deprecated Watson2014DisplaceMap uses horizontal-meridian fits across entire hemifields and has no inverse.

Laterality

The visual field is mirrored on the retina. In a right eye, a retinal meridian lies at the negated visual-field polar angle: nasal retina maps to the temporal visual field. eye on a RetinalImplant sets where models place the optic disc. Montesano2020Map takes its own eye, because its displacement field depends on the meridian; the left-eye map is the horizontal mirror of the right-eye map.

Cortical maps

Cortical maps derive from CorticalMap and can cover v1, v2 and v3.

Map	Reference	Description
`Polimeni2006Map`	[Polimeni2006]	Wedge-dipole model of V1-V3; the default for cortical models
`NeuropythyMap`	[Benson2018]	Subject-specific retinotopy estimated from MRI

Polimeni2006Map has six parameters: global scale k, wedge-dipole parameters a and b, and azimuthal shear alpha1, alpha2, alpha3 for V1-V3. Defaults are from [Polimeni2006]; individual retinotopy varies substantially.

NeuropythyMap gives subject-specific retinotopy. It requires the optional neuropythy package (pip install neuropythy) and a subject: 'fsaverage', a Benson-Winawer 2018 subject ('S1201'-'S1208'), or a FreeSurfer subject directory. Subject data is downloaded on first use. It also takes a cortical surface ('midgray' by default, 'white', or 'pial'); its points are 3D, so plot them with plot3d().

nmap = p2p.topography.cortex.NeuropythyMap(subject='fsaverage',
                                           regions=['v1'])
model = p2p.models.cortex.ScoreboardModel(
    implant=p2p.implants.cortex.NeuroPortArray(),
    visual_field_map=nmap, regions=['v1'])

Subject-specific phosphene locations

A map gives the canonical visual-field location of each electrode; measured phosphenes scatter around it. location_noise (dva) adds a fixed random offset per electrode:

\[\mathbf{p}'_i = \mathbf{p}_i + \boldsymbol{\epsilon}_i, \qquad \boldsymbol{\epsilon}_i \sim \mathcal{N}(0, \sigma^2 I),\]

where \(\mathbf{p}_i\) is the canonical location and \(\sigma\) is location_noise. Offsets are drawn once per model instance.

Retinal phosphenes move without changing shape. On cortex, a displaced electrode samples a different cortical magnification, so its phosphene size changes too. Electrode positions and the map are unchanged. location_noise requires an invertible map.

Visual field to scene: gaze

Scene coordinates are eye-centered coordinates plus gaze. Each electrode follows this chain:

device coordinate (um)
  -> implant_position / rotation / depth -> tissue coordinate (um)
  -> visual_field_map.ret_to_dva -> eye-centered visual field (dva)
  -> + gaze, for eye-coupled input only -> scene coordinate (dva)

See Visual Input and Simulated Vision for what gaze moves.

Custom maps

A custom retinal map subclasses RetinalMap and implements dva_to_ret. location_noise and scenes also require ret_to_dva.

class MyVisualFieldMap(p2p.topography.retina.RetinalMap):

    def dva_to_ret(self, xdva, ydva):
        return xdva, ydva

    def ret_to_dva(self, xret, yret):
        return xret, yret

Pass it to a model with visual_field_map=MyVisualFieldMap().