Metadata-Version: 2.4
Name: roiapp
Version: 0.1.0
Summary: Python port of the MATLAB 2P-imaging ROI selector GUI (app2.mlapp), bundled with movieslider (Python port of the MovieSlider playback/scrubbing widget, used by the 'Play Movie' button)
Author-email: Adam <adamshch@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/adamshch/NeuroImaging_ROIapp
Project-URL: Repository, https://github.com/adamshch/NeuroImaging_ROIapp
Keywords: neuroscience,calcium-imaging,two-photon,roi,microscopy,PySide6
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PySide6>=6.6
Requires-Dist: pyqtgraph>=0.13
Requires-Dist: numpy>=1.26
Requires-Dist: scipy>=1.11
Requires-Dist: scikit-image>=0.22
Requires-Dist: h5py>=3.10
Requires-Dist: hdf5storage>=0.1.19
Requires-Dist: pymatreader>=0.0.32
Requires-Dist: tifffile>=2024.1.30
Requires-Dist: astropy>=6.0
Requires-Dist: imageio>=2.34
Requires-Dist: imageio-ffmpeg>=0.4.9
Requires-Dist: matplotlib>=3.8
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-qt>=4.4; extra == "dev"
Dynamic: license-file

# ROIapp (Python)

Python port of the MATLAB App Designer ROI-selector GUI
(`ROIapp_matlab/roiSelectApp-main/app2.mlapp`) used to hand-curate ROIs (cell
footprints) from 2-photon calcium-imaging movies: click a pixel, grow a mask
via correlation- or fluorescence-based region growing, review/dilate/contract
it, stage it into a "temp" and then a "curated" set, and save the result as a
`.mat` file for the broader MATLAB analysis pipeline (`analyze_ASC-utils`).

`app1.mlapp` (a strict subset of app2) was not ported separately -- app2's
functionality is a superset, so nothing is lost.

This single pip package also bundles `movieslider` (Python port of the
MATLAB `MovieSlider` widget -- smooth movie playback/scrubbing/contrast),
which backs the **Play Movie** button (see "Run" below). It lives at the
top-level `movieslider/` package alongside `roiapp/`, built and installed
together from this one `pyproject.toml` -- originally developed as the
standalone `MovieSlider_python` project, later merged in here so
`pip install` gets everything in one step.

## Setup

```bash
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"
```

Requires system support for `python3-venv` (Debian/Ubuntu:
`sudo apt install python3.13-venv`, adjusted for your Python version) and a
Qt-capable environment (a real display, or `QT_QPA_PLATFORM=offscreen` for
headless use).

## Run

```bash
.venv/bin/roiapp        # console script
python3 -m roiapp        # equivalent, module form
```

`movieslider`'s own standalone viewer (File > Open, independent of ROIapp)
is also installed as a second console script / module:
`.venv/bin/movieslider` or `python3 -m movieslider`.

Use the **Load File** / **Load Folder** buttons to open a movie:

- **Load File** autodetects TIFF (`.tif`/`.tiff`), FITS (`.fits`/`.fit`/`.fts`),
  or MAT (`.mat`) by extension, falling back to magic-byte sniffing for
  unrecognized extensions. A `.mat` file's largest top-level 3-D numeric
  array is taken as the movie.
- **Load Folder** treats every TIFF file in the chosen folder as one frame
  (or chunk of frames) of a single movie, concatenated in natural
  (digit-aware) filename order.

See `roiapp/io/movie_loader.py` for the loader implementation. This replaces
the original MATLAB app's `MONKEYDIR`/`Day`/`Rec`/"Data to load" dropdown
convention, which assumed a specific lab directory layout -- the Python port
instead loads whatever file or folder the user points it at.

Mean/median/variance/Fano-factor/95th-percentile/local-correlation
projection images are computed from the loaded movie and cached next to it
(a `<name>.roiapp_projections.mat` sidecar, invalidated automatically if the
source file/folder changes -- see `roiapp/io/projection_cache.py`).
"Save ROIs" writes an auto-incrementing `<name>.manual_rois_v<N>.mat` next to
a file source, or inside a folder source (named after the folder).

**Play Movie** opens the loaded movie in a `MovieSlider_python` viewer
window (`movieslider.gui.movie_slider_widget.MovieSliderWidget`, that
project's smooth-playback/scrubbing/contrast widget) -- `gui/main_window.py`'s
`_on_play_movie` reuses a single player window across clicks rather than
opening a new one each time, and warns instead if no movie is loaded yet.

The neuropil ring size used for background subtraction (previously inferred
from a `MONKEYDIR` path substring in the MATLAB app) is now an explicit
Small/Large control next to "Bound radius". Each segmentation tab's
"Thresh. Auto Method" also has a "None (manual)" option (beyond app2.m's
own two) that keeps a manually-set threshold sticky across clicks, instead
of app2.m's default behavior of silently resetting to auto-threshold-search
on every fresh click.

**Debugging aid:** the "Image view" dropdown includes a "Local Corr" option
(no MATLAB equivalent) showing, per pixel, the Pearson correlation of its
own timeseries with the mean timeseries of its up-to-8 spatial neighbors --
a standard technique (also used by Suite2p/CaImAn) for visually surfacing
real cell footprints, useful for finding good seed pixels when Corr-tab
auto-segmentation looks too aggressive. The left control panel has a
colorbar (below the ROI counts) showing the exact value scale for whatever
projection/contrast is currently displayed -- shared between both FOV
panels (`gui/colorbar_panel.py`) rather than duplicated per panel, since
temp/curated always show the same projection and levels. This also
sidesteps a real layout problem an earlier, per-panel-embedded colorbar
had: pairing a colorbar with an aspect-locked ViewBox in the same
pyqtgraph layout row means lockAspect's letterboxing/pillarboxing (for a
non-square image, or a panel whose cell isn't the same aspect as the
image) leaves the colorbar visibly taller than the actually-displayed
image, and attempts to fix that by shrinking the shared row shrank the
image too (see `gui/colorbar_panel.py`'s module docstring for the full
history) -- living in its own widget outside any image's layout row
removes the conflict entirely, and each FOV panel is now image-only,
using the full available space.

The Corr tab also has a **"3x3 block seed"** checkbox (no MATLAB
equivalent): by default, a click correlates every nearby pixel against the
*single* clicked pixel's raw timeseries, exactly matching
`corrBasedMasks_new.m`; checking this instead averages a 3x3 block centered
on the click into the seed trace first. A single pixel's raw trace can
decorrelate with distance much faster than the (neighbor-averaged) Local
Corr view suggests it should -- the block-averaged seed is less noisy and
recovers more of a cell's true extent at the same threshold (see
`algorithms/corr_based_masks.py`'s `_extract_seed_trace`).

Building this surfaced one more real bug: the single-clicked-pixel seed's
self-correlation is always exactly 1.0, so the auto-threshold sweep's
lowest candidate (0.6) always captures at least that one pixel -- but a
block-averaged seed has no such guarantee, and on weakly-correlated data
every candidate's mask can legitimately come back empty, making the
auto-threshold objective all-`NaN`. `np.nanargmax` raises on an all-`NaN`
slice; real MATLAB's `max([NaN,NaN,NaN])` instead returns `ind=1` (no
error) -- confirmed directly against MATLAB R2025a. This previously crashed
the click handler silently (Qt swallows slot exceptions by default, so it
looked exactly like nothing happened), fixed via
`algorithms/seed_region.py`'s `matlab_style_nanargmax` and a defensive
error dialog now wrapping `MainWindow._on_pixel_clicked`.

Even a block-averaged seed is still just **one fixed reference point** --
correlation with it necessarily decays with distance, no matter how clean
the seed trace itself is, which is structurally different from Local Corr
(a moving, per-pixel reference that only requires each pixel to correlate
with its own immediate neighbors, letting correlation "chain" across an
extended region). A first attempt at closing that gap added a **"Flood-fill
growth"** checkbox: instead of thresholding one fixed correlation map, it
starts from the seed and iteratively grows -- each round, every pixel
adjacent to the current mask is tested against the *current mask's own mean
trace* (which gets less noisy, not just the seed, as the mask grows), and
whichever pass threshold get added, repeating until nothing new qualifies
(see `algorithms/corr_based_masks.py`'s `_flood_fill_corr_mask`). Growth is
bounded only by the search disk (`max_dist`) -- there's no other limit once
the growing mean becomes a generically clean local signal, so on data with
substantial shared background/neuropil contamination this can in principle
grow to fill the whole disk; that's real, observed behavior to threshold
against, not silently capped. Verified on real data: at the same threshold,
flood-fill recovered 155px vs fixed-seed's 141px (Local Corr's own
thresholded blob at that location: 238px) -- and on a deterministic
synthetic test (an AR(1) spatial correlation chain, where adjacent
positions are always strongly correlated but a fixed reference's
correlation decays geometrically with distance) flood-fill recovers the
*entire* chain while fixed-seed reaches less than half of it.

Flood-fill still visibly disagreed with Local Corr in practice, though --
its growing reference is still fundamentally *one* trace (even if it
evolves), not Local Corr's fully local, per-pixel one. So the Corr tab's
**default** growth method is now `local_corr_threshold`: it thresholds the
already-computed Local Corr image *directly* and takes the connected
component containing the click (reusing the same island-removal/hole-fill
logic as every other method) -- no new correlation is computed from the
click at all. This reproduces exactly what the Local Corr debugging view
shows, by construction (verified: the resulting mask is byte-for-byte the
thresholded, click-connected blob of the Local Corr image, at every
threshold tested). `fixed_seed` is no longer reachable from the GUI (only
via direct API calls, e.g. tests wanting strict MATLAB fidelity); the
"Flood-fill growth" checkbox now toggles between `local_corr_threshold`
(unchecked, default) and `flood_fill` (checked).

## Test

```bash
.venv/bin/pytest
```

Tests run fully headless (`QT_QPA_PLATFORM=offscreen`, set automatically in
`tests/conftest.py`) against a small, deterministic synthetic 2P movie with
known ground-truth cell footprints (`roiapp/testing_utils/synthetic_data.py`,
written out as synthetic TIFF/FITS files/folders for the loader tests) --
no real data is required.

## Architecture

- `roiapp/model/` -- pure dataclasses (`AppState`, `ROICollection`,
  `ROIParams`, ...), no Qt or file I/O dependency.
- `roiapp/algorithms/` -- numpy/scipy ports of the MATLAB segmentation math
  (`cleanROI.m`, `corrBasedMasks_new.m`, `fluorBasedMasks_new.m`,
  `dilate_constrict_ROI.m`, neuropil ring construction, trace extraction),
  unit-testable without Qt or files.
- `roiapp/io/` -- movie loading (`movie_loader.py`: TIFF/FITS/MAT
  autodetection, folder-of-TIFFs concatenation), projection-image caching
  (`projection_cache.py`), `.mat` read/write (including full `-v7.3`
  struct/cell-array round-tripping), Suite2p import, legacy and modern
  `manual_rois*.mat` load/save.
- `roiapp/gui/controller.py` -- `AppController`: the full app2.m callback
  set as headless, Qt-free business logic, exercised directly by pytest.
- `roiapp/gui/*.py` (everything else) -- PySide6 + pyqtgraph widgets and the
  signal wiring connecting them to `AppController`.
- `movieslider/` -- a second, independent GUI package (Python port of the
  MATLAB `MovieSlider` widget) bundled in this same distribution; `roiapp`
  only touches it at one point, `gui/main_window.py`'s `_on_play_movie`.
  See `movieslider/`'s own module docstrings for its architecture
  (`model/`, `playback/`, `algorithms/`, `io/`, `gui/` mirror the same
  layering convention as `roiapp/`'s). Its tests live under
  `tests/movieslider/` rather than mixed into `tests/gui/`, `tests/io/`,
  etc., to avoid filename collisions with `roiapp`'s own tests (both
  packages have an `io/movie_loader.py`, for instance).

## Known deliberate deviations from the MATLAB source

Found while tracing exact source semantics; each documented at its call site
and covered by a regression test:

- **The FOV display's mean/median/var/fano/pct95/local-corr transpose**
  (`algorithms/projections.py`): app2.m displays `app.data.mean` (etc.)
  transposed relative to the raw movie, but feeds a mouse click's
  coordinates from that transposed display straight into
  `corrBasedMasks_new`/`fluorBasedMasks_new`, which index the *untransposed*
  raw movie directly -- so clicking a cell that's clearly visible on screen
  seeds the correlation/fluorescence search at the wrong pixel unless the
  image happens to be symmetric about its diagonal (never true for real
  data). Confirmed directly against real MATLAB with a synthetic asymmetric
  test image. This affects the **existing MATLAB app too**, not just the
  port -- fixed here by simply not replicating the transpose, so the FOV
  display, mouse clicks, mask growth, and overlay rendering all share one
  coordinate system with no compensating swap needed anywhere. See
  "A real bug this caught" below for the full story.
- **`selectROI`'s temp/curated trace mixup** (`gui/controller.py`): app2.m,
  when updating an *existing* temp ROI, wrote the recomputed trace into
  `curROIs.Fall`/`Fneu` instead of `tempROIs.Fall`/`Fneu` -- a copy-paste bug
  that can silently corrupt an unrelated curated ROI's saved trace. This
  affects the **existing MATLAB app too**, not just the port.
- **`deleteCurROI`'s sourcemat color-erase** (`gui/controller.py`): missing
  the explicit `color=[0,0,0]` argument its sibling `deleteTempROI` uses --
  cosmetic only (sourcemat is a display aid, never saved).
- **The current-mask overlay's alpha scale** (`gui/overlay_render.py`): a
  mask virtually always covers <1% of a full frame, making
  `scale = 0.5/quantile(scaled,0.99)` legitimately `Inf` (clamped to fully
  opaque) in the overwhelmingly common case -- handled via
  `nan_to_num(posinf=1.0, nan=0.0)` rather than a naive divide-by-zero guard.
- A few MATLAB source quirks are preserved *faithfully* rather than "fixed"
  since they don't corrupt data (e.g. `cleanROI`'s area-band component
  filtering is position-independent, not distance-based; the fluor
  segmentation's `cellnpdecorr` objective is `abs(correlation)` while the
  corr segmentation's is `-correlation`; state-button dilate/contract fires
  on both press and release) -- see the relevant module docstrings for the
  full list.

## Real-data validation (Phase 7)

Validated against a real 500x500x500 (uint16) 2-photon movie and, in an
environment with MATLAB R2025a available, directly against the **original
MATLAB source** (`corrBasedMasks_new.m`, `fluorBasedMasks_new.m`,
`computeImageMean.m`, `strel('sphere',r)`+`imdilate`/`imerode`, `prctile`),
not just Python-side round-tripping:

- **`.mat` v7.3 write fidelity**: a Python-saved `manual_rois_v2.mat`
  (real 500x500x5 masks, Nx1 cell-array-of-structs `params`, nested
  cell-array `roi_history`) loads in real MATLAB with every field intact.
- **`strel('sphere', r)` vs `algorithms/morphology.py`'s explicit
  Euclidean-disk footprint**: pixel-for-pixel identical to
  `imdilate`/`imerode` for r in {1,2,3,4,8} (both dilate and erode).
- **`prctile`/`quantile` vs `numpy.percentile(..., method='hazen')`**:
  identical to floating-point precision (1e-14).
- **`corrBasedMasks_new.m` vs `algorithms/corr_based_masks.py`**: pixel-for-
  pixel identical mask on real data at an explicit threshold.
- **`fluorBasedMasks_new.m` vs `algorithms/fluor_based_masks.py`**:
  pixel-for-pixel identical mask on real data, *after* fixing a real bug
  this comparison caught (below).

### Two real bugs this caught, one leading to the other

**First**, an earlier version of this port initially returned the raw,
untransposed projection values, then was "corrected" to match `app2.m`'s
`LoaddataButtonPushed`, which calls `computeImageMean(...)'` (and
identically for median/var/fano/pct95) -- an explicit trailing transpose, in
*both* the cache-hit and fresh-compute branches -- after confirming that
transposed value matches real MATLAB's `app.data.mean` to floating-point
precision. Every synthetic test passed throughout, because a global
transpose applied consistently doesn't break Python-to-Python
self-consistency checks; it only showed up as a real mismatch against
actual MATLAB output on real, spatially-asymmetric imaging data.

**Second**, much later, matching that transpose byte-for-byte turned out to
be the wrong call: `app2.m`'s `mouseClick` reads a click's `currentLoc` from
that same transposed `app.data.mean` display, then feeds it *directly* into
`corrBasedMasks_new`/`fluorBasedMasks_new`, which index the **untransposed**
`datamap.Data(1).IMG` with it -- so clicking a cell that's clearly visible
on screen seeds the correlation/fluorescence search at the *wrong* raw
pixel, unless the image happens to be symmetric about its diagonal (never
true for real data). Confirmed directly against real MATLAB: a synthetic
cell placed at raw (row 11, col 51) appeared on MATLAB's own displayed
`app.data.mean` at (row 50, col 12), and clicking there (mirroring
`mouseClick`'s real behavior) produced a degenerate 1-pixel mask in *both*
MATLAB and this port -- this is a genuine bug in the original MATLAB app,
not something the port introduced, and it explains why real interactive use
of the Corr tab could feel broken (a click on an obvious cell landing on
essentially noise instead). Fixed by reverting to the untransposed value
project-wide: the FOV display, `on_pixel_click`, mask growth, and overlay
rendering now all share one coordinate system, so no compensating swap is
needed anywhere. Trade-off: the displayed FOV image is a mirror of what
real MATLAB shows on screen for the same file (movie data and every
algorithm's math are unaffected -- only display/click orientation) -- a
deliberate choice, matching a buggy display convention isn't worth
inheriting the interactivity bug it causes. See
`algorithms/projections.py`'s module docstring for the full account, and
`tests/gui/test_controller.py::test_clicking_where_a_cell_visually_appears_on_the_displayed_fov_finds_it`
for the regression test (deliberately clicks where a cell *appears on the
display*, not at its known ground-truth location, since every other test
bypasses the display and would not have caught this).

### Also discovered (not a port issue, informational)

- `computeCurrentTraces.m` (the standalone file used by both segmentation
  functions' auto-threshold search loops) hardcodes `IMG(...,1:1000)`.
  Confirmed by direct MATLAB execution: **this crashes app2.m itself**
  ("Index exceeds array bounds") whenever a user auto-segments on a
  recording with fewer than 1000 frames. This port's equivalent
  (`algorithms/traces.py`) uses ordinary Python slicing, which truncates
  gracefully instead of erroring -- a robustness improvement, not a
  fidelity gap, since there's no sensible MATLAB behavior to match here.
- The `cellnpdecorr` auto-threshold method can degenerate to a 1-2 pixel
  mask on real data with denser neuropil contamination (confirmed
  mathematically correct per the ported formula, and this is inherent to
  the objective itself -- maximizing decorrelation from the surround can
  trivially "win" by shrinking to the single most cell-specific pixel).
  `cellnpdiff` produced a much better-sized mask on the same real test
  point. Worth knowing if auto-segmentation looks too aggressive in
  practice -- try the other auto method or a manual threshold.

### Still not validated (no real files available for these specifically)

- A real Suite2p `Fall.mat` and a real legacy (pre-app2) `manual_rois.mat`.

This validation was originally run through the retired `MONKEYDIR`/`Day`/`Rec`
flow and re-confirmed after the data-loading redesign (Load File/Load
Folder, see "Run" above) by loading the same real TIFF directly through
`AppController.load_movie()`: the mean projection, a corr-based mask, and a
fluor-based mask all still match the real-MATLAB references above to
floating-point/pixel precision (once accounting for the old flow's
`nfr - 50` trailing-frame trim, which the new loader deliberately does not
apply -- see `io/movie_loader.py`). The segmentation/projection math is
unchanged; only how a movie gets opened changed.
