Metadata-Version: 2.4
Name: opal-studio
Version: 0.1.5
Summary: Cross-platform viewer and analysis application for multiplexed (IMC) and H&E OME-TIFF images
Author-email: Tristan Whitmarsh <tw401@cam.ac.uk>
License: MIT + Commons Clause
Project-URL: Homepage, https://github.com/TristanWhitmarsh/opal-studio
Project-URL: Source, https://github.com/TristanWhitmarsh/opal-studio
Project-URL: Issues, https://github.com/TristanWhitmarsh/opal-studio/issues
Project-URL: Models, https://github.com/TristanWhitmarsh/opal-studio/releases/tag/models-v1
Keywords: imaging mass cytometry,IMC,multiplexed imaging,H&E,OME-TIFF,cell segmentation,phenotyping,microscopy
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: <3.11,>=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PySide6==6.4.3
Requires-Dist: tifffile==2024.8.30
Requires-Dist: imagecodecs==2024.12.30
Requires-Dist: numpy==1.26.4
Requires-Dist: scikit-image==0.24.0
Requires-Dist: pandas==2.3.3
Requires-Dist: tensorflow==2.8.0
Requires-Dist: tensorflow-addons==0.16.1
Requires-Dist: tensorflow-io-gcs-filesystem==0.26.0
Requires-Dist: protobuf==3.20.3
Requires-Dist: spektral==1.0.6
Requires-Dist: torch==2.1.2
Requires-Dist: torchvision==0.16.2
Requires-Dist: einops==0.8.2
Requires-Dist: deepcell==0.12.10
Requires-Dist: deepcell-toolbox==0.12.1
Requires-Dist: deepcell-tracking==0.6.5
Requires-Dist: stardist==0.9.2
Requires-Dist: csbdeep==0.8.2
Requires-Dist: multiplex2brightfield>=0.2.8
Requires-Dist: cellpose==3.1.1.3
Requires-Dist: omnipose==1.0.6
Requires-Dist: instanseg-torch==0.1.1
Requires-Dist: torchvf==0.1.4
Requires-Dist: scipy==1.13.1
Requires-Dist: scikit-learn==1.6.1
Requires-Dist: anndata==0.10.9
Requires-Dist: scanpy==1.10.3
Requires-Dist: phenograph==1.5.7
Requires-Dist: umap-learn==0.5.12
Requires-Dist: leidenalg==0.10.2
Requires-Dist: louvain==0.8.2
Requires-Dist: matplotlib==3.9.4
Requires-Dist: pillow==11.3.0
Requires-Dist: h5py==3.14.0
Requires-Dist: opencv-python-headless==4.11.0.86
Requires-Dist: tqdm==4.67.3
Requires-Dist: natsort==8.4.0
Requires-Dist: psutil==7.2.2
Requires-Dist: zarr==2.18.2
Requires-Dist: numcodecs==0.12.1
Requires-Dist: geopandas==1.0.1
Requires-Dist: shapely==2.0.7
Requires-Dist: pyarrow==21.0.0
Requires-Dist: seaborn==0.13.2
Requires-Dist: peakdetect==1.2
Requires-Dist: mahotas==1.4.18
Requires-Dist: igraph==0.11.9
Requires-Dist: roifile==2024.9.15
Requires-Dist: fastremap==1.18.0
Requires-Dist: fill-voids==2.1.1
Requires-Dist: edt==3.1.1
Requires-Dist: mgen==1.2.1
Requires-Dist: ncolor==1.5.3
Requires-Dist: pywin32==311; sys_platform == "win32"
Requires-Dist: setuptools<70.0.0,>=68.0
Dynamic: license-file

# Opal Studio

<img src="https://raw.githubusercontent.com/TristanWhitmarsh/opal-studio/main/screenshot.jpg" width="100%">

**Opal Studio** is a cross-platform viewer and analysis application for highly multiplexed imaging data, including Imaging Mass Cytometry (IMC), large OME-TIFF files, pyramid TIFF data, and SpatialData/Zarr V3 image directories.

The application combines fast multi-channel image rendering with practical workflows for preprocessing, segmentation, mask refinement, cell positivity, phenotype gating, clustering, and export.

## Quick Start

### Install From PyPI

Opal Studio needs Python 3.9 or 3.10 (TensorFlow 2.8, used by Mesmer, is not built for
newer versions).

```bash
conda create -n opal-env python=3.9
conda activate opal-env
pip install opal-studio
opal-studio
```

### Install From Source

```bash
git clone https://github.com/TristanWhitmarsh/opal-studio.git
cd opal-studio
conda create -n opal-env python=3.9
conda activate opal-env
pip install -r requirements.txt
pip install --no-deps -e .
python -m opal_studio
```

You can also launch an installed package with:

```bash
opal-studio
```

### Create A Desktop Launcher

```bash
python -m opal_studio --create-launcher
```

On Windows this creates an `Opal Studio.lnk` shortcut on the desktop. On Linux it creates an `OpalStudio.desktop` launcher.

### University Server / Darkroom Setup

On the server, Opal Studio is deployed as a source checkout installed into a conda env,
so it can run the latest code rather than the last PyPI release. Install or update
with:

```bash
# 1) Get / update the code
cd /home/tristan/Storage/scratch.space/users/tristan/opal-studio
git pull        # first time instead: git clone https://github.com/TristanWhitmarsh/opal-studio.git .

# 2) Activate the conda env
source /opt/conda/etc/profile.d/conda.sh
conda activate /home/tristan/Storage/scratch.space/envs/opal-env-j4

# 3) Install the pinned dependencies, then the package itself
PIP_REQUIRE_VIRTUALENV=0 pip install -r requirements.txt
PIP_REQUIRE_VIRTUALENV=0 pip install --force-reinstall --no-deps --no-build-isolation .

# 4) Create the desktop launcher
python -m opal_studio --create-launcher
```

Then double-click the **Opal Studio** icon on the desktop.

Notes:
- `pip install -r requirements.txt` installs the full dependency stack; run it on the
  first install and whenever `requirements.txt` changes. (Skipping it is why clustering,
  phenotyping, brightfield, or project-save features may report missing modules.)
- `--force-reinstall --no-deps` reinstalls only the `opal_studio` code — picking up new
  code even when the version number is unchanged — without re-resolving dependencies.
- `--no-build-isolation` builds in place instead of copying the whole checkout to `/tmp`.
- Do **not** launch with `python -m opal_studio` from *inside* the checkout directory —
  that imports the checkout instead of the installed package. Use the desktop launcher,
  the `opal-studio` console script, or run from `~`.

#### Models

Model weights are not part of the package — they come to about 640 MB. They are
downloaded automatically the first time they are needed:

- **Opal Studio's own models** (cell positivity, and the IMC / General segmentation
  models) come from the [`models-v1` release](https://github.com/TristanWhitmarsh/opal-studio/releases/tag/models-v1)
  of this repository.
- **StarDist, Cellpose and InstanSeg** fetch their own pretrained models into their own
  caches, from their own sources.

Downloaded models are stored inside the installed package, in `opal_studio/models/`
under your environment's `site-packages` — next to the code, so nothing is left
elsewhere on the machine. Deleting the `opal_studio` folder deletes its models too.

pip only removes files it installed itself, so downloaded models survive
`--force-reinstall`, and `pip uninstall` leaves the `models/` folder behind in
`site-packages/opal_studio/`. To fetch a model again, delete its folder there; it is
downloaded afresh the next time it is used.

**Working offline, or on a cluster whose compute nodes have no internet:** run
**File → Download All Models** once on a machine that is online (a login node, say).
To keep the models somewhere else — shared storage, for instance — set
`OPAL_STUDIO_MODELS` to a directory before starting Opal Studio:

```bash
export OPAL_STUDIO_MODELS=/home/tristan/Storage/scratch.space/opal-models
```

To use a model you trained yourself, place its folder under
`<models>/<engine>/<name>/` (for example `stardist/MyModel/`) and it will appear in
that engine's model list. See [MODELS.md](https://github.com/TristanWhitmarsh/opal-studio/blob/main/MODELS.md) for every model, its source and
its licence.

## Supported Data

Opal Studio can open:

- OME-TIFF and TIFF image files, including multichannel and pyramidal data.
- SpatialData directories containing Zarr V3 image groups under `images/`.
- RGB-style TIFF series for brightfield/H&E viewing.
- Imported mask and cell label maps from OME-TIFF files.
- Phenotyping definitions from CSV files.

For OME-TIFF, channel names are read from OME metadata when available. For SpatialData, channel names are read from OME-Zarr metadata and can be enriched from `extras/mcd_schema.xml` when present.

## Project Format (SpatialData)

`File > Save Project` writes the whole session as a spec-compliant **SpatialData**
store (Zarr v3, OME-NGFF `0.5-dev-spatialdata`, store format `0.2`). The stores
Opal writes can be opened directly by the `spatialdata` / `squidpy` ecosystem —
Opal does not depend on the `spatialdata` package, it drives the underlying
standards (Zarr v3, GeoParquet, AnnData) directly.

Segmentation masks become `labels`, hand-drawn regions become `shapes`, processed
channels and the generated brightfield become `images`, and the per-cell table
becomes `tables` (linked to the cell mask via `region` / `region_key` /
`instance_key`, so e.g. `obs['region'].value_counts()` gives cells-per-region).

```
project.zarr/
├── zarr.json                      # root group: spatialdata_attrs (v0.2) +
│                                  #   opal_studio {source_image, session} +
│                                  #   consolidated_metadata (flat list of all nodes)
│
├── images/
│   ├── zarr.json                  # group
│   ├── derived/                   # one OME-NGFF image element per key
│   │   ├── zarr.json              #   ome (multiscales; axes c,y,x; omero channels;
│   │   │                          #        coordinateTransformations) + spatialdata_attrs v0.3
│   │   └── 0/                     # scale-0 array (C,Y,X)
│   │       ├── zarr.json
│   │       └── c/ …               # chunk files  (c/<c>/<y>/<x>)
│   └── brightfield/
│       ├── zarr.json
│       └── 0/ { zarr.json, c/… }
│
├── labels/
│   ├── zarr.json                  # group
│   └── Cell Mask/                 # OME-NGFF label element (axes y,x, no channel)
│       ├── zarr.json              #   ome + spatialdata_attrs v0.3
│       └── 0/ { zarr.json, c/… }  # scale-0 array (Y,X)
│
├── shapes/
│   ├── zarr.json                  # group
│   └── Region 1/                  # ngff:shapes element
│       ├── zarr.json              #   encoding-type ngff:shapes, axes, transforms, v0.3
│       └── shapes.parquet         # GeoParquet (WKB polygons) — NOT a zarr node
│
├── tables/
│   ├── zarr.json                  # group
│   └── cells/                     # anndata-encoded Zarr-v3 group (ngff:regions_table v0.2)
│       ├── zarr.json              #   encoding-type anndata + region/region_key/instance_key
│       ├── X/            { zarr.json, c/… }        # array
│       ├── obs/                                    # dataframe
│       │   ├── zarr.json          #   column-order, _index
│       │   ├── _index/ { zarr.json, c/… }          # string-array
│       │   ├── cell_id/ { zarr.json, c/… }         # array
│       │   ├── area_px/ { zarr.json, c/… }         # array
│       │   └── region/                             # categorical
│       │       ├── zarr.json
│       │       ├── categories/ { zarr.json, c/… }  # string-array
│       │       └── codes/      { zarr.json, c/… }  # array
│       ├── var/         { zarr.json, marker/ }     # dataframe
│       ├── obsm/        { zarr.json, spatial/ }    # dict → arrays
│       ├── layers/      { zarr.json, positive/ }   # dict → arrays
│       ├── uns/                                    # dict
│       │   ├── zarr.json
│       │   └── spatialdata_attrs/ { region, region_key, instance_key }  # string scalars
│       ├── obsp/ varm/ varp/  { zarr.json }        # empty dict groups
│       └── raw/         { zarr.json }              # null
│
└── opal_aux/                      # Opal-private (ignored by SpatialData readers)
    ├── zarr.json                  # group
    └── cluster_labels/            # plain Zarr-v3 array (+ opal_key, opal_orig_shape attrs)
        ├── zarr.json
        └── c/ …
```

Notes:

- `images`, `labels`, `shapes`, and `tables` are standard SpatialData elements.
  `opal_aux` and the `opal_studio` root attributes are Opal-private extras that
  SpatialData readers ignore.
- Element directory names are filesystem-sanitized; the true layer name is
  preserved in each element's attributes.
- Only the **root** `zarr.json` carries `consolidated_metadata`; every other group
  omits it (so readers that open a sub-group directly fall back to a disk scan).
- `shapes.parquet` is a regular GeoParquet file, deliberately not part of the zarr
  hierarchy.
- Element names must be unique across element types (a SpatialData requirement).

## Application Layout

The main window has three working areas:

- **Left panel**: layer management for Channels, Masks, Positivity, Types, and Regions.
- **Center tabs**: Image, Phenotyping, Heatmap, t-SNE, and UMAP.
- **Right panel**: collapsible operation sections for Pre-processing, Segmentation, Mask Processing, Cell positivity, and Cell identification.

The status bar shows cursor position and the selected channel value while hovering over the image.

## Core Functionality

### Image Viewing

- Lazy, tiled rendering for large images.
- Pyramid-aware zooming so low-resolution levels are used when zoomed out.
- Background rendering with cached tiles to keep panning and zooming responsive.
- Mouse wheel zoom and left/middle mouse drag panning.
- Multi-channel compositing with per-channel color, alpha, and display limits.
- Global brightness control.
- Mask overlays with opacity and optional vector contours.
- Per-cell positivity overlays and phenotype/cluster type overlays.

### Layer Management

The left panel separates generated and source layers:

- **Channels**: raw and processed image channels. Toggle visibility, change color, adjust alpha, adjust intensity limits, and show/hide all source channels.
- **Masks**: segmentation masks. Toggle raster overlay and contour visibility, adjust mask opacity, and delete generated masks.
- **Positivity**: marker positivity cell layers. Positive and negative cells share the same label map but use a positivity lookup table for display.
- **Types**: phenotype or cluster masks. Adjust shared type opacity and show/hide all type layers.
- **Regions**: hand-drawn polygon regions used for selected-region segmentation.

Generated processed channels, masks, positivity layers, type masks, and regions can be selected from this panel and reused by later steps.

### Region Drawing

Use the **Regions** tab in the left panel to draw analysis regions:

1. Click the draw button.
2. Drag on the Image tab to trace a polygon.
3. Release to create a region layer.
4. Select the region layer before running segmentation in **Selected region** mode.

The simplification control reduces polygon point density. Existing region vertices can be dragged while draw mode is active.

## Recommended Workflow

1. **Open data** with `File > Open Image...` or `File > Open SpatialData...`.
2. **Set up display** in the Channels tab: choose visible markers, colors, alpha, brightness, and intensity limits.
3. **Preprocess channels** if needed: merge markers, remove hot pixels, subtract background, rescale intensity, or create CLAHE-enhanced channels.
4. **Draw regions** if you want to test or restrict analysis to a tissue area.
5. **Run segmentation** on a full image, visible viewport, or selected region.
6. **Refine masks** with size filtering, CellSampler mask fusion, or label expansion.
7. **Call marker positivity** with AI or threshold-based per-cell intensity calls.
8. **Define phenotypes** in the Phenotyping tab using marker positive/negative rules.
9. **Identify cells** to create phenotype type masks, or run clustering to discover unsupervised cell populations.
10. **Inspect analysis views** in Heatmap, t-SNE, and UMAP.
11. **Export results** as OME-TIFF masks/cells, GeoJSON contours, and CSV phenotyping definitions.

## File Menu

| Menu action | Purpose |
| --- | --- |
| `Open Image...` | Open OME-TIFF/TIFF image data. |
| `Open SpatialData...` | Open a SpatialData root directory. |
| `Load Masks...` | Import label masks from OME-TIFF/TIFF. |
| `Load Cells...` | Import cell/positivity label maps from OME-TIFF/TIFF. |
| `Load Phenotyping...` | Import phenotype definitions from CSV. |
| `Save Masks...` | Export mask layers as OME-TIFF. |
| `Save Cells...` | Export cell/positivity layers as OME-TIFF. |
| `Save Contours...` | Export selected mask/cell contours as GeoJSON. |
| `Save Phenotyping...` | Export phenotype definitions as CSV. |

Mask and cell OME-TIFF exports are written as `CYX` data with channel names preserved in OME metadata.

## Pre-processing

Open **Pre-processing** in the right panel.

### Merge

The **Merge** tab averages two selected image channels and creates a new processed channel named from the source pair.

### Filter

The **Filter** tab creates a new processed channel from one source or processed channel. Available filters:

- **Median**: median filtering with a disk footprint.
- **Opening**: morphological opening for small-object/noise suppression.
- **CLAHE**: percentile normalization followed by contrast-limited adaptive histogram equalization.
- **Subtract Background**: Gaussian smoothing plus rolling-ball background subtraction.
- **Remove Hotpixels**: hot-pixel removal with threshold, pass count, and filter size controls.
- **Intensity Rescale**: percentile-based intensity rescaling.

Processed channels appear in the Channels tab and can be used for segmentation, positivity, and clustering.

## Segmentation

Open **Segmentation** in the right panel. Choose a region mode, a target mode, and one segmentation engine.

### Region Modes

- **Full image**: segment the entire image.
- **Visible region**: segment only the current canvas viewport, useful for fast parameter testing.
- **Selected region**: segment inside the selected polygon region. Only detections whose centroids fall inside the polygon are kept.

### Target Modes

- **New mask**: create a new mask layer.
- **Overwrite selected mask**: update an existing selected mask. For region-based overwrite, existing cells in the affected area are removed and new detections are merged back in.

### Engines

| Engine | Inputs and controls | Typical use |
| --- | --- | --- |
| **Watershed** | One channel, Voronoi or Gaussian labeller, spot sigma, outline sigma, threshold, minimum mean intensity. | Fast classical nuclei/cell segmentation and parameter testing. |
| **InstanSeg** | One channel, model name, pixel size, optional hole filling and largest-component cleanup. | Fast learned nuclei/cell segmentation. |
| **Mesmer** | Nuclear channel, optional membrane channel, DeepCell/default or local `.keras` model, nuclear or whole-cell compartment, pixel size, watershed post-processing. | Nuclear or whole-cell segmentation for multiplexed imaging. |
| **StarDist** | One channel, pretrained or local model, probability threshold, NMS threshold. | Nuclear segmentation with star-convex objects. |
| **Cellpose** | One channel, nuclei/cyto/cyto2 or local model, diameter, cell probability threshold, flow threshold. | Flexible cell or nuclei segmentation. |
| **Omnipose** | One channel, specialized Omnipose/custom model, diameter, mask threshold, flow threshold. | Bacteria, elongated objects, plant cells, worms, and other non-round shapes. |

Deep-learning engines run in a separate worker process to reduce TensorFlow/PyTorch conflicts. Models are downloaded on first use (see [Models](#models)); your own model folders are auto-discovered under `<models>/<engine>/`.

## Mask Processing

Open **Mask Processing** in the right panel.

| Tab | Function |
| --- | --- |
| **Filter** | Remove labels below a minimum area or above a maximum area. |
| **Sampler** | Merge multiple masks with CellSampler/Ubermasking. Strategies include largest cell count, highest Jaccard, and minimum area variance. |
| **Expand** | Expand labels by a chosen number of pixels. Binary Mask mode uses watershed-style separation lines; Label Map mode preserves integer labels with label expansion. |

Expanded binary masks keep their internal label map so threshold positivity and clustering can still operate per cell.

## Cell Positivity

Open **Cell positivity** in the right panel after creating or importing a cell mask.

### AI Positivity

The **AI** tab runs the packaged marker-positivity model against every non-mask image channel. For each channel it creates a Positivity layer that stores positive/negative calls per cell while preserving the original cell label IDs and contours.

### Threshold Positivity

The **Thresholds** tab computes per-cell mean intensity for every image channel:

1. Select a mask.
2. Click **Get Thresholds**.
3. Opal Studio computes per-cell means and an Otsu threshold for each channel.
4. Positivity layers are created immediately for all channels.
5. Use the channel dropdown, numeric threshold field, or threshold slider to adjust a channel interactively.

The count label shows positive cells over total signal-bearing cells for the selected marker.

## Phenotyping And Cell Identification

Use the **Phenotyping** center tab to define cell types:

1. Enter a cell type name and click **Add Cell Type**.
2. Click table cells to cycle marker rules through blank, `Pos`, and `Neg`.
3. Double-click a column header to rename a cell type.
4. Right-click a column header to delete a cell type.

Then open **Cell identification > Gating** and click **Identify Cells**. Opal Studio combines the marker positivity layers with the phenotype table and creates one Type mask per matching cell type. Cells that do not match any defined type are added to an `Unknown` type mask.

Phenotyping definitions can be saved and loaded as CSV files.

## Clustering

Open **Cell identification > Clustering** for unsupervised population discovery.

Inputs and options:

- Select the mask that defines individual cells.
- Choose which image or processed channels to include.
- Choose a normalization method: Yeo-Johnson, arcsinh, log-z, z-score, min-max, or none.
- Optionally enable PCA. If the PCA component count is blank, Opal Studio uses parallel analysis; DBSCAN uses PCA automatically.
- Choose a clustering method: Leiden, Louvain, PhenoGraph, FlowSOM, KMeans, Hierarchical, or DBSCAN.

Outputs:

- Type masks for each cluster, plus a grey Noise mask for DBSCAN noise when present.
- A Heatmap tab showing per-cluster mean channel intensity.
- t-SNE and UMAP plots colored by cluster.
- Clustering metrics including cell count, cluster count, PCA details, silhouette score, Davies-Bouldin index, Calinski-Harabasz index, and cluster sizes.

Cluster names can be edited in the Heatmap tab. Type mask color changes in the left panel are synchronized to the t-SNE and UMAP plots.

## Model Selection Notes

For IMC datasets, start with models trained or tuned for IMC when available. If no custom model is available, a practical workflow is:

1. Test quickly with **Watershed** or **Visible region** mode.
2. Try **InstanSeg** or **StarDist** for nuclei-rich marker channels.
3. Use **Mesmer** when nuclear and membrane/cytoplasm channels are available.
4. Use **Cellpose** or **Omnipose** when object morphology differs from round nuclei.
5. Refine with size filtering, CellSampler, and expansion before positivity or clustering.

Indicative speed from local high-resolution testing:

| Segmentation Engine | Approximate Speed |
| --- | ---: |
| Watershed | 1 sec |
| InstanSeg | 10 sec |
| Mesmer | 26 sec |
| Cellpose | 14 sec |
| Omnipose | 25 sec |
| StarDist | 60 sec |

Typical IMC segmentation quality depends strongly on staining, tissue, resolution, and model weights. In earlier local testing, the rough ordering was:

```text
StarDist > InstanSeg > Mesmer > Cellpose > Watershed > Omnipose
```

Treat this as a starting point rather than a rule.

## Tips

- Use **Visible region** segmentation to tune parameters before running a full image.
- Use **Overwrite selected mask** when iterating on a segmentation to avoid clutter.
- Draw and select a region before using **Selected region** mode.
- Use processed channels as segmentation inputs when raw channels are noisy or low contrast.
- Keep one clean cell label mask for downstream positivity, phenotyping, and clustering.
- Save masks/cells before closing if you want to reuse generated label maps in another session.

## License

Opal Studio is licensed under the **MIT License with the Commons Clause**.

- Free to use for research, internal analysis, and development.
- Free to inspect, modify, and build upon.
- You may not sell Opal Studio or offer it as a paid hosted service, paid software product, or commercial service whose value derives substantially from Opal Studio.

See `LICENSE` for the full license text.
