Metadata-Version: 2.5
Name: ophist
Version: 0.1.0
Summary: End-to-end, segmentation-free analysis for image-based spatial transcriptomics (Xenium, Atera, CosMx, MERSCOPE): a Snakemake pipeline with a command line.
Project-URL: Repository, https://gitlab.svi.edu.au/biocellgen-public/ophist
Author-email: Patrick Crock <pat.crock@svi.edu.au>, Yan Pan <anne.pan@svi.edu.au>
License-Expression: BSD-3-Clause
License-File: LICENSE
Keywords: CosMx,MERSCOPE,Snakemake,Xenium,domains,imaging spatial transcriptomics,niches,spatial domains,spatial transcriptomics
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.11
Requires-Dist: pandas
Requires-Dist: pyyaml
Requires-Dist: snakemake<10,>=9
Provides-Extra: slurm
Requires-Dist: snakemake-executor-plugin-slurm; extra == 'slurm'
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# Overview

OphiST is a Snakemake pipeline for the analysis of image-based spatial transcriptomics (iST) data in a segmentation-free manner. 

The accurate assignment of transcripts to cells presents a major bottleneck in conventional analysis of iST data. Issues such as transcript bleed from adjacent cells, cells overlapping in the z-plane, and segmentation failures on irregularly-shaped cells can all lead to complications in downstream analyses. OphiST applies several tools from the Bioinformatics and Cellular Genomics Lab at SVI designed to operate directly on the raw, individual transcript data output by iST platforms, with no cell segmentation required. Namely, [SpatialRNA](https://pypi.org/project/spatialrna/) applies graph neural network (GNN) to embed each individual transcript in a latent space reflecting its local molecular context, and [DuetiST](https://pypi.org/project/duetist/) then derives both fine-grained molecular niches and broader spatial domains from the transcript-embedding data. These tools both operate downstream of sensible preprocessing steps such as demultiplexing, QC filtering (if possible), and tissue-detection. Further along the pipeline, basic downstream analyses such as domain-wise (and condition-wise, if contrasts are supplied) differential gene expression and gene-set enrichment analysis are performed. Everything is summarised in a self-contained HTML report output by the final step of the pipeline.

## Documentation

See the following sub-pages for further details about the pipeline: 

- [Configuration](docs/config.md) explains every parameter in the `config` should you wish to tweak the defaults. 
- [Outputs](docs/outputs.md) details the outputs of the pipeline. 
- [Containers](docs/containers.md) contains information about the Apptainer containers and Conda environments used for running the pipeline, including instructions on building them yourself. 
- [Command line](docs/cli.md) describes the `ophist` command: `run`, `init`, `examples` and `profile`.

## Installation

OphiST needs Python 3.11 or later. Install it with pip, including the SLURM plugin if you run
on a SLURM cluster:

```bash
pip install "ophist[slurm]"
```

This installs the `ophist` command, the workflow and [Snakemake](https://snakemake.readthedocs.io).
The latest development version installs straight from the repository:
`pip install "ophist[slurm] @ git+https://gitlab.svi.edu.au/biocellgen-public/ophist.git"`.
On a cluster, install into an environment on a shared filesystem, because the compute nodes run
the workflow's scripts from there.

To work on the workflow itself, clone the repository. An editable install then makes `ophist`
run the clone:

```bash
git clone https://gitlab.svi.edu.au/biocellgen-public/ophist.git
pip install -e "./ophist[slurm]"
```

A clone also runs without pip, through Snakemake directly; see
[docs/cli.md](docs/cli.md#without-the-command).

## Requirements

- [Apptainer](https://apptainer.org). Each step runs in a prebuilt image, pulled on first use.
  Conda works too; see [docs/containers.md](docs/containers.md).
- An NVIDIA GPU for step 3 (spatialRNA training and embedding).
- To run on a SLURM cluster, the
  [SLURM executor plugin](https://snakemake.github.io/snakemake-plugin-catalog/plugins/executor/slurm.html),
  installed by the `slurm` extra above.

## Quick start

1. Start a run directory from the example closest to your data (`ophist examples` lists
   them):

   ```bash
   ophist init my_run --example xenium_tma
   ```

2. List your samples in `my_run/config/samples.tsv`. Adjust `my_run/config/config.yml` as
   needed. A config only needs the settings it changes; see [docs/config.md](docs/config.md).

3. Once, write a Snakemake profile, and fill in your account, partition and the data paths to
   bind into the images:

   ```bash
   ophist profile slurm        # writes profiles/slurm/config.yaml
   ```

   On a single GPU machine, use `ophist profile local` instead, or no profile at all.

4. Run. Output goes to `my_run/`; add `-n` first for a dry run.

   ```bash
   ophist run my_run/config/config.yml --profile profiles/slurm
   ```

   A long run is best launched from a batch job of its own. That head job needs about 1 CPU
   and 8 GB.

## Examples

[`examples/`](examples/) holds configs for three data types:

| Example | Data |
|---|---|
| [`xenium_tma`](examples/xenium_tma/) | Xenium slides cut into TMA cores, with a condition contrast |
| [`atera_kidney`](examples/atera_kidney/) | one Atera whole-transcriptome slide |
| [`ophist_logo`](examples/ophist_logo/) | the OphiST simulated dataset, a small test that runs end to end in under an hour |

## Overview of the pipeline

| Step | Rule file | Main output | Description |
|---|---|---|---|
| 1. Preprocessing | `rules/01_preprocessing.smk` | `data/{sample}/qc_transcripts.zarr` | Raw transcripts are QC-filtered if possible. (Optional: demultiplexing of slides) |
| 2. Tissue detection | `rules/02_tissue_detection.smk` | `data/{sample}/tissue_mask.npz`, `data/cohort.zarr` | Tissue outline detected from transcript density; transcripts are filtered to this outline. |
| 3. spatialRNA embeddings | `rules/03_spatialrna.smk` | `data/spatialrna/cohort_embeddings.zarr` | SpatialRNA applies a GNN to give each transcript an embedding vector describing its spatial context. |
| 4. Lumen detection | `rules/04_lumen_detection.smk` | `data/{sample}/masks.geojson` | (Optional: Lumen within the tissue are detected from transcript density; this mask is used downstream in smoothing.) |
| 6. Smoothing | `rules/06_smoothing.smk` | `data/{sample}/smoothed.npz` | The transcript-embeddings are smoothed onto a grid for tractability. |
| 7. Niches and domains | `rules/07_hmrf_potts.smk` | `data/{sample}/{niches,domains}.npz`, `data/hmrf/` | DuetiST derives niches and domains shared across the samples from the GNN embeddings. | 
| 8. Domain analysis | `rules/08_domain_analysis.smk` | `output/domain_dge/` | Transcripts are pseudobulked by domain. Domains are tested for marker genes and cell-type signatures. |
| 9. Condition DGE | `rules/09_dge.smk` | `output/condition_dge/` | (Optional: Further pseudobulk analysis of transcripts within domains, across contrasts specified in inputs.) |
| 10. Report | `rules/10_report.smk` | `output/report.html` | Summaries of all above steps, including QC figures, training and model reports, PCA analysis, spatial maps, DGE and GSEA tables. |

Each rule file's scripts are in `scripts/<rule file name>/`. Helpers shared between steps are
in `scripts/utils/`.

## Credits

- Patrick Crock: Lead engineer of the pipeline.
- Anne Pan: Current maintainer of the [spatialrna](https://pypi.org/project/spatialrna/) package. Designed rules 03 and 10. 
- Ruqian Lyu: Original author of the [spatialrna](https://pypi.org/project/spatialrna/) package.
- Smoothing and the Potts model both use [duetist](https://gitlab.svi.edu.au/biocellgen-public/duetist).

## Citation

A manuscript is in preparation.

## License

BSD 3-Clause; see [LICENSE](LICENSE).
