Metadata-Version: 2.4
Name: sampledisco
Version: 0.1.2
Summary: Cross-omics, cross-condition sample embedding for single-cell data (RNA + ATAC)
Author: Hongkai Ji
Author-email: Harry Jiang <hjiang55@jh.edu>
License: MIT License
        
        Copyright (c) 2026 Harry Jiang, Hongkai Ji, and the SampleDisco authors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/J041120h/GenoDistance
Project-URL: Documentation, https://j041120h.github.io/SampleDisco_tutorial/
Project-URL: Repository, https://github.com/J041120h/GenoDistance
Keywords: single-cell,sample-embedding,multi-omics,scRNA-seq,scATAC-seq,bioinformatics
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.23
Requires-Dist: pandas>=1.5
Requires-Dist: scipy>=1.9
Requires-Dist: scikit-learn>=1.2
Requires-Dist: scikit-misc>=0.3
Requires-Dist: statsmodels>=0.13
Requires-Dist: patsy>=0.5
Requires-Dist: scanpy>=1.11
Requires-Dist: anndata>=0.10
Requires-Dist: muon>=0.1
Requires-Dist: harmonypy<0.0.10,>=0.0.9
Requires-Dist: python-igraph>=0.10
Requires-Dist: leidenalg>=0.9
Requires-Dist: networkx>=3.0
Requires-Dist: POT>=0.9
Requires-Dist: pygam>=0.9
Requires-Dist: numba>=0.57
Requires-Dist: matplotlib>=3.6
Requires-Dist: seaborn>=0.12
Requires-Dist: pyyaml>=6.0
Requires-Dist: psutil>=5.9
Requires-Dist: pybedtools>=0.9
Requires-Dist: pyensembl>=2.3
Requires-Dist: combat>=0.3
Requires-Dist: adjustText>=1.0
Provides-Extra: multiomics
Requires-Dist: torch>=2.0; extra == "multiomics"
Requires-Dist: scglue>=0.3; extra == "multiomics"
Requires-Dist: harmony-pytorch>=0.1; extra == "multiomics"
Dynamic: license-file

# SampleDisco

A cross-omics, cross-condition **sample embedding** tool for single-cell data.

SampleDisco takes a cell-level embedding (from any standard scRNA / scATAC / multi-omics integration method) and lifts it to a **sample-level embedding** that captures both cell-type composition and the per-cell-type state of each sample. Every downstream analysis — sample-to-sample distance, clustering, trajectory inference, phenotype association — then runs on that single shared sample embedding, regardless of modality.

Paper draft: [`/users/hjiang/GenoDistance/SampleDisco_Draft-11.pdf`](../SampleDisco_Draft-11.pdf)

---

## What the method does

For each modality (RNA, ATAC, or integrated multi-omics) the pipeline produces two cell-level views:

| Key | Role | Source |
|---|---|---|
| **`Z_clust`** | sample-removed embedding — used for clustering and composition blocks | Harmony (single-omics) / Harmony post-pass on scGLUE (multi-omics) |
| **`Z_cmd`**  | sample-preserved embedding — used for the counterfactual displacement (CMD) block | second Harmony pass (single-omics) / scGLUE primary output (multi-omics) |

It then assembles **four blocks** per sample (or per sample × modality for multi-omics):

1. **A1** — one-hot cell-type composition
2. **A2** — soft k-means composition at K_med (≈120)
3. **A3** — soft k-means composition at K_fine (≈300)
4. **CMD** — leave-one-out cell-type-resolved displacement on `Z_cmd`

The four blocks are inverse-variance weighted, Frobenius-stacked, PCA-reduced to 10 dimensions, and Harmony-corrected at sample level. The result is stored as `adata.uns['X_DR_sample']` and feeds every downstream module.

---

## Repository layout

```
code/
├── SampleDisc.py              # CLI entry point (simple or complex mode)
├── config/                    # 9 YAML configs covering covid / blood / eye / heart / ENCODE / tabula / long_covid / unpaired / default
├── wrapper/                   # Orchestration
│   ├── wrapper.py             # Master wrapper; gates RNA + ATAC + multiomics + shared downstream
│   ├── rna_wrapper.py
│   ├── atac_wrapper.py
│   └── multiomics_wrapper.py
├── preparation/               # Preprocessing
│   ├── rna_preprocess_{cpu,gpu}.py   # QC → HVG → PCA → dual Harmony → Z_clust + Z_cmd
│   ├── atac_preprocess_{cpu,gpu}.py  # QC → TF-IDF → HVF → LSI → dual Harmony → Z_clust + Z_cmd
│   ├── cell_type_{cpu,gpu}.py        # Leiden clustering on Z_clust (RNA or ATAC)
│   ├── ATAC_cell_type{,_gpu}.py      # ATAC-specific cell typing variants
│   ├── multi_omics_glue.py           # scGLUE integration (cross-modality VAE + guidance graph)
│   ├── multi_omics_batch_correction.py # Harmony post-pass on X_glue → Z_clust
│   ├── multi_omics_merge.py          # post-GLUE merge + per-modality preprocess/slimming
│   └── multi_omics_cell_type_{cpu,gpu}.py  # RNA-Leiden + k-NN label transfer to ATAC
├── sample_embedding/          # Core method
│   ├── blocks.py              # composition, CMD, weighting, Frobenius stack, final PCA + Harmony
│   ├── sample_embedding.py    # CPU pipeline
│   └── sample_embedding_gpu.py # GPU pipeline (cuML + cupy)
├── parameter_selection/
│   └── autotune.py            # Bayesian GP sweep over CMD α; adaptive proxy ensemble
├── sample_distance/           # Pairwise sample distances (DR / EMD / chi-square / JS)
├── sample_clustering/         # Hierarchical (HRA / HRC / NN / UPGMA / consensus), K-means, proportion test, RAISIN
├── sample_trajectory/         # CCA (supervised) and TSCAN (unsupervised) + GAM-based trajectory DGE
├── sample_association/        # Per-PC variance explained vs sample-level covariates (permutation FDR)
├── visualization/             # Embedding plots, dendrograms, DGE volcanos, modality-aware multi-omics scatters
├── utils/                     # Shared helpers: seed, safe h5ad I/O, limma, TF-IDF, batch regress, Grouping
├── gene_activity/             # ATAC peak → gene activity inference + RNA-ATAC validation
└── claude/                    # Active one-off run scripts (rerun launchers, monitored SE, parameter sweeps)
```

---

## Usage

### Complex mode (recommended) — YAML-driven

```bash
python SampleDisc.py -m complex --config config/config.yaml
```

The YAML drives every flag and parameter for all three pipelines:

- **Pipeline gates** (top-level): `run_rna_pipeline`, `run_atac_pipeline`, `run_multiomics_pipeline`
- **Per-modality phase gates** (Phase 1): `*_preprocessing`, `*_cell_type_cluster`, `*_derive_sample_embedding`
- **Per-modality downstream gates** (Phase 2): `*_sample_distance_calculation`, `*_trajectory_analysis`, `*_trajectory_dge`, `*_sample_cluster`, `*_proportion_test`, `*_cluster_dge`, `*_visualize_data`, `*_dimension_association_analysis`
- **Multi-omics-specific**: `multiomics_run_glue_*`, `multiomics_treat_sample_as_batch`, `multiomics_run_glue_twice_for_sample_removal`

The 9 ready-to-use configs in `config/` are point-in-time snapshots for the datasets used in the paper; copy one and adjust paths / column names for your own data.

### Simple mode — one positional file, defaults everywhere

```bash
python SampleDisc.py -m simple -c <count_data.h5ad> -o <output_dir>
```

---

## Inputs

A standard scanpy AnnData file with at minimum:
- `.X` — count matrix (genes for RNA, peaks for ATAC)
- `.obs['sample']` — sample column (required)
- Optional: `.obs['batch']`, `.obs['cell_type']`, sample-level metadata file (CSV) to merge

For multi-omics, the pipeline takes two separate h5ads (RNA + ATAC) and integrates them via scGLUE; samples may be **paired** (1:1 cell correspondence) or **unpaired**.

---

## Outputs (under `output_dir`)

```
<output_dir>/
├── rna/
│   ├── preprocess/adata_preprocessed.h5ad
│   ├── sample_embedding/sample_embedding.csv
│   ├── Sample_distance/{cosine,correlation}/*
│   ├── CCA/  or  TSCAN/                       # whichever trajectory mode
│   ├── trajectoryDEG/
│   ├── sample_cluster/{kmeans_*,proportion_test/}
│   ├── sample_association/variance_explained_sample.csv + figures/
│   └── visualization/*.png
├── atac/   (parallel structure)
├── multiomics/
│   ├── integration/glue/{rna-pp,atac-pp,guidance.graphml.gz}
│   ├── preprocess/adata_sample.h5ad           # post-GLUE merged adata with Z_clust + Z_cmd
│   ├── sample_embedding/sample_embedding.csv
│   └── (same downstream subdirs as single-omics)
└── sys_log/main_process_status.json           # which stages completed
```

---

## Installation

SampleDisco is **one package**. The CPU install is pip-only; **GPU acceleration is
activated simply by installing the GPU libraries separately** — the same package
detects and uses them at runtime. There is no separate "GPU build" of SampleDisco.

### 1. Core install (CPU)

```bash
pip install sampledisco          # once published — or `pip install -e .` from a clone
```

### 2. System prerequisite — bedtools

scGLUE (the multi-omics integrator) calls the `bedtools` binary, which pip cannot
provide:

```bash
conda install -c bioconda bedtools
```

### 3. GPU acceleration (optional, install yourself)

The GPU functions (RAPIDS-accelerated normalization, Harmony, k-means / PCA, Leiden,
scGLUE training) turn on **only when the RAPIDS stack is present** in your
environment. RAPIDS is CUDA-driver-specific and conda-only, so you install it
separately, matching your driver (the pins below target a CUDA-12.5 driver such as
the cluster's GPU nodes):

```bash
conda install -c rapidsai -c conda-forge -c nvidia \
    cuml=24.12 cudf=24.12 cugraph=24.12 rmm=24.12 cuvs=24.12 cupy=13 cuda-version=12.5
pip install rapids-singlecell==0.13.1 --no-deps
```

Then set `use_gpu: true` in your config. **You do not reinstall SampleDisco** — once
those packages are importable the GPU paths activate automatically; if they are
missing or the driver is too old, SampleDisco falls back to CPU equivalents
(`harmonypy` / linear regression, scikit-learn k-means, PyTorch CPU).

### One-command environments (recommended)

For a fully reproducible environment (including bedtools), use the provided conda
files instead of the manual steps above — see `INSTALL.md` for the driver/version
notes:

```bash
conda env create -f environment-cpu.yml      # CPU
conda env create -f environment-gpu.yml      # GPU (RAPIDS 24.12)
```
