Metadata-Version: 2.4
Name: hide-deconv
Version: 0.4.2
Summary: HIDE-Deconv - Hierarchical Cell Type Deconvolution
License-Expression: MIT
License-File: LICENSE
Keywords: Bioinformatics,Computational Biology,Deconvolution,Tumor Microenvironment,RNA-seq
Author: Dennis Voelkl
Author-email: dennis.k.volkl@uib.no
Requires-Python: >=3.12,<3.15,!=3.14.1
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Dist: anndata (>=0.12.10,<0.14.0)
Requires-Dist: click (>=8.3.2,<9.0.0)
Requires-Dist: cvxpy (>=1.8.2,<2.0.0)
Requires-Dist: inquirerpy (>=0.3.4,<0.4.0)
Requires-Dist: lifelines (>=0.30.3,<0.31.0)
Requires-Dist: networkx (>=3.6.1,<4.0.0)
Requires-Dist: numpy (>=2.4.4,<3.0.0)
Requires-Dist: pandas (>=2.3.3)
Requires-Dist: pydeseq2 (>=0.5.4,<0.6.0)
Requires-Dist: requests (>=2.33.1,<3.0.0)
Requires-Dist: rich (>=14.3.3)
Requires-Dist: scanpy (>=1.12.1,<2.0.0)
Requires-Dist: scikit-learn (>=1.8.0,<2.0.0)
Requires-Dist: scikit-posthocs (>=0.12.0)
Requires-Dist: scipy (>=1.17.1,<2.0.0)
Requires-Dist: seaborn (>=0.13.2,<0.14.0)
Requires-Dist: statsmodels (>=0.14.6)
Requires-Dist: torch (>=2.11.0,<3.0.0)
Requires-Dist: umap-learn (>=0.5.12,<0.6.0)
Project-URL: Documentation, https://dvoelkl.github.io/HIDE-deconv/
Project-URL: Homepage, https://github.com/dvoelkl/HIDE-deconv
Project-URL: Issues, https://github.com/dvoelkl/HIDE-deconv/issues
Project-URL: Repository, https://github.com/dvoelkl/HIDE-deconv
Description-Content-Type: text/markdown

# HIDE-deconv

**A framework for characterizing cellular remodeling from bulk RNA-seq data through hierarchical cell-type deconvolution.**

---
![HIDE-Deconv Summary](https://github.com/dvoelkl/HIDE-deconv/blob/main/figures/HIDE-deconv-overview_extended.png)
---

## Key Features

- Bulk RNA-seq cell-type deconvolution using annotated single-cell references
- Hierarchical estimation of cell-type compositions across multiple annotation levels
- Characterization of cellular remodeling across cell populations and biological conditions
- Configurable domain-transfer and library-size correction
- Downstream analysis of deconvolution results, including PCA, UMAP, clustering, survival analysis, and differential composition analysis
- Python API and command-line interface for reproducible workflows
- Supports AnnData based single-cell workflows
- Local execution suitable for secure research environments

For a detailed overview of all features and usage, please read the [Userguide](https://github.com/dvoelkl/HIDE-deconv/blob/main/docs/Userguide.md). The API documentation can be found on [GitHub Pages](https://dvoelkl.github.io/HIDE-deconv/).

## Real world performance: Morandini

As an additional evaluation on real world data, HIDE-deconv was benchmarked on the Morandini bulk RNA-seq dataset using the HaoSub single-cell reference, following the evaluation procedure of the `deconvBench` framework from Dietrich et al. (2026).

| Method          | Pearson R   |    RMSE   |
| --------------- | ----------: | --------: |
| AutoGeneS       |       0.819 |     0.182 |
| BayesPrism      |       0.254 |     0.295 |
| Bisque          |       0.872 |     0.122 |
| CIBERSORTx      |       0.648 |     0.193 |
| DWLS            |       0.909 |     0.103 |
| MuSiC           |       0.319 |     0.275 |
| Scaden          |       0.795 |     0.150 |
| SCDC            |       0.468 |     0.239 |
| **HIDE-deconv (1)** |   **0.934** | **0.094** |
| **HIDE-deconv (2)** |   **0.933** | **0.094** |

HIDE-deconv (1) was trained and evaluated at the resolution of the cell types defined in the Hao single-cell reference, without specifying a hierarchy.

HIDE-deconv (2) was trained and evaluated using a three-level cell-type hierarchy, analogous to the hierarchical aggregation used for the FACS benchmarking.

HIDE-deconv was evaluated independently ([Benchmark scripts](https://github.com/dvoelkl/HIDE-Deconv-Benchmarks)). Results for the other methods are taken from [Dietrich et al. (2026)](https://doi.org/10.1186/s13059-026-03955-w)





## Installation

```bash
# Create and activate a new virtual environment (recommended)
python3 -m venv .venv
source .venv/bin/activate

# Install HIDE-deconv
pip install hide-deconv
```

---

## Necessary Data

- **Single-cell data:** Annotated AnnData (.h5ad) file with gene names in `adata.var_names` and cell type annotations for each desired layer in `adata.obs` (at least one layer of cell type annotations is necessary).
- **Bulk RNA-seq data:** CSV file, genes as row index, samples as columns. Gene IDs must match single-cell data.
- **Sample sheet (optional):** CSV with sample meta-information (e.g., cohort, survival time, event).
- **Data Normalization:** We recommend to use raw counts for all datasets.

---

## Command Line Workflow

**Deconvolution (standard workflow):**
```bash
hide-deconv run --path <project_dir>
```
This initializes the project, preprocesses data, trains the model, and runs deconvolution.

**Difference in composition & survival analysis:**
```bash
hide-deconv analyze diff --path <project_dir>
hide-deconv analyze survival --path <project_dir>
```
- For difference analysis, the sample sheet must contain columns for sample ID and cohort.
- For survival analysis, the sample sheet must contain columns for sample ID, survival time and event. Values in the event column must be either "1": event occured or "0": no event.

**Command overview:**
```bash
hide-deconv help
```

This displays a short introduction to the command line interface and gives an overview of all available commands.

---

### Tutorials
[![Basic Usage Tutorial](https://img.youtube.com/vi/bB4dcJt_WCw/0.jpg)](https://www.youtube.com/watch?v=bB4dcJt_WCw)

---

## Quick Deconvolution API Example
HIDE-deconv contains a "lazy" API for users who want quick access to deconvolution.

```python
import anndata as ad
import pandas as pd
from hide_deconv import deconvolution

adata = ad.read_h5ad("single_cells.h5ad")
bulk = pd.read_csv("bulk.csv", index_col=0) # genes x samples

results = deconvolution(adata, bulk, celltype_cols=["cell_type", "major"]) # Two layer hierarchy
# results = deconvolution(adata, bulk, celltype_cols="major") # Deconvolution only on major level

# results is a list of pandas DataFrames, one per hierarchy layer
# results[0] is the finest layer, results[1] the next coarser layer
```

If you only pass `adata` and `bulk`, `celltype_cols` defaults to `"cell_type"`.

## Extended API Example
For experienced users offers HIDE-deconv a highly customizable API, where they can integrate their own preprocessing steps into the pipeline. An example is given below. For a detailed description of the API we refer to [GitHub Pages](https://dvoelkl.github.io/HIDE-deconv/)

```python
import anndata as ad
import pandas as pd
import numpy as np
from hide_deconv.preprocessing import create_reference, create_bulks
from hide_deconv.models import HIDE
from hide_deconv.statistic import run_mann_whitney_u
from hide_deconv.preprocessing.bulk_preprocessing import get_domain_transfer_factor
from hide_deconv.pipelines.deconvolve_hide_pipeline import normalize_bulk_to_cpm

# 1. Load single-cell, bulk and sample sheet data
adata = ad.read_h5ad("single_cells.h5ad")
bulk = pd.read_csv("bulk.csv", index_col=0)  # genes x samples
sample_sheet = pd.read_csv("sample_sheet.csv", index_col=0)

# The sample sheet index must contain the corresponding bulk sample IDs.
sample_sheet = sample_sheet.reindex(bulk.columns)
bulk = normalize_bulk_to_cpm(bulk)
healthy_bulk = bulk.loc[:, sample_sheet["disease"] == "healthy"]
bulk = bulk.loc[:, sample_sheet["disease"] != "healthy"]
common_genes = adata.var_names.intersection(bulk.index).intersection(healthy_bulk.index)
adata = adata[:, common_genes].copy()
healthy_bulk = healthy_bulk.loc[common_genes]
bulk = bulk.loc[common_genes]

# 2. Create reference profiles and hierarchy (single layer example)
X_sub = create_reference(adata, celltype_col="cell_type")
A_l = [pd.DataFrame(np.eye(X_sub.shape[1]), index=X_sub.columns, columns=X_sub.columns)]
X_l = [X_sub]

# 3. Simulate training bulks
Y_train, C_train = create_bulks(adata, n_bulks=1000, n_cells_per_bulk=100, celltype_col="cell_type")

# 4. Initialize and train model
hide = HIDE(X_l, A_l)
hide.train(Y_train, C_train, iter=1000)

# 5. Calculate domain-transfer factors from the healthy bulk reference
Y_domain, _ = create_bulks(
	adata,
	n_bulks=1000,
	n_cells_per_bulk=100,
	celltype_col="cell_type",
)
Y_domain = Y_domain.loc[common_genes]
alpha = get_domain_transfer_factor(healthy_bulk, Y_domain)
alpha_inv = (1 / alpha).replace([np.inf, -np.inf], 0).fillna(1.0)
bulk = bulk.mul(alpha_inv, axis=0)

# 6. Calculate library sizes on the complete single-cell gene set
library_sizes = (
	adata.obs.assign(library_size=np.asarray(adata.X.sum(axis=1)).ravel())
	.groupby("cell_type")["library_size"]
	.median()
)

# 7. Deconvolution on the batch-corrected bulk data
results = hide.predict(
	bulk.loc[hide.gene_labels],
	norm=True,
	library_sizes=library_sizes,
)["prediction"]

# 8. Optional: Difference in composition analysis
# diff = run_mann_whitney_u(results[0], sample_sheet, sample_id_col="SampleID", cohort_col="Cohort")
```

---

## Citation
If you use HIDE-deconv, please cite the following preprint.

HIDE-Deconv: A hierarchical deconvolution framework for multiscale characterization of cellular remodeling

Dennis Voelkl, Sarah Bolz, Austin Rayford, Thomas Stevenson, Thomas Sterr, Malte Mensching-Buhr, Nicole Seifert, Julia Arp, Cornelia Schuster, Jana Tausche, Laurenz Engel, Helena U. Zacharias, Michael Altenbuchinger, Franziska Görter
bioRxiv 2026.08.24.746754; doi: https://doi.org/10.64898/2026.08.24.746754

---

## License

This project is licensed under the MIT License.

---

## Contact

For questions, support or scientific collaboration:
- Dennis Voelkl: dennis.k.voelkl(at)uib.no
- Franziska Goertler: Franziska.Gortler(at)uib.no

---

