Metadata-Version: 2.4
Name: scarf
Version: 1.0.0rc1
Summary: Out-of-core, graph-first workflows for million-cell RNA-seq and CITE-seq
Author-email: Parashar Dhapola <parashar.dhapola@gmail.com>
License-Expression: BSD-3-Clause
Project-URL: Homepage, https://github.com/NygenAnalytics/scarf
Project-URL: Documentation, https://scarf.readthedocs.io/en/latest/
Project-URL: Installation, https://scarf.readthedocs.io/en/latest/installation.html
Project-URL: Issues, https://github.com/NygenAnalytics/scarf/issues
Keywords: single-cell,scRNA-seq,scATAC-seq,CITE-seq,Harmony,WNN,batch-correction,Zarr,memory-efficient
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: h5py>=3.11
Requires-Dist: hnswlib>=0.8
Requires-Dist: leidenalg>=0.12
Requires-Dist: loguru>=0.7
Requires-Dist: networkx>=3
Requires-Dist: numba>=0.60
Requires-Dist: numcodecs>=0.16
Requires-Dist: numpy>=2
Requires-Dist: packaging
Requires-Dist: pandas>=2.2.2
Requires-Dist: scikit-learn>=1.5
Requires-Dist: scipy!=1.17.0,>=1.13
Requires-Dist: setuptools>=61
Requires-Dist: statsmodels>=0.14.2
Requires-Dist: threadpoolctl
Requires-Dist: tqdm>=4.60
Requires-Dist: umap-learn>=0.5.12
Requires-Dist: zarr>=3.2.0
Requires-Dist: obstore>=0.5.1
Requires-Dist: sgtsnepi
Requires-Dist: huggingface-hub>=1.24.0
Requires-Dist: zstandard>=0.25.0
Provides-Extra: extra
Requires-Dist: anndata>=0.12; extra == "extra"
Requires-Dist: kneed>=0.8; extra == "extra"
Requires-Dist: matplotlib>=3.9; extra == "extra"
Requires-Dist: seaborn>=0.13; extra == "extra"
Requires-Dist: ipython-autotime>=0.3; extra == "extra"
Requires-Dist: ipywidgets>=8; extra == "extra"
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Requires-Dist: pytest-cov>=4; extra == "test"
Requires-Dist: pytest-xdist>=3; extra == "test"
Requires-Dist: topacedo>=0.2.10; extra == "test"
Requires-Dist: anndata>=0.12; extra == "test"
Requires-Dist: scikit-network>=0.33.1; extra == "test"
Requires-Dist: rdata>=1.1.0; extra == "test"
Provides-Extra: docs
Requires-Dist: jinja2; extra == "docs"
Requires-Dist: jedi; extra == "docs"
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: myst-nb; extra == "docs"
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: sphinx_autodoc_typehints; extra == "docs"
Requires-Dist: sphinx-book-theme; extra == "docs"
Requires-Dist: sphinx-copybutton; extra == "docs"
Requires-Dist: sphinx-external-toc; extra == "docs"
Requires-Dist: sphinx-tabs; extra == "docs"
Requires-Dist: topacedo; extra == "docs"
Requires-Dist: nbclient; extra == "docs"
Requires-Dist: jupyter-cache>=1.0.1; extra == "docs"
Requires-Dist: sphinx-reredirects>=1.1.0; extra == "docs"
Requires-Dist: sphinxcontrib-mermaid>=2.1.0; extra == "docs"
Dynamic: license-file

# Scarf

<p align="left">
  <a href="https://pypi.org/project/scarf"><img src="https://img.shields.io/pypi/v/scarf.svg?color=4c72b0" alt="PyPI"></a>
  <a href="https://pypi.org/project/scarf"><img src="https://img.shields.io/badge/python-3.12%20%7C%203.13%20%7C%203.14-4c72b0.svg" alt="Python 3.12, 3.13, and 3.14"></a>
  <a href="https://scarf.readthedocs.io"><img src="https://readthedocs.org/projects/scarf/badge/?version=latest" alt="Docs"></a>
  <a href="https://github.com/NygenAnalytics/scarf/actions/workflows/pytest.yml"><img src="https://github.com/NygenAnalytics/scarf/actions/workflows/pytest.yml/badge.svg" alt="Tests"></a>
  <a href="https://codecov.io/gh/NygenAnalytics/scarf"><img src="https://codecov.io/gh/NygenAnalytics/scarf/graph/badge.svg?token=ZvJXuYq3pd" alt="Coverage"></a>
</p>

Scarf is a Python framework for analysing single-cell RNA, ATAC, protein, and multi-omic data, from a few thousand cells to tens of millions.

| Problem | How Scarf solves it | What you get |
| :-- | :-- | :-- |
| Your **dataset is larger than RAM** | Out-of-core algorithms , and neighbour search streams from cell-major and gene-major layouts, inside a memory budget you set | No subsampling, so rare populations survive, [benchmarked to 10M cells](profiling/BENCHMARKS.md) |
| The **data is stored remotely** and requires downloading | Fetches only the chunks an operation touches, and writes results to a store you own | Start analysing immediately, with one authoritative copy |
| A **single parameter change costs hours** of computation | Each step is fingerprinted by its settings and inputs, so reuse is by content, not by layer name | Only what changed recomputes, and the old version stays for comparison |
| Sub-population analysis leaves **scattered copies that nobody can trace back** | Subsets are masks in one file, and every result carries the cells and parameters behind it | A year later, a result still explains itself |

## Install

Python 3.12+.

```bash
uv venv --python 3.12
uv pip install --python .venv "scarf[extra]"
```

Detailed installation instructions [here](https://scarf.readthedocs.io/en/latest/installation.html)

## Quick start

```python
import scarf

reader = scarf.CrH5Reader("filtered_feature_bc_matrix.h5")
scarf.CrToZarr(reader, zarr_loc="data.zarr").dump()

ds = scarf.DataStore("data.zarr", nthreads=4)
ds.pipeline.run()

ds.plots.embedding(
    layout_key="RNA_UMAP",
    color_by="RNA_clusters",
)
```

Read the [scRNA-seq tutorial](https://scarf.readthedocs.io/en/latest/tutorials/scrna_seq.html) for granular analysis workflow.

## Scarf's capabilities

<p align="center">
  <img src="https://raw.githubusercontent.com/NygenAnalytics/scarf/master/docs/source/_static/overview.png" alt="Compressed count chunks feed incremental algorithms and a neighbourhood graph that serves embedding, clustering, mapping, imputation, downsampling, and pseudotime" width="520">
</p>

| Area | Methods |
| :-- | :-- |
| Modalities | scRNA-seq, scATAC-seq, CITE-seq, matched multi-omics |
| Core workflow | Quality control, feature selection, normalization, PCA and LSI, KNN graph, UMAP, densMAP, t-SNE, Leiden, Paris, marker search |
| Integration | Harmony, partial PCA, shared and weighted nearest neighbours, integration metrics |
| Mapping | Symphony-style reference mapping, label transfer, projection diagnostics |
| Trajectory | Population Balance Analysis pseudotime, expression dynamics and modules, multi-sink fate probabilities |
| Also included | Cell-cycle scoring, gene-set activity, graph-diffusion imputation, doublet scores, HTO demultiplexing, TopACeDo downsampling, pseudobulk export |

## Documentation

Read workflow vignettes and API references on **[Read The Docs 📖](https://scarf.readthedocs.io/en/latest/)**

AI-assisted and autonomous workflows should start with **[Analysis with AI agents](https://scarf.readthedocs.io/en/latest/analysis_with_agents.html)**.

## Citation

Dhapola et al. Scarf enables a highly memory-efficient analysis of large-scale single-cell genomics data. [*Nature Communications* 13, 4616 (2022).](https://doi.org/10.1038/s41467-022-32097-3)

## Support

[GitHub issues](https://github.com/NygenAnalytics/scarf/issues)

Scarf is open source with [BSD 3-Clause License](LICENSE) and maintained by [Nygen](https://nygen.io).
