Metadata-Version: 2.4
Name: count_split
Version: 1.0.1
Summary: Reproducible molecule-level count allocation
Home-page: https://github.com/scottyler89/count_split
Author: Scott Tyler
Author-email: scottyler89@gmail.com
License: AGPL-3.0-only
Project-URL: Changelog, https://github.com/scottyler89/count_split/blob/main/CHANGELOG.md
Project-URL: Source, https://github.com/scottyler89/count_split
Project-URL: Issues, https://github.com/scottyler89/count_split/issues
Project-URL: Methodology, https://anna-neufeld.github.io/countsplit/
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: h5py
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: numba
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# count_split

[![CI](https://github.com/scottyler89/count_split/actions/workflows/ci.yml/badge.svg)](https://github.com/scottyler89/count_split/actions/workflows/ci.yml)

`count_split` reproducibly allocates observed count molecules into statistically
independent analysis folds. It is a Python implementation inspired by Neufeld
et al.'s count-splitting approach for avoiding double use of the same counts in
single-cell clustering and differential-expression analysis.

- Paper: [Inference after latent variable estimation for single-cell RNA sequencing data](https://arxiv.org/abs/2207.00554)
- Reference R package and methodology: [countsplit](https://anna-neufeld.github.io/countsplit/)

Please cite the original methodology when using this package.

## Allocation contract

For every observed molecule, the implementation draws one uniform random value
and assigns that molecule to exactly one requested fold. Multi-fold allocation
is simultaneous: `[0.325, 0.325, 0.35]` means those final probabilities, not a
sequence of percentages applied to a shrinking remainder.

The implementation guarantees:

- element-wise count conservation across all output folds;
- deterministic results for the same seed and canonically ordered matrix;
- identical seeded allocation for equivalent dense, CSR, and CSC inputs;
- no sparse-to-dense conversion;
- preservation of integer input dtypes on output (CSC for every sparse input
  format); and
- explicit rejection of negative, non-finite, fractional, boolean, complex, or
  out-of-range counts.

## Installation

From a checkout:

```bash
python -m pip install .
```

For development:

```bash
python -m pip install -e .
python -m pytest
```

Python 3.10 or newer is required.

## Usage

The legacy API expects variables in rows and samples in columns. For a
single-cell matrix, that means genes × cells.

```python
import numpy as np
from count_split.count_split import multi_split

counts = np.random.default_rng(7).negative_binomial(
    2,
    0.4,
    size=(1_000, 500),
)

train, validation, test = multi_split(
    counts,
    percent_vect=[0.325, 0.325, 0.35],
    seed=123456,
)

assert np.array_equal(train + validation + test, counts)
```

Weights are normalized, so `[1, 1, 2]` is equivalent to
`[0.25, 0.25, 0.5]`. A zero-weight fold is retained as an all-zero result.
`multi_split` continues to return a list for backward compatibility.

### Sparse matrices

CSR, CSC, and other SciPy sparse inputs are accepted. Duplicate sparse
coordinates are summed once in safe `int64` scratch space. All sparse outputs
are canonical CSC matrices and preserve the input integer dtype when it can
represent the canonicalized logical counts.

```python
from scipy import sparse
from count_split.count_split import multi_split

counts_csr = sparse.csr_matrix(counts)
train, validation, test = multi_split(
    counts_csr,
    percent_vect=[0.325, 0.325, 0.35],
    seed=123456,
)

assert ((train + validation + test) != counts_csr).nnz == 0
```

### AnnData

AnnData normally stores cells in rows and genes in columns, so transpose `X`
for this legacy API:

```python
train, validation, test = multi_split(
    adata.X.T,
    percent_vect=[0.325, 0.325, 0.35],
    seed=123456,
)
```

The returned matrices are genes × cells. Downstream code that expects the
AnnData orientation should transpose each result back.

### Two-fold APIs

The original pairwise functions and argument order remain available. The new
`seed` parameter is appended, so existing positional calls remain valid.

```python
from count_split.count_split import split_mat_counts, split_sparse

dense_a, dense_b = split_mat_counts(counts, percent_1=0.4, seed=17)
sparse_a, sparse_b = split_sparse(counts_csr, percent_1=0.4, seed=17)
```

Calling `seed_rng(value)` before a call that omits `seed` remains supported for
legacy integrations. New code should use the explicit `seed` argument. Valid
explicit seeds are integers from 0 through 4,294,967,295.

### Dense HDF5

`split_mat_counts_h5` preserves the original two-output file interface and
copies all non-target HDF5 content into both outputs. Only the selected dense,
two-dimensional dataset is replaced with split counts.

```python
from count_split.count_split import split_mat_counts_h5

split_mat_counts_h5(
    "input.h5",
    "fold_a.h5",
    "fold_b.h5",
    percent_1=0.5,
    bin_size=5_000,
    key="infile",
    seed=17,
)
```

## Memory and reproducibility notes

All cumulative offsets and total-molecule bookkeeping use `int64` internally,
but this does not force `int64` storage on output. Each output entry is bounded
by its corresponding canonicalized input count. Integral real-valued inputs are
accepted for legacy compatibility but return integer (`int64`) folds.

The chosen non-parametric algorithm explicitly materializes one uniform value
and one fold index per observed molecule, plus a folds × nonzero-entry
re-collation buffer. Its peak memory therefore scales with the total molecule
count, not only matrix dimensions or sparse nonzeros. This is deliberate and
should be capacity-planned for very deep matrices.

For ordinary dense and sparse calls, `bin_size` is accepted only for signature
compatibility and does not change matrix-wide allocation. The HDF5 function
uses `bin_size` to process contiguous column chunks while maintaining one RNG
stream; seeded output is invariant to that chunk size.

Reproducibility requires the same package/runtime stack, input values, matrix
shape, row/column order, fold weights, and seed. Equivalent dense, CSR, and CSC
representations are canonicalized to the same column-major allocation order.

## Compatibility

Version 1.0.0 retains the original public function names, existing positional
arguments, HDF5 behavior, CSC sparse return convention, and list return from
`multi_split`. Correctness-related behavior changes are documented in
[CHANGELOG.md](CHANGELOG.md).

## License and contact

Licensed under the GNU Affero General Public License v3.0; see
[LICENSE](LICENSE).

Source: [github.com/scottyler89/count_split](https://github.com/scottyler89/count_split)

Repository owner: scottyler89@gmail.com
