Metadata-Version: 2.4
Name: fastmixture2
Version: 2.0.5
Summary: Biobank-scale ancestry estimation in unrelated individuals
Author-email: "J. Meisner" <meisnerucph@gmail.com>, "L. Thunegard" <lisathunegard@icloud.com>
License-Expression: GPL-3.0-only
Project-URL: Homepage, https://github.com/MeisnerLab/fastmixture2
Project-URL: Documentation, https://github.com/MeisnerLab/fastmixture2#readme
Project-URL: Issues, https://github.com/MeisnerLab/fastmixture2/issues
Project-URL: Source, https://github.com/MeisnerLab/fastmixture2
Keywords: admixture,ancestry,bioinformatics,genomics,population-genetics
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0
Requires-Dist: scipy>=1.14
Requires-Dist: threadpoolctl>=3.5
Provides-Extra: gpu-cuda13
Requires-Dist: cupy-cuda13x[ctk]>=14.0; extra == "gpu-cuda13"
Dynamic: license-file

# fastmixture2

[![Tests](https://github.com/MeisnerLab/fastmixture2/actions/workflows/ci.yml/badge.svg)](https://github.com/MeisnerLab/fastmixture2/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/fastmixture2.svg)](https://pypi.org/project/fastmixture2/)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://github.com/MeisnerLab/fastmixture2/blob/main/LICENSE)

`fastmixture2` estimates ancestry proportions (`Q`) and ancestral allele
frequencies (`P`) by maximum likelihood. It reads SNP-major PLINK genotypes
or ANGSD Beagle genotype likelihoods. Unsupervised initialization uses SVD/ALS
for K > 1, with subset likelihood refinement for large panels.
All modes use safeguarded SQP/ZAL updates on CPU or, with `--gpu`, on NVIDIA CUDA.

## Installation

Use Python 3.11 or newer. For best CPU performance, build from source.
This uses `-march=native` to tune the code to the build machine and can be
faster than portable wheels.

Install the current GitHub version:

```bash
git clone https://github.com/MeisnerLab/fastmixture2.git
cd fastmixture2
python -m pip install .
```

Source builds need a C compiler with OpenMP. On macOS, install OpenMP with
`brew install libomp`. On HPC, build on the compute-node type used for your
analyses.

With Conda, the [CPU](https://github.com/MeisnerLab/fastmixture2/blob/main/environment-cpu.yml)
and [GPU](https://github.com/MeisnerLab/fastmixture2/blob/main/environment-gpu.yml)
environments retain native CPU tuning. Run from the cloned repository:

```bash
conda env create --file environment-cpu.yml
conda activate fastmixture2-cpu
```

For GPU use, select `environment-gpu.yml` and activate `fastmixture2-gpu`.

PyPI wheels cover Linux (glibc) and macOS 14 or newer, on x86-64 and ARM64.
They use AVX2 on supported x86-64 CPUs and NEON on ARM64, including Apple Silicon:

```bash
python -m pip install fastmixture2
```

To build from PyPI source with native CPU tuning:

```bash
python -m pip install --no-cache-dir --no-binary=fastmixture2 fastmixture2
```

With [uv](https://docs.astral.sh/uv/):

```bash
uv venv
source .venv/bin/activate
uv pip install fastmixture2
```

Use `uv pip install .` from the cloned repository for native CPU tuning.
To build from source without tying the binaries to the build CPU:

```bash
FASTMIXTURE2_PORTABLE=1 python -m pip install .
```

For NVIDIA CUDA 13, install `"fastmixture2[gpu-cuda13]"` from PyPI or
`'.[gpu-cuda13]'` from source with pip or uv. A compatible NVIDIA driver is
required. Add `--gpu` when running an analysis.

## Quick start

Run an unsupervised fit with five ancestry components:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 --out cohort
fastmixture2 --beagle cohort.beagle.gz --K 5 --threads 8 --out cohort.gl
```

- `--bfile` takes one or more ordered prefixes of SNP-major PLINK `.bed`, `.bim`
  and `.fam` files. Genotypes stay packed on CPU and GPU. Missing genotypes are
  supported.
- `--beagle` takes a gzipped ANGSD Beagle file. Likelihood triplets are normalized
  internally. Exactly uniform triplets contribute no ancestry information.

Per-chromosome inputs are fitted jointly without creating a merged BED:

```bash
fastmixture2 --bfile chr{1..22} --K 5 --threads 8 --out cohort
```

The shell expands `chr{1..22}` in chromosome order. Explicit prefixes also work,
such as `--bfile chr1 chr2 chr3`. All FAM files must have identical FID/IID pairs
in the same order. Marker rows, including annotated P output, follow the supplied
file order. The files remain memory-mapped, with an additional native address
per marker (8 bytes on 64-bit systems) and bounded transfer tiles. This works
with CPU, CUDA, supervised estimation, projection, CV, bootstrap and checkpoints.

Perform sample and variant QC first. For each K, try several seeds and keep
the converged fit with the highest log-likelihood. Component labels may change
between runs, so align them before comparing Q or P.

Add `--random-init` to draw Q and P randomly instead of using SVD/ALS or pooled
frequencies. It uses `--seed` and works with either input format on CPU or CUDA
in unsupervised mode. With `--exact-cv`, each fold also starts randomly.
Resuming a checkpoint retains its saved estimates.

Add `--restarts N` to try N additional restart attempts in unsupervised mode (default 0).
Each starts from the original finished fit, permutes Q columns while retaining P, and
jointly refines a fresh genome-wide 50,000-SNP subset. Smaller panels use all SNPs.
Larger panels then project P and refit the full data. Only full-data improvements are
retained. This works with both input formats on CPU and CUDA, uses `--seed`,
and keeps the same `--iter` and `--tole` per stage. CV and bootstrap retain their
usual refit settings.
At K = 1 it has no effect.

Add `--gpu` to use CUDA with either input format:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 --gpu --out cohort.gpu
```

CPU and CUDA support `1 <= K <= 128` for both input formats in all modes.
CUDA keeps input data on the GPU when they fit and streams in blocks otherwise.
Small CPU/GPU differences can arise from floating-point arithmetic.

At K > 1, unsupervised initialization uses 10 SVD power iterations on all SNPs
up to 50,000, or a genome-wide 50,000-SNP subset otherwise. Larger panels refine
subset Q/P with SQP/ZAL, then fit full-marker P by least squares at fixed Q and
retain the refined subset rows. This applies to both input formats, including
missing genotypes.
Panels up to 50,000 SNPs proceed directly from SVD/ALS to full-data refinement,
without a duplicate subset fit or projection. For PLINK, subset refinement is
skipped if any sample has no observed subset genotypes. Exact CV folds skip
subset likelihood refinement.

Subsets exceeding 512 MiB on CPU or 256 MiB on CUDA are built in bounded blocks
into a temporary memory-mapped file. CUDA streams the subset when it does not
fit in VRAM. Temporary files use `TMPDIR`, which can point to disk-backed scratch
storage.

Refinement uses safeguarded SQP/ZAL with EM/QN rescue when needed, for both
input formats and all modes. CPU refinement uses float64. CUDA uses mixed
SQP statistics before final float64 refinement. Parameters, constrained solves,
EM/QN and likelihood checks stay in float64. EM/QN runs on CUDA for both
resident and streamed input. PLINK rescue falls back to CPU if a CUDA allocation
fails.

Workspaces use bounded blocks. Input storage, Q, P and optimizer history are
additional allocations that grow with panel size and K.

The defaults are `--threads 1`, `--seed 42`, `--iter 1000` and `--tole 1e-10`.
`--threads` caps CPU computation, including BLAS and GPU host copies.
The optimizer scales the tolerance by observed genotypes or informative
likelihood triplets. `--iter` and `--tole` apply to each refinement stage.
Check the log for convergence. Run `fastmixture2 --help` for the full CLI.

### Beagle cache

A cache avoids decompressing the Beagle file again for each K or seed:

```bash
fastmixture2 --beagle cohort.beagle.gz --K 5 --threads 8 \
  --gl-cache cohort.gl.npy --out cohort.gl
```

The cache stores three normalized float32 likelihoods per sample and variant,
using about `12 × variants × samples` bytes. An invalid or outdated cache is
rebuilt. The cache is optional.

### Supervised estimation

Use known reference assignments to fix their Q rows while estimating P and
the Q rows of unassigned samples:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 \
  --supervised populations.txt --out cohort.supervised
```

Supply one label per sample in input order: `1` through `K` for reference
populations, or `0` for an unassigned sample. At least one reference sample is
required. Include IDs so the program can check sample order:

```text
FID IID POP
family1 sample1 1
family2 sample2 0
```

A one-column label file also works, but sample order cannot then be checked.
For Beagle input, use each sample label as both FID and IID.

### Projection

Given reference allele frequencies, estimate Q for new samples while keeping
P fixed:

```bash
fastmixture2 --bfile new_samples --K 5 --threads 8 \
  --projection reference.P --out new_samples.projected
```

An annotated P matrix lets the program check marker order and alleles:

```text
SNP A1 A2 POP1 POP2 POP3 POP4 POP5
rs1 A G 0.02 0.08 0.15 0.04 0.10
```

Marker IDs and order must match. Swapped and strand-complemented alleles are
handled automatically. A/T and C/G variants are rejected by default. Use
`--allow-ambiguous` only when both panels share a strand convention. Plain
numeric P matrices also work, but their marker and allele alignment cannot be
checked. Use `--write-info` when creating an annotated reference P matrix.

Supervision and projection also accept `--beagle` and `--gpu`.

### Cross-validation

To compare K values, use the same seed and number of folds. Lower scores are
better:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 --seed 42 --cv 5 --out cohort.cv
```

CV works with either input format in unsupervised mode. By default, each fold
starts from the full-data fit, so held-out entries have influenced its starting
point. Use `--exact-cv` to initialize each fold from its training entries.

Use `--K 1` as a pooled-frequency baseline for CV verification. Q is all ones,
and initialization skips SVD/ALS.

PLINK CV reports one-half binomial deviance. Beagle CV withholds complete
triplets and reports mean held-out negative log marginal likelihood over
nonuniform triplets. Missing genotypes are excluded from PLINK CV. The log
identifies folds that did not converge. Adjust fold tolerance with `--cv-tole`.
With `--gpu`, refits run on CUDA. Exact CV initialization remains on CPU.

### Bootstrap uncertainty

Estimate Q standard errors by resampling blocks of neighboring SNPs:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 \
  --bootstrap 200 --bootstrap-block 1500 --out cohort
```

`--bootstrap` defaults to 200 replicates. Each replicate starts from the
full-data fit and takes up to three weighted SQP/ZAL iterations with EM rescue.
Use `--bootstrap-iter` for longer refits. Add `--bootstrap-tole` to stop at a mean
likelihood gain per informative entry, within that iteration limit. The log
reports refit termination. Short refits give a local estimate of uncertainty.

For PLINK input, blocks stay within chromosomes. Without `--bootstrap-block`,
block lengths average 10 cM using BIM genetic-map spans. Set a block length in
SNPs when map positions are absent or zero. Beagle input always requires
`--bootstrap-block` and treats the marker stream as one segment.

Add `--gpu` for CUDA refits. Projection errors are conditional on the supplied
P matrix. Supervised reference samples have zero Q uncertainty. Bootstrap and
CV must be run separately.

### Checkpointing

Save accepted estimates and resume an interrupted CPU or GPU fit:

```bash
fastmixture2 --bfile cohort --K 5 --threads 8 --checkpoint run.ckpt --out cohort
fastmixture2 --bfile cohort --K 5 --threads 8 --resume run.ckpt --out cohort
```

Keep the same input, supervision and projection files when resuming.
With `--restarts`, checkpoints retain the best full-data fit. Resuming runs the
requested number of additional restarts after the resumed fit.

## Output

For `--out cohort --K 5 --seed 42`:

| File | Contents |
| --- | --- |
| `cohort.K5.s42.Q` | Ancestry proportions, one row per sample |
| `cohort.K5.s42.P` | Allele frequencies, one row per variant |
| `cohort.K5.s42.Q_se` | Q standard errors when bootstrap is requested |
| `cohort.K5.s42.log` | Settings, likelihood, convergence and CV/bootstrap summaries |

Rows follow input order and columns correspond to components. Use
`--write-info` for headers and sample or variant IDs, or `--no-freqs` to omit
P. Projection does not write P. For PLINK, P is the frequency of BIM A1.
For Beagle, it is the frequency of allele2.

Q and Q_se contain N x K values, and P contains M x K values. Matrices use
ten decimal places. The log records export counts and full-precision
likelihoods before output rounding. Console likelihoods use six decimal
places. Numbers have no thousands separators.

Outputs, checkpoints and caches use standard file permissions controlled by
your umask.

The log distinguishes convergence, iteration limits, stalled optimization and
numerical failure. Unsuccessful fits retain the best accepted parameters.

Parallel jobs should use distinct output prefixes. Build each likelihood cache
with one job, then reuse it across fits, or use a separate cache path per job.
fastmixture2 does not create filesystem locks or require private lock directories.

## Authors and license

Lisa Thunegard and Jonas Meisner. Developed at **MeisnerLab** and distributed
under the [GNU General Public License v3.0](https://github.com/MeisnerLab/fastmixture2/blob/main/LICENSE).
