fastmixture2

fastmixture2 estimates ancestry proportions (Q) and ancestral allele frequencies (P) by maximum likelihood. It reads SNP-major PLINK genotypes or ANGSD Beagle genotype likelihoods. Unsupervised initialization uses SVD/ALS for K > 1, with subset likelihood refinement for large panels. All modes use safeguarded SQP/ZAL updates on CPU or, with --gpu, on NVIDIA CUDA.

Installation

Use Python 3.11 or newer. For best CPU performance, build from source. This uses -march=native to tune the code to the build machine and can be faster than portable wheels.

Install the current GitHub version:

git clone https://github.com/MeisnerLab/fastmixture2.git
cd fastmixture2
python -m pip install .

Source builds need a C compiler with OpenMP. On macOS, install OpenMP with brew install libomp. On HPC, build on the compute-node type used for your analyses.

With Conda, the CPU and GPU environments retain native CPU tuning. Run from the cloned repository:

conda env create --file environment-cpu.yml
conda activate fastmixture2-cpu

For GPU use, select environment-gpu.yml and activate fastmixture2-gpu.

PyPI wheels cover Linux (glibc) and macOS 14 or newer, on x86-64 and ARM64. They use AVX2 on supported x86-64 CPUs and NEON on ARM64, including Apple Silicon:

python -m pip install fastmixture2

To build from PyPI source with native CPU tuning:

python -m pip install --no-cache-dir --no-binary=fastmixture2 fastmixture2

With uv:

uv venv
source .venv/bin/activate
uv pip install fastmixture2

Use uv pip install . from the cloned repository for native CPU tuning. To build from source without tying the binaries to the build CPU:

FASTMIXTURE2_PORTABLE=1 python -m pip install .

For NVIDIA CUDA 13, install "fastmixture2[gpu-cuda13]" from PyPI or '.[gpu-cuda13]' from source with pip or uv. A compatible NVIDIA driver is required. Add --gpu when running an analysis.

Quick start

Run an unsupervised fit with five ancestry components:

fastmixture2 --bfile cohort --K 5 --threads 8 --out cohort
fastmixture2 --beagle cohort.beagle.gz --K 5 --threads 8 --out cohort.gl

• --bfile takes the shared prefix of SNP-major PLINK .bed, .bim and .fam files. Genotypes stay packed on CPU and GPU. Missing genotypes are supported.
• --beagle takes a gzipped ANGSD Beagle file. Likelihood triplets are normalized internally. Exactly uniform triplets contribute no ancestry information.

Perform sample and variant QC first. For each K, try several seeds and keep the converged fit with the highest log-likelihood. Component labels may change between runs, so align them before comparing Q or P.

Add --random-init to draw Q and P randomly instead of using SVD/ALS or pooled frequencies. It uses --seed and works with either input format on CPU or CUDA in unsupervised mode. With --exact-cv, each fold also starts randomly. Resuming a checkpoint retains its saved estimates.

Add --gpu to use CUDA with either input format:

fastmixture2 --bfile cohort --K 5 --threads 8 --gpu --out cohort.gpu

CPU and CUDA support 1 <= K <= 128 for both input formats in all modes. CUDA keeps input data on the GPU when they fit and streams in blocks otherwise. Small CPU/GPU differences can arise from floating-point arithmetic.

At K > 1, unsupervised initialization uses 10 SVD power iterations on all SNPs up to 50,000, or a genome-wide 50,000-SNP subset otherwise. Larger panels refine subset Q/P with SQP/ZAL, then fit full-marker P by least squares at fixed Q and retain the refined subset rows. This applies to both input formats, including missing genotypes. Panels up to 50,000 SNPs proceed directly from SVD/ALS to full-data refinement, without a duplicate subset fit or projection. For PLINK, subset refinement is skipped if any sample has no observed subset genotypes. Exact CV folds and --iter 0 retain SVD/ALS without subset likelihood refinement.

Subsets exceeding 512 MiB on CPU or 256 MiB on CUDA are built in bounded blocks into a temporary memory-mapped file. CUDA streams the subset when it does not fit in VRAM. Temporary files use TMPDIR, which can point to disk-backed scratch storage.

CUDA SVD starts at 256 samples. Both devices use CPU ALS with float64 contractions. Fixed-Q P projection uses float64 products on CPU and float32 products on CUDA.

Resident and streamed PLINK CUDA fits use float32 SQP statistics while gains remain material, then switch to float64 refinement. There is no fixed float32 iteration cap. Tile products use float32, with float64 sums across tiles. Parameters, constrained solves and likelihood checks stay in float64.

Genotype-likelihood fits use Q-first SQP/ZAL on both CPU and CUDA, with exact EM/QN rescue and an independent EM check before convergence. CUDA fits start with float32 probabilities, derivatives and statistics in either storage mode, then switch to float64 refinement. Near frequency bounds, GL probabilities and derivatives use float64. cuBLAS tile sums, parameters, solves and scores stay in float64. CPU and CUDA start directly with SQP and use ZAL history depth 3. --iter and --tole apply to each SQP/ZAL stage. EM skips missing genotypes and exactly uniform likelihood triplets, with normalization over informative entries. Nonuniform likelihoods are retained. Full likelihood scores retain the constant contribution from uniform triplets.

CPU SQP uses tiled BLAS statistics for Q and P at every supported K. This applies to PLINK and genotype likelihoods in all modes. BLAS scratch and P statistics each have a separate 512 MiB cap. Q statistics use 4096-sample shards with a separate 4 GiB cap. Optional update caches remain capped at 64 MiB. Input storage, factors and optimizer history are additional allocations.

At K <= 16, CUDA fits use native kernels at K = 1..3 and tiled cuBLAS at K = 4..16 during float32 refinement. Resident tiles share 64 MiB of scratch and keep float32 copies of P and Q. At K = 8..16, resident mixed fits reuse frequency scratch for cooperative float64 solves. Tiles that cannot hold the solve tables use the native solve. Streamed fits keep float32 copies within tiles and reuse their float64 Q-statistics buffers, with up to 128 MiB of compute scratch. Final refinement uses native float64 kernels, except resident GL P updates at K = 8..16 with up to 1024 samples. Those share the tiled cuBLAS P routine and keep combined scratch within 64 MiB. Unsupervised resident PLINK fits at K = 10..16 keep the first eight iterations fresh, then alternate Q-Hessian reuse during float32 refinement. The cache is capped at 64 MiB. Weak cached proposals retry with fresh curvature, and float64 refinement always uses fresh curvature.

At K > 16, CUDA uses tiled cuBLAS statistics and a cooperative active-set solver for both input formats. Resident and bootstrap maps share 64 MiB of scratch. Streamed maps allow up to 128 MiB of compute scratch and 512 MiB of Q statistics, with free-VRAM guards. Streamed tiles adapt to the panel width, with at most 8192 rows within the same memory caps. Curvature stays fresh.

The defaults are --threads 1, --seed 42, --iter 1000 and --tole 1e-10. --threads caps CPU computation, including BLAS and GPU host copies. The optimizer scales the tolerance by observed genotypes or informative likelihood triplets. Check the log for convergence. Run fastmixture2 --help for the full CLI.

Beagle cache

A cache avoids decompressing the Beagle file again for each K or seed:

fastmixture2 --beagle cohort.beagle.gz --K 5 --threads 8 \
  --gl-cache cohort.gl.npy --out cohort.gl

The cache stores three normalized float32 likelihoods per sample and variant, using about 12 × variants × samples bytes. An invalid or outdated cache is rebuilt. The cache is optional.

Supervised estimation

Use known reference assignments to fix their Q rows while estimating P and the Q rows of unassigned samples:

fastmixture2 --bfile cohort --K 5 --threads 8 \
  --supervised populations.txt --out cohort.supervised

Supply one label per sample in input order: 1 through K for reference populations, or 0 for an unassigned sample. At least one reference sample is required. Include IDs so the program can check sample order:

FID IID POP
family1 sample1 1
family2 sample2 0

A one-column label file also works, but sample order cannot then be checked. For Beagle input, use each sample label as both FID and IID.

Projection

Given reference allele frequencies, estimate Q for new samples while keeping P fixed:

fastmixture2 --bfile new_samples --K 5 --threads 8 \
  --projection reference.P --out new_samples.projected

An annotated P matrix lets the program check marker order and alleles:

SNP A1 A2 POP1 POP2 POP3 POP4 POP5
rs1 A G 0.02 0.08 0.15 0.04 0.10

Marker IDs and order must match. Swapped and strand-complemented alleles are handled automatically. A/T and C/G variants are rejected by default. Use --allow-ambiguous only when both panels share a strand convention. Plain numeric P matrices also work, but their marker and allele alignment cannot be checked. Use --write-info when creating an annotated reference P matrix.

Supervision and projection also accept --beagle and --gpu.

Cross-validation

To compare K values, use the same seed and number of folds. Lower scores are better:

fastmixture2 --bfile cohort --K 5 --threads 8 --seed 42 --cv 5 --out cohort.cv

CV works with either input format in unsupervised mode. By default, each fold starts from the full-data fit, so held-out entries have influenced its starting point. Use --exact-cv to initialize each fold from its training entries.

Use --K 1 as a pooled-frequency baseline for CV verification. Q is all ones, and initialization skips SVD/ALS.

PLINK CV reports one-half binomial deviance. Beagle CV withholds complete triplets and reports mean held-out negative log marginal likelihood over nonuniform triplets. Missing genotypes are excluded from PLINK CV. The log identifies folds that did not converge. Adjust fold tolerance with --cv-tole. With --gpu, refits run on CUDA. Exact CV initialization remains on CPU.

Bootstrap uncertainty

Estimate Q standard errors by resampling blocks of neighboring SNPs:

fastmixture2 --bfile cohort --K 5 --threads 8 \
  --bootstrap 200 --bootstrap-block 1500 --out cohort

--bootstrap defaults to 200 replicates. Each replicate starts from the full-data fit and takes up to three weighted SQP/ZAL iterations with EM rescue. Use --bootstrap-iter for longer refits. Add --bootstrap-tole to stop at a mean likelihood gain per informative entry, within that iteration limit. The log reports refit termination. Short refits give a local estimate of uncertainty.

For PLINK input, blocks stay within chromosomes. Without --bootstrap-block, block lengths average 10 cM using BIM genetic-map spans. Set a block length in SNPs when map positions are absent or zero. Beagle input always requires --bootstrap-block and treats the marker stream as one segment.

Add --gpu for CUDA refits. Projection errors are conditional on the supplied P matrix. Supervised reference samples have zero Q uncertainty. Bootstrap and CV must be run separately.

Checkpointing

Save accepted estimates and resume an interrupted CPU or GPU fit:

fastmixture2 --bfile cohort --K 5 --threads 8 --checkpoint run.ckpt --out cohort
fastmixture2 --bfile cohort --K 5 --threads 8 --resume run.ckpt --out cohort

Keep the same input, supervision and projection files when resuming.

Output

For --out cohort --K 5 --seed 42:

Rows follow input order and columns correspond to components. Use --write-info for headers and sample or variant IDs, or --no-freqs to omit P. Projection does not write P. For PLINK, P is the frequency of BIM A1. For Beagle, it is the frequency of allele2.

Q and Q_se contain N x K values, and P contains M x K values. Matrices use ten decimal places. The log records export counts and full-precision likelihoods before output rounding. Console likelihoods use six decimal places. Numbers have no thousands separators.

Outputs, checkpoints and caches use standard file permissions controlled by your umask.

The log distinguishes convergence, iteration limits, stalled optimization and numerical failure. Unsuccessful fits retain the best accepted parameters.

Parallel jobs should use distinct output prefixes. Build each likelihood cache with one job, then reuse it across fits, or use a separate cache path per job. fastmixture2 does not create filesystem locks or require private lock directories.

Authors and license

Lisa Thunegard and Jonas Meisner. Developed at MeisnerLab and distributed under the GNU General Public License v3.0.