BioSieve

BioSieve is a Python toolkit for preparing biological sequence datasets for machine learning.

It covers two main workflows:

• Redundancy reduction — remove near-duplicate sequences before training using sequence, embedding, descriptor, or structural similarity
• Leakage-aware splitting — partition datasets into train/val/test (or k-folds) with strategies that respect biological structure (homology clusters, sequence distance, groups, time)

Installation

BioSieve supports Python 3.11+.

pip install biosieve

Install optional extras as needed:

• minhash for approximate Jaccard-based deduplication (minhash_jaccard strategy)
• faiss for GPU-accelerated embedding similarity search (embedding_cosine strategy)

pip install 'biosieve[minhash]'
pip install 'biosieve[faiss]'

The mmseqs2 reducer and homology_aware splitter require the MMseqs2 binary to be available in PATH.

Quick Start

Redundancy reduction

Remove near-duplicate sequences using k-mer Jaccard similarity:

biosieve reduce \
  -i dataset.csv \
  -o dataset_nr.csv \
  --strategy kmer_jaccard \
  --mapping-output mapping.csv \
  --report-output report.json

Pass parameters via a YAML file:

biosieve reduce \
  -i dataset.csv \
  -o dataset_nr.csv \
  --strategy kmer_jaccard \
  --config params.yaml

# params.yaml
kmer_jaccard:
  threshold: 0.8
  k: 5

Check a params file without reading any data (exit 0 if valid, 2 if not; --json prints the problems as JSON):

biosieve validate-config params.yaml --strategy kmer_jaccard

Validate a config from Python (a mapping or a YAML/JSON path; returns a list of ConfigIssue(path, message), empty when valid):

from biosieve import validate_config

issues = validate_config("params.yaml", strategy="kmer_jaccard", kind="reduce")

Strategies that need input files take them as options: --embeddings (.npy or .npz) and --embedding-ids for embedding_cosine, --edges for structural_distance. The ids file is <stem>.ids.txt (one id per line), a .npz with an ids array, or a CSV with an id column; when omitted, the .npz's own ids or the sibling <stem>.ids.txt are used.

Override a single parameter inline without a file:

biosieve reduce -i dataset.csv -o out.csv --strategy kmer_jaccard --set kmer_jaccard.threshold=0.9

Dataset splitting

Split with a leakage-aware strategy:

biosieve split \
  -i dataset_nr.csv \
  -o splits/ \
  --strategy homology_aware \
  --clusters clusters.csv \
  --config params.yaml

# params.yaml
homology_aware:
  mode: precomputed
  clusters_format: csv
  member_col: id
  cluster_col: cluster_id
  test_size: 0.2

Strategies

Redundancy reduction

Splitting

All strategies also support k-fold variants (random_kfold, stratified_kfold, group_kfold, stratified_numeric_kfold, distance_aware_kfold).

Outputs

Every run produces consistent artefacts:

• Reduced or split tables, CSV or Parquet (by the output extension for reduce, by the input format for split), with the id column first
• Mapping CSV (removed_id, representative_id, cluster_id, score) for reduction runs
• JSON report with the tool version, strategy name, effective parameters, and reduction/split statistics
• Provenance sidecar following the shared ecosystem CLI contract: <output>.provenance.json for reduce and <outdir>.provenance.json for split

Exit codes: 0 success, 2 invalid config or usage (also validate-inputs failures), 3 runtime or data failure, 4 internal error. homology_aware runs MMseqs2 in a temporary directory unless work_dir is set; keep_work: true keeps it.

Learn More

• examples/README.md for runnable scripts and config files

Citation

If you use BioSieve in your research, please cite:

Fernández Villegas, D.; Álvarez Saravia, D.; Medina-Ortiz, D. BioSieve: Redundancy Reduction and Leakage-Aware Dataset Partitioning for Biological ML. Version 0.1.2, 2026. Zenodo. https://doi.org/10.5281/zenodo.21930919

@software{biosieve2026,
  author    = {Fernández Villegas, Diego and Álvarez Saravia, Diego and Medina-Ortiz, David},
  title     = {{BioSieve}: Redundancy Reduction and Leakage-Aware Dataset Partitioning for Biological ML},
  year      = {2026},
  version   = {0.1.2},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21930919},
  url       = {https://doi.org/10.5281/zenodo.21930919}
}

To cite this exact release, use the version DOI 10.5281/zenodo.21930920. Machine-readable metadata lives in CITATION.cff.

License

MIT. See LICENSE.

Acknowledgements

Built on top of scikit-learn, polars, NumPy, and optionally datasketch, FAISS, and MMseqs2.

Developed by KREN AI Lab at Universidad de Magallanes, Chile.