Turbo Picard

Documentation · Command coverage · Evaluate on your data · Research

Run a selected Picard step in Rust

Run selected Picard tools in Rust, using the command names and arguments your pipelines already understand. Evaluate one command before changing a pipeline.

Turbo Picard is built for teams maintaining SAM/BAM/CRAM, VCF and sequencing-QC steps in Nextflow, WDL, Snakemake and shell workflows. Native commands avoid the JVM; the documented interface keeps migration local to the task you replace.

Start with one representative input. Native coverage is command- and option-specific; this is not the full upstream suite. Keep Picard for work outside the documented scope.

For coding agents and workflow generators, inspect the complete decision surface in one call:

turbo-picard capabilities --json

The schema-versioned response contains every Picard command's native/fallback status and trial fit together with the checked-in parity-gated benchmark evidence. Use turbo-picard trial --json <PicardCommand> ... for the exact task being considered. See the agentic-coder guide for the selection rule and safe substitution pattern.

Compact, native-only automation

Version 0.1.14 improves native MarkDuplicates handling for coordinate ties, unplaced reads, and repeated read names across read groups. It bounds a speculative shortcut and lowers peak memory on the checked-in large spill fixtures. See the release notes for the measured runtime and memory trade-off and the evidence limits.

Version 0.1.13 adds capabilities --json --command MarkDuplicates for compact discovery, executable argument arrays in trial JSON, and the strict TURBO_PICARD_REQUIRE_NATIVE=1 policy. See the agentic-coder guide. Inspection reports describe command scope, not proof that a particular input or option is validated.

Quick Start

Install from PyPI:

python3 -m pip install turbo-picard==0.1.14

For a containerized trial, use the published release image:

docker run --rm ghcr.io/dnncha/turbo-picard:0.1.14 --version

Installing from PyPI currently gives you both commands:

• turbo-picard: the explicit command for evaluation and normal use.
• picard: a compatibility shim for environments where you deliberately want existing picard calls to resolve to this package.

Use the explicit turbo-picard command while testing. Add the shim to a pipeline environment only after the specific commands you need have been checked.

Check the install and print a trial contract before changing a workflow:

turbo-picard --version
turbo-picard MarkDuplicates --help
turbo-picard doctor
turbo-picard trial MarkDuplicates I=input.bam O=marked.bam M=metrics.txt

The trial command prints matching Picard and turbo-picard invocations, declared outputs, fallback state, and comparison notes. Then run the chosen command on a representative input:

turbo-picard MarkDuplicates I=input.bam O=marked.bam M=metrics.txt

From a repository checkout:

cargo install --locked --path crates/turbo-picard-cli --bin turbo-picard --bin picard

Compare before changing a workflow

The repository includes an evaluator that runs both implementations in separate output paths and records versions, timings and output digests. It does not upload your data. See the real-data evaluation guide for a copyable command and the interpretation of a match or mismatch.

The evaluator in this source checkout uses disk-backed sorting for large comparisons, preserves existing evaluation directories, and retains failed runs for diagnosis. The evaluator records command runtimes separately from comparison work; improvements to its sorting helper do not establish faster native commands.

When It Helps

• You already run Picard commands and want to trial one slow step first.
• You need Picard-style command names and KEY=VALUE arguments to stay stable.
• You want a command-by-command rollout with upstream Picard available for unsupported or unchecked behavior.
• You can compare outputs on a representative BAM, CRAM, FASTQ, VCF, or metrics file before changing the workflow.

Good first trials are usually MarkDuplicates, SortSam, SamToFastq, FastqToSam, FixMateInformation, BuildBamIndex, and repeated metrics commands. Use turbo-picard trial <PicardCommand> ... to print a side-by-side Picard and turbo-picard evaluation contract before changing a workflow.

When To Stay With Picard

• You need an option or command that is not inside the documented native scope and cannot use fallback.
• You require Picard-equivalent chart rendering rather than checked metrics text.
• You have not compared the exact command, input shape, sidecars, metrics, exit code, and error behavior your workflow depends on.
• You need broad cohort evidence before trying a representative shard.

Documentation

The full docs are on Read the Docs: installation, command scope, evaluation, and maintenance.

Useful starting points:

• Quickstart
• Agentic coder guide
• Is this for you?
• Choose your first command
• Evaluation playbook
• Command coverage
• Picard alternatives
• Trying it in a pipeline
• Parity guide
• Fallback to Picard
• Benchmarks
• Citation
• Packaging
• Troubleshooting

Starter workflow files live in packaging/workflows/. The smallest trial shape is packaging/workflows/one-command-trial.md. Migration patterns that usually keep the surrounding workflow stable include per-read-group SamToFastq, sequential-shard FastqToSam, and mate-repair boundaries around FixMateInformation. If you run a real one-command evaluation, share the result through the trial report issue form; successful matches, mismatches, and adoption blockers are all useful evidence. If GitHub does not offer new-issue creation, add the same redacted report as a comment on the public trial report thread. From a repository checkout, tools/compare_real_data.py --shareable-report can create a reviewed, privacy-conscious starting point for that report.

Benchmark evidence

The saved suite records 32 command comparisons against Picard 3.4.0, each checked against its specified output contract. These are small-fixture results: startup overhead contributes substantially to the measured ratios. The saved suite reports a 22.88x minimum, 84.52x geometric mean, and 272.12x maximum speedup on those fixtures. In the saved MarkDuplicates case, the generator used reads=50000; median wall times were 0.075464 seconds for Turbo Picard and 2.238986 seconds for Picard across three runs. This is not a whole-genome performance estimate.

See the benchmark guide for methods and real-data scope, the saved record for all 32 measurements and reproduction commands, and the parity guide for what was compared. Metrics-text agreement does not establish Picard-equivalent charts.

Cheerful Duck Research publishes our broader bioinformatics software investigations, reproduction material, and corrections.

Saved benchmark record

python3 tools/update_real_data_manifest.py
python3 tools/verify_benchmark_log_evidence.py
python3 tools/verify_benchmark_suite_coverage.py
python3 tools/verify_benchmark_thresholds.py
python3 tools/verify_real_data_evidence.py
python3 tools/verify_real_data_evidence.py --release-ready

Check the saved real-data record

The benchmarks/real-data/ manifest records pinned inputs for gatk-na12878-mito, picard-snvq, and gatk-na12878-mito-cram. From a checkout, use the existing evidence tools:

python3 tools/update_real_data_manifest.py
python3 tools/verify_real_data_evidence.py
python3 tools/verify_real_data_evidence.py --release-ready

These commands check the repository's saved evidence. They do not validate a new input file or replace a comparison of your own workflow.

Workflow evaluation

The project publishes a workflow validation protocol, a compatibility contract, and a production-scale benchmark format. Use these before changing a workflow. The opt-in Nextflow process candidate is documented under packaging/nf-core.

A command-level speedup is not a universal replacement claim. Keep upstream Picard available until representative BAM/CRAM evidence, output parity, failure behaviour, and independent review pass for the exact workflow.

Packaging Status

The current source release is 0.1.14. Release builds target Linux x86_64 and ARM64, macOS Intel and Apple Silicon, plus a source distribution. Publication is gated on artifact validation and installation smoke tests; the release page and PyPI identify the published artifacts. The Linux ARM64 wheel is cross-built and artifact-validated.

Read the release notes for the release scope and evidence boundaries.

The submitted Bioconda recipe PR covers the main package and an optional shim. Use PyPI or the container image until Bioconda accepts the PR and the packages appear in its indexes. The main package installs turbo-picard; the separate shim package installs the picard command only for environments that choose it.

Citation

Cite the archived turbo-picard release you used with CITATION.cff. Benchmark and validation inputs should be cited separately with immutable source URLs, commits or accessions, and input SHA-256 hashes.

Docs source lives in docs/. JOSS submission notes are tracked in docs/joss-submission.rst.

Contributing

Bug reports, parity evidence, documentation fixes, and small command-coverage improvements are welcome. Start with CONTRIBUTING.md and the development docs.

Support: SUPPORT.md. Security: SECURITY.md.