tt-bio 0.13.0

████████╗████████╗        ██████╗  ██╗  ██████╗
╚══██╔══╝╚══██╔══╝        ██╔══██╗ ██║ ██╔═══██╗
   ██║      ██║    █████╗ ██████╔╝ ██║ ██║   ██║
   ██║      ██║    ╚════╝ ██╔══██╗ ██║ ██║   ██║
   ██║      ██║           ██████╔╝ ██║ ╚██████╔╝
   ╚═╝      ╚═╝           ╚═════╝  ╚═╝  ╚═════╝

TT-Bio runs biomolecular structure prediction, protein design and protein language models on Tenstorrent Blackhole and Wormhole hardware, on one card or many (a QuietBox has 4 cards, a Galaxy server 32).

Every model is validated against its official reference implementation on the same input and reproduces it within that reference's own run-to-run noise. The methodology, per-target results and reproduction commands are in docs/implementation-parity.md. Predictions and designs per hour per server, and throughput per dollar of purchase price and of total cost of ownership against NVIDIA DGX H200, B200 and A100, are on tt-bio.com. The NVIDIA figures run each model's own upstream code, not BioNeMo or Anthropic's life-sciences kit; an optimised serving stack might be faster and is not measured. The benchmark page has the measured seconds behind every figure, the fixtures, the run conditions and the cost model.

Each runs as tt-bio <command> --model <name>:

Several machines each run their own controller, and a scheduler of your choice spreads the work between them: Running tt-bio on many machines.

Installation

Create a Python virtual environment with Python 3.10 or 3.12, install with the Tenstorrent extra, then install the matching Tenstorrent system dependencies.

We test on Ubuntu 24.04 (glibc 2.39, Python 3.12). The ttnn wheel is tagged manylinux_2_34 and imports GLIBC_2.34 symbols, so glibc 2.34 or newer is required: RHEL 8 and its rebuilds ship glibc 2.28 and cannot install it. See Troubleshooting for the CPU frequency driver check, which matters more than anything else about the host.

python3.10 -m venv env
source env/bin/activate
pip install 'tt-bio[tenstorrent]'
tt-bio install-deps

tt-bio install-deps installs the Tenstorrent system dependencies that match this release. It may ask for your sudo password.

On a host without a Tenstorrent card, plain pip install tt-bio is enough: the Boltz-2 CPU/GPU path (--accelerator cpu / --accelerator gpu) and the CLI work without the Tenstorrent SDK. The other models run on Tenstorrent only.

From GitHub / source

Pin to a tagged release, track nightly main (may be untested), or work from an editable clone:

pip install "tt-bio[tenstorrent] @ git+https://github.com/moritztng/tt-bio.git@v0.13.0"   # pinned release, see Releases for the latest
pip install "tt-bio[tenstorrent] @ git+https://github.com/moritztng/tt-bio.git@main"     # nightly
# or
git clone https://github.com/moritztng/tt-bio.git
cd tt-bio
pip install -e '.[tenstorrent]'
tt-bio install-deps

Drop the [tenstorrent] extra on a host without a Tenstorrent card.

Optional: Build TT-Metal / TT-NN from Source

If you need to build from source, follow the Tenstorrent Installation Guide.

Verify Installation

tt-bio --version   # or -V; prints the installed version
tt-bio --help
tt-bio predict --help
tt-bio msa --help

Then fold a structure, predict affinity, design binders or embed sequences. For reference: the input format, output files and scores, weights and MSA databases, every command-line option, tuning flags and training.

Troubleshooting

Check the CPU frequency driver first. On an AMD host:

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver   # want: amd-pstate-epp

If it reads acpi-cpufreq, every ttnn op pays a fixed extra host cost, measured at roughly 5 to 12 microseconds, and throughput drops across every model. The symptom is distinctive: per-op device time is unchanged and the loss tracks a model's op count rather than its size, so a fold that issues 25k ops loses far more than an embedding that issues 1k. One user measured Boltz-2 at 1.29 structures/s under a 5.4 kernel with acpi-cpufreq and 2.14 structures/s on the same box, same binaries, after booting a kernel that gives amd-pstate-epp.

amd-pstate-epp needs Linux 6.3 or newer. A kernel configured with CONFIG_X86_AMD_PSTATE_DEFAULT_MODE=3 selects it with no boot flag; Ubuntu 24.04 does this. On an older kernel that has the driver but does not default to it, add amd_pstate=active to the kernel command line. Kernels before 6.3 (RHEL 8, Oracle UEK 6, anything 4.18 or 5.4) have no amd-pstate at all.

Single-host prediction needs no MPI setup of yours. tt-metal ships the OpenMPI it wants, and a system OpenMPI on the environment breaks that build instead of replacing it: with OMPI_MCA_* or OPAL_PREFIX set, or a foreign libmpi on LD_LIBRARY_PATH, MPI_Init aborts before the fold starts. tt-bio warns when it sees one and changes nothing for you. To clear it for the current shell:

unset OMPI_MCA_pml OMPI_MCA_plm OPAL_PREFIX LD_LIBRARY_PATH

Structure Prediction

tt-bio predict examples/prot.yaml --model boltz2 --override

Every command names its model with --model:

• boltz2: folds complexes of proteins, DNA, RNA, and ligands and predicts binding affinity. MSA-dependent (uses an MSA by default).
• esmfold2 / esmfold2-fast: fold complexes of proteins, DNA, RNA, and ligands (CCD code or SMILES) on-device, no MSA required (esmfold2-fast is the lighter, faster checkpoint). Modified residues are folded as the modified chemistry, and covalent bond constraints and cyclic peptides are supported. Templates and pocket constraints are refused.
• protenix-v1 / protenix-v2: fold complexes of proteins, RNA, DNA, and ligands (an AlphaFold3-family model, the Protenix reproduction); MSA-dependent for proteins (uses an MSA by default), and also emit a PAE/PDE matrix with --write_pae. protenix-v1 is upstream's own v0.5.0 base checkpoint: half the pair width and 4 trunk recycles against protenix-v2's 10, so it is the cheaper of the two. Covalent bond constraints and modified residues are supported on both; cyclic peptides and templates, as a structure file or a precomputed alignment (see Templates), on protenix-v2 only. protenix-v1's checkpoint ships no template blocks, and it does not close a cyclic peptide's ring. tt-bio does not download the protenix-v2 weights: upstream's v2 release note says they may not be redistributed without the rights holder's consent, so --model protenix-v2 runs only on a copy you already hold (see docs/weights.md). protenix-v1 downloads as usual.
• openfold3: folds proteins, RNA and DNA (an AlphaFold3-family model, the OpenFold3 reproduction); MSA-dependent (uses an MSA by default), with optional templates, as a structure file or a precomputed alignment (see Templates). Polymer chains only, so ligands are refused. Cyclic peptides, modified residues and a covalent bond to a modified residue are supported; a bond between two standard residues (a disulfide) is refused. Weights download from the OpenFold consortium's bucket on first use.
• openbind: OpenBind-0, the same OpenFold3 stack on upstream's v0.5.0 checkpoint, tuned for protein-ligand co-folding. Takes ligands by SMILES or CCD code alongside protein, RNA and DNA chains; MSA-dependent (uses an MSA by default), with optional templates, as a structure file or a precomputed alignment (see Templates). Cyclic peptides, modified residues and covalent bonds to a ligand or modified residue are supported; a bond between two standard residues (a disulfide) is refused. Weights are a separate file from openfold3 and also download on first use.
• saprot: structure-aware protein embeddings, an ESM-2 encoder over a fused amino-acid + Foldseek-3Di vocabulary (446 tokens). Needs a structure for the 3Di structural tokens (--structure); runs sequence-only without it. Use for variant-effect / mutation-fitness scoring and function prediction.
• nesso1 (tt-bio affinity): protein-ligand binding affinity without a structure. Predicts a soft distogram and reads the affinity off that, so it is much cheaper than folding and it returns no coordinates. Proteins and ligands only.
• opendde / opendde-abag: antibody-antigen co-folding built on the Protenix-v2 stack plus a structural-token expander; opendde-abag selects the antibody-antigen checkpoint. Protein, RNA, DNA and ligand chains, with covalent bond constraints, cyclic peptides, modified residues and templates in either form (see Templates). Proteins are MSA-dependent (uses an MSA by default, like Protenix-v2). opendde-abag matches the upstream checkpoint on the standard 1AHW antibody-antigen target; both implementations perform poorly on 9DSG.
• af2ig: AlphaFold2 initial-guess, the filter binder-design pipelines use to tell a real design from a plausible one. It takes a design you already have (a structure carrying the target chain and the binder backbone, plus the binder's sequence), re-predicts the complex starting from those coordinates, and reports pLDDT, pTM, ipTM, pAE and interface pAE. Single-sequence, no diffusion, no seed: the same input gives the same answer. Input format and an example are in examples/af2_designed_complex.yaml; the file takes target: and binder: and nothing else, and any other key is refused rather than ignored. Weights are DeepMind's AlphaFold2 monomer pTM parameters; tt-bio weights --download af2ig fetches them (4 GB, one file kept).
• rf3: folds complexes of proteins, RNA, DNA, and ligands (an AlphaFold3-family model, RoseTTAFold3 from the Institute for Protein Design); MSA-dependent for proteins (uses an MSA by default). Writes AlphaFold3-style <name>_summary_confidences.json (pTM, ipTM, chain-pair PAE/PDE, ranking score) next to each structure. Modified residues and covalent bonds to a ligand or modified residue are supported; cyclic chains, a bond between two standard residues (a disulfide) and pocket constraints are refused. Templates work in either form (see Templates). Weights download from the IPD on first use.

tt-bio predict examples/prot.fasta --model esmfold2-fast --fast
tt-bio predict examples/fkg_ligand.yaml --model esmfold2   # protein + ligand co-fold, no MSA
tt-bio predict examples/prot.yaml --model protenix-v2   # needs your own checkpoint; MSA on by default
tt-bio predict examples/prot.yaml --model protenix-v1   # upstream's v0.5.0 base checkpoint, 4 recycles
tt-bio predict examples/prot.fasta --model openfold3    # MSA on by default
tt-bio predict examples/affinity.yaml --model openbind  # protein + ligand co-fold
tt-bio predict examples/9dsg_abag.yaml --model opendde-abag   # antibody-antigen co-fold, MSA on by default
tt-bio predict examples/prot.yaml --model rf3            # MSA on by default; weights fetch from the IPD
tt-bio predict examples/prot.yaml --model rf3 \
    --partial_t 150 --partial_structure start.cif       # refine start.cif instead of folding from scratch
tt-bio predict targets.yaml --model rf3 --early_stop_plddt 0.5   # skip the rollout on hopeless targets
tt-bio predict examples/af2_designed_complex.yaml --model af2ig  # score a designed complex: ipTM + interface pAE

Ligands, nucleic acids, modified residues, cyclic chains, covalent and pocket constraints, templates and affinity are one generated matrix: docs/model-capabilities.md. Anything a model cannot honour is refused by name before the fold starts, never accepted and dropped.

Size Limits

Every structure model folds at least 1536 residues on a single 12 GiB Wormhole chip. The limit below is the largest size that folded, under the first measured failure where one was found, walked with the settings the platform sends:

For every model here that reads an alignment, the limit was measured with 16384 alignment rows, the most any of them reads, so a deeper a3m does not lower it.

Ask for more than a model's limit and tt-bio refuses before it opens a device, naming the model, the limit and any model that does take the input. nesso1 and af2ig have no measured limit and are never refused: nothing above the top of either ladder has been run, so there is no failing size to refuse on. AF2-IG's top rung is 1024 tokens (944 target residues plus an 80-residue binder) in 488 s on one Wormhole chip, and 1536 tokens (1008 plus a 528-residue binder) in 665 s on a Blackhole p150a. What sets each wall is in docs/large-targets.md.

boltzgen is the one model sized on atoms rather than residues, because its wall follows the target's atom count and atoms per residue vary with what the target is made of: the 14786-atom rung is 1831 residues of a deposited protein, and a lighter target reaches more residues in the same number of atoms. The designed binder is outside the number, which was 80 residues at every rung. tt-bio counts the atoms out of the structure file your file: entity points at, so the refusal still lands before a device opens.

Blackhole has more memory per chip, so it is a separate table and the Wormhole numbers do not carry over. Six models have been walked to a failure on a p150a:

The embedding wall is one allocation: the model asks for the whole sequence-by-sequence attention matrix as a single buffer that grows with the square of the sequence length, 34.4 GB at 131072 residues against a 31.9 GiB card. saprot-35m gets one rung further than the rest because its weights are the smallest and leave more room for it. Everything else is unmeasured on Blackhole and refused nothing, including boltzgen, which designs against a 16948-atom target there.

The Blackhole ladders have been walked separately. This is the largest rung each structure model is on record folding, from docs/size_ladder_baseline.d/:

Only opendde and opendde-abag are enforced on Blackhole, at 1024; every other model here accepts an oversized request rather than refusing it, so the table is what you have and not a guard. A blank rung above a model's top is unwalked, not a measured failure. 1536 is the top of the ladder itself, so the models sitting there have no failure on record at all.

The limits were measured with an MSA, which is the default for the models that take one, and at the deepest alignment the MSA pipeline actually produces. Folding single-sequence is roomier, so if you know your run is lighter than the ladder that set the limit, TT_BIO_SIZE_LIMIT=0 turns the refusal into a warning and runs it anyway.

A ligand counts against these limits. Its heavy atoms are tokens the model pays for exactly like residues, and on esmfold2, esmfold2-fast, openbind and protenix-v2 the wall is on tokens, so a cocrystal is checked on residues plus ligand atoms rather than on the residue count alone. esmfold2 folds 1664 residues, which leaves no room at all: 1664 residues plus any ligand is refused, and 1640 residues with a 24-atom ligand is admitted. openbind is the same: its 1664 was walked apo, so a ligand counts against it atom for atom. Either way the refusal names the token count and the wall, and it arrives before a device is opened instead of as an out-of-memory error part way through the fold.

esmfold2-fast is the same architecture at half the trunk depth. It has its own row because it was walked on its own; today both fold 1664 residues and fail at 1792.

The pair track switches to row-blocked execution at a size threshold smaller targets never reach, so their speed and numerics are untouched. See docs/large-targets.md. Perf levers are gated at several sequence lengths, not just one; the release gate re-checks the ladder against a recorded baseline. See docs/size-generality.md.

Options, Weights and MSA

All structure models support the sampling, output-format, and scheduling options. MSA, affinity, constraint, and auxiliary-output options apply only where listed below. Each model downloads its weights automatically on first use, except Protenix-v2, whose weights upstream does not allow us to redistribute (see docs/weights.md). tt-bio weights lists every artifact with its status, size and path; --download prefetches, --prune reclaims disk. Set TT_BIO_CACHE to move all of it (both ~/.boltz and the Hugging Face cache, about 65 GiB) somewhere with room. See docs/weights.md.

Boltz-2, Protenix-v1, Protenix-v2, OpenFold3, OpenDDE, and RF3 are MSA-dependent and use an MSA by default, a local ColabFold DB (~/.boltz/msa_db) if one is set up (see Offline MSA), otherwise the online ColabFold server. Sending sequences to the online server (api.colabfold.com) leaves your machine; a one-line notice is printed when that fallback is used. Pass --msa_db_path for a private offline database, or --single_sequence to deliberately fold without an MSA (lower accuracy; for batch-screening orphan sequences). A complex with two or more different protein sequences also gets a species-paired MSA, searched once per complex, the way each model's upstream pairs; a homodimer is not paired. Two models fold unpaired because their upstream does: RF3 pairs by taxonomy IDs that ColabFold alignments do not carry, and OpenFold3's preview2 checkpoint runs on an upstream release that drops the paired rows (OpenBind pairs). ESMFold2 needs no MSA and uses one when a source is given.

--fast makes some operations use a lower-precision numeric format that runs faster. Accuracy is typically very close; the Boltz-2 measurement is in docs/boltz2-fast-parity.md.

Many Inputs and Cards

predict accepts either a single YAML/FASTA file or a directory containing many input files. An input the model refuses is reported, recorded as failed in results.json and skipped, and the rest of the directory folds; the exit status is 2 when some inputs failed and 1 when all did. Every chain comes back under the id you gave it.

A live display shows the progress of each target. Prediction uses up to one card per pending target, labelled in the display (quietbox:tt0, quietbox:tt1, ...). Models load once per active card and stay resident:

tt-bio predict proteins/ --model boltz2 --out_dir results --fast

Pass --devices 0,1,2,3 to pick or limit the available cards. A single target remains a single-card fold; additional cards increase throughput only when multiple targets are queued.

Once enough folds run at once that each worker is down to a couple of CPU threads, tt-bio also has their idle thread pools sleep between device syncs instead of spinning on them. The output is identical either way, and a single fold is unaffected. See Tuning flags for the measurement, or set OMP_WAIT_POLICY yourself to take the decision back.

To spread work across several machines, see Running tt-bio on many machines.

Binding Affinity

Binding Affinity Prediction (Boltz-2)

tt-bio predict examples/affinity.yaml --model boltz2 --use_msa_server --override --affinity_mw_correction

The --affinity_mw_correction flag applies molecular weight correction for more accurate predictions.

An affinity run folds the complex and then runs a second model that has its own 64-block trunk, so it costs more than a structure-only fold. All of it runs on the card. FKBP12+SB3 at the default affinity protocol (200 sampling steps, 5 affinity samples, single sequence) takes about 206 s per ligand on one Blackhole p150a, measured as a whole tt-bio predict invocation with model load included. --sampling_steps_affinity and --diffusion_samples_affinity are the two flags that move that wall most.

The affinity trunk runs in fp32 because the predicted log10(IC50) is sensitive to activation precision, and that is not configurable. Earlier releases ran it in fp32 on the host CPU instead, which is why affinity used to take minutes per ligand and looked CPU-bound.

Binding Affinity Without a Structure (Nesso-1)

tt-bio affinity predicts protein-ligand affinity without folding anything. Nesso-1 has no structure module, so it returns a number, not coordinates:

tt-bio affinity examples/affinity.yaml                  # one complex
tt-bio affinity ligands/ --out_dir screen               # a directory is a screen

A directory keeps the model resident across inputs, so a ligand series against one target pays the weight load and the kernel compile once. Output is one <id>_affinity.json per input plus an affinity.csv for the whole run: the affinity value (mean of a two-member ensemble, and each member), a binary binder probability, and six distogram entropies.

On DAVIS it reaches 0.662 mean within-target Pearson against measured Kd (0.175 for a molecular-weight-only control), matching the 0.636 the upstream implementation gets on an H200.

It is far cheaper than folding for the same question. One 512 aa prediction takes 8.3 s of model time on one Blackhole card, 33 s for the whole command including featurisation; Boltz-2 affinity takes 386 s for the same command on the same input, which is what tt-bio shipped for this before. Against a GPU it is 7.9x off an H200 at that size, so choose it for what the answer costs on this hardware rather than expecting it to beat a GPU.

Use it to rank a series; use predict --model boltz2 when you need the pose. Proteins and ligands only, one ligand scored per input. One Wormhole chip scores a 3072-residue target with any ligand up to cobalamin's size in about 15 minutes. 3584 residues also completes but takes about two hours, because the chip's memory spills to the host; 4096 is refused. The trunk runs bf16 by default: it is about 6x faster than fp32 and no less accurate from 276 tokens up, and fp32 runs out of DRAM around 1000 tokens. On inputs under ~150 tokens fp32 is the more faithful arm, and --trunk fp32 switches back. See docs/nesso1.md for the input schema, the four upstream limits, and what to watch when comparing numbers against another implementation.

Design

Design new binders and protein structures from a target or motif specification, with one command:

tt-bio design examples/binder.yaml --model boltzgen --num_designs 10
tt-bio design specs.json --model rfd3 --from_pdb --out_dir designs/

BoltzGen designs binders against a target structure. The pipeline runs design → inverse folding → folding → analysis → filtering and writes the top-ranked binders to <out_dir>/final_ranked_designs/. Pass --seed N to make a design reproducible; without it every run draws fresh. Input grammar, protocols, pipeline subsets, and options: docs/boltzgen-design.md. Designability (scRMSD) QA: docs/boltzgen-designability.md.

RFdiffusion3 (RFD3) is an all-atom generative model that designs new protein structures and sequences from a specification, rather than folding an existing one. Design modes, the contig-string input grammar, and which conditioning fields a spec can and cannot ask for: docs/rfd3-design.md.

PXDesign generates binder backbones against a target structure, conditioned on a distogram of the target rather than its coordinates. Input is a target YAML naming a structure file, the chains to condition on (with optional per-chain crop and hotspots) and a binder_length; each design is written as a CIF in the target structure's own frame, so it opens alongside your input file. A designs.json lands beside them with each design's numbers: fit RMSD against the target, binder residue and atom counts, and how many target tokens it was conditioned on. The binder is written as GLY because PXDesign generates a backbone with no sequence. Hotspot residues are label_seq numbers, not the author numbering a viewer shows, and a number that names no residue is refused rather than dropped. --num_designs is also the batch axis for this model: every requested design comes from one batched diffusion trajectory, and the gain per design grows with the batch and shrinks with the target: 2.7x at 8 designs against a 256-residue target, 1.5x against a 512-residue one, and flat from 16 up rather than turning back. A given --seed and --num_designs always reproduce the same designs, but --num_designs 1 and --num_designs 2 do not share their design 0: asking for more designs currently changes which ones you get, so pin both values when you want a run back. Selecting designs, which upstream does with a Protenix and an AF2-IG filter, is not on the CLI yet.

Each model downloads its weights automatically on first use. BoltzGen and RFdiffusion3 fan out across every available card (--devices 0,2 restricts); PXDesign runs on one card locally, or one design per card across a host's controller with --controller http://127.0.0.1:8765. tt-bio gen still works as a deprecated alias for tt-bio design --model boltzgen.

How many designs a card returns per hour, how --num_designs and --devices move it, and how to size a campaign: docs/design-throughput.md.

BindCraft 2 is not a tt-bio model and has no CLI entry; it is a third-party design loop you install yourself, and tt_bio.bindcraft2 gives it an AlphaFold 2 Evoformer that runs on a card, interleaving as many design trajectories over that one chip as the box and card have memory for. On a roomy box that is three for a complex up to 352 tokens and fewer above it; at 288 tokens three take a completed trajectory from 927 s to 774 s and a round from 7.41 s to 6.00 s, 8.6x an H200 rather than 10.6x, with the same designs accepted; that is v0.11.0 on a Blackhole p150a. This release's gradient kernels take the same round on a p150a to 3.603 s against 0.11.0's 5.836 s on that card, 1.62x, 5.18x an H200, and on a p300c chip to 3.653 s against 5.883 s, 1.61x, both measured with the two releases alternating at AICLK 1350; both boards and the per-lever ratios. Its gradient loop runs on card, with the validation ensemble on BindCraft 2's own trunk so it stays the reference's. Its design objective stays yours: bindcraft2.loss_terms adds a loss term of your own, reweights or switches off one of BindCraft 2's 37, or replaces what one computes, from ordinary Python with the settings file untouched, and the term reaches the gradient, the acceptance scoring and losses.csv alike; bindcraft2.check_gradient grades it against float64 finite differences first, and a term whose gradient is silently zero is refused by name before a campaign spends on it (how). On the shipped PD-L1 example it accepts binders at 7 per 31 trajectories against BindCraft 2's own JAX at 1 per 5, which Fisher exact does not separate (p = 1.00). One p150a carries a complex up to 864 tokens at the default, including the 768 axis that crashed 0.11.0, and one Wormhole Galaxy chip up to 512 at full speed, counting the fused complex rounded to 32 rather than target plus binder; past that a Wormhole chip keeps going in a slower memory mode rather than refusing, which trades round time for room and is chosen for you by the default memory='auto': the largest axis supported on one Wormhole Galaxy chip is 864 tokens at 262.5 s a round and a 10.32 GB peak, 1.69x what the fast mode carries, and 800 tokens, a whole EGFR ectodomain plus a 150 aa binder with nothing cropped, runs at 222.9 s a round; tt_bio.bindcraft2.design_tokens reads that count off your settings in under a second without opening a card, so you can tell which board a target needs before you queue it, and a complex too large is refused with a message naming the axis, the memory and a smaller size that actually fits on the board in hand; an input it cannot design against, from a hotspot in an unresolved loop to a nucleotide sequence pasted in where the protein goes, is refused before a card is opened rather than quietly worked around. BindCraft 2 is under its authors' hosting-restricted licence: running it on your own cards is your own use under that licence, while offering it to third parties as a hosted service, API or agent tool needs a commercial licence from them, and tt-bio is not affiliated with or endorsed by the BindCraft 2 authors. What you need, how to point a campaign at a chip, what fits, and how to read that comparison: docs/bindcraft2.md.

Embeddings

Protein Embeddings (ESMC)

Turn protein sequences into ESMC language-model embeddings on-device (no folding, no MSA). DATA is a FASTA file, a directory of them, a YAML {id: sequence} mapping, or a bare sequence string:

tt-bio embed proteins.fasta --model esmc-600m --out_dir embeddings
tt-bio embed "MQIFVKTLTGKTITLEV..." --model esmc-600m   # one-off sequence

--model selects the ESMC variant (esmc-300m, esmc-600m, esmc-6b). For each sequence you get its per-residue embeddings ([length, d_model] float32, one row per amino acid, row order == input order) and a pooled whole-sequence vector ([d_model] float32, --pool mean/max/cls). --out_dir (default ./embeddings) gets:

• <id>.npz per sequence: per_residue, pooled (+ logits with --logits); --format npz, default
• embeddings.parquet: pooled vectors, one row per sequence; --format parquet
• manifest.json: model/pool/shapes/dtype and which file holds each sequence

Add --logits for the per-residue amino-acid predictions (300M/600M only), and --fast for the lower-precision weight path. Weights download automatically on first use.

Sequences batch automatically on 300M/600M (--batch_size, default 8): a padded, length-bucketed device forward per batch, masked so results are identical to running each sequence alone. Single-sequence calls (--batch_size 1, e.g. serving one sequence at a time) replay through a captured device trace once a length bucket repeats, up to ~1.5x faster per call on QuietBox-class hosts, bit-identical, no flags needed.

To embed a large batch faster, shard it across several cards with --devices 0,1,2,3: one worker per card, results reassembled in input order and identical to a single-card run:

tt-bio embed proteins.fasta --model esmc-600m --devices 0,1,2,3

Fanout only pays off when there's enough work per shard to amortize each worker's model-load and device-init cost. On small batches it can be flat or worse than a single card. esmc-6b scales to 4 cards on suitably large batches. Reach for --devices on large batches, not small ad-hoc jobs; use --controller (below) for repeated/production embedding.

For repeated/production embedding, submit to a persistent pool instead: a worker loads its model once and keeps it resident across every call, so the reload cost above is paid once per worker, not once per invocation:

tt-bio controller --port 8765            # starts + keeps a worker per local card
tt-bio embed proteins.fasta --model esmc-6b --controller http://127.0.0.1:8765

The same capability is available from Python:

from tt_bio import esmc

emb = esmc.embed("MQIFVKTLTGKTITLEV...", model="esmc-600m")[0]
emb.per_residue   # [L, d_model] float32
emb.pooled        # [d_model] float32

# Shard a large set across cards (data-parallel, order preserved):
embs = esmc.embed(sequences, model="esmc-600m", devices=[0, 1, 2, 3])

Structure-Aware Protein Embeddings (SaProt)

SaProt is a structure-aware protein language model, an ESM-2 encoder over a fused amino-acid + Foldseek 3Di vocabulary (446 tokens). Where ESMC is sequence-only, SaProt also encodes local structure, so its embeddings and MLM logits reflect both sequence and shape. Use it for variant-effect / mutation-fitness scoring and function prediction when you have a structure (predicted or experimental: fold it with tt-bio predict first, then score it with SaProt).

tt-bio saprot proteins.fasta --model saprot-650m --structure structs/ --out_dir embeddings
tt-bio saprot proteins.fasta --model saprot-650m                # sequence-only (3Di = '#')
tt-bio saprot proteins.fasta --model saprot-650m --devices 0,1    # data-parallel across 2 cards

--structure is a PDB/cif file (single sequence) or a directory of <id>.pdb/<id>.cif files, one per FASTA id. The 3Di structural tokens are computed on host with Foldseek (conda install -c bioconda foldseek; --foldseek PATH or FOLDSEEK_BIN if it is not on PATH); it runs off-device. Residues the structure does not resolve get the # unknown-structure token, and a structure that is not of the sequence you passed is refused rather than lined up by length. Omit --structure for sequence-only mode (lower accuracy for 35M/650M; the 1.3B works sequence-only).

For each sequence you get per-residue structure-aware embeddings ([length, d_model] float32) and a pooled vector, plus per-residue MLM logits ([length, 446] with --logits) over the fused vocabulary, the log-likelihoods used for zero-shot mutation scoring. Output layout matches tt-bio embed (<id>.npz / embeddings.parquet / manifest.json).

--model selects the variant (saprot-35m, saprot-650m, saprot-1.3b). --devices 0,1,2,3 shards the input across cards data-parallel (one pinned subprocess each, results reassembled in input order), bit-exact vs single-card with --batch_size 1. Parity vs the reference HuggingFace checkpoint, the multi-card bit-exactness check, and warm throughput are in docs/saprot-parity.md.

Python:

from tt_bio import saprot

emb = saprot.embed(("MQIFVKTLTGKTITLEV...", "dweweaepvrdidi..."), model="saprot-650m")[0]
emb.per_residue   # [L, d_model] float32, structure-aware
emb.logits        # [L, 446] float32 (with return_logits=True)

Input Format

ESMFold2, Protenix-v1 and Protenix-v2 accept proteins, DNA, RNA, ligands and covalent bond constraints. OpenFold3 accepts proteins, DNA and RNA plus per-chain templates, and refuses ligands. OpenDDE accepts proteins and ligands with bond constraints. Boltz-2 additionally supports affinity, pocket/contact constraints, potentials, and user-supplied templates. Which model takes cyclic chains and which kind of bond is in docs/model-capabilities.md.

Create a YAML file describing your complex:

version: 1
sequences:
  - protein:
      id: A
      sequence: MVTPEGNVSLVDESLLVGVTDEDRAVRSAHQFYERLIGLWAPAVMEAAHELGVFAALAEAPADSGELARRLDCDARAMRVLLDALYAYDVIDRIHDTNGFRYLLSAEARECLLPGTLFSLVGKFMHDINVAWPAWRNLAEVVRHGARDTSGAESPNGIAQEDYESLVGGINFWAPPIVTTLSRKLRASGRSGDATASVLDVGCGTGLYSQLLLREFPRWTATGLDVERIATLANAQALRLGVEERFATRAGDFWRGGWGTGYDLVLFANIFHLQTPASAVRLMRHAAACLAPDGLVAVVDQIVDADREPKTPQDRFALLFAASMTNTGGGDAYTFQEYEEWFTAAGLQRIETLDTPMHRILLARRATEPSAVPEGQASENLYFQ
  - ligand:
      id: B
      smiles: 'N[C@@H](Cc1ccc(O)cc1)C(=O)O'
properties:
  - affinity:
      binder: B

Entity Types:

• Polymers (protein, dna, rna): provide sequence
• Ligands (ligand): provide smiles or ccd code

Multiple Identical Chains:

- protein:
    id: [A, B]  # Two identical chains
    sequence: ...

Proteins with Custom MSA

- protein:
    id: A
    sequence: MVTPEGNVSLVDES...
    msa: ./path/to/msa.a3m

Proteins with Modifications

- protein:
    id: A
    sequence: MVTPEGNVSLVDES...
    modifications:
      - position: 5
        ccd: PTR  # Modified residue code

Ligands

- ligand:
    id: B
    smiles: 'CC1=CC=CC=C1'  # SMILES string
    # OR
    ccd: ATP                # CCD code

Constraints

Pocket and contact constraints are Boltz-2 only (they need a trained constraint embedder). A covalent bond to a ligand or a modified residue works on every structure model that takes the ligand. A bond between two standard residues (a disulfide) works on Boltz-2, ESMFold2, Protenix and OpenDDE; RF3 and the OpenFold3 family refuse it by name.

Pocket Constraints (binding site):

constraints:
  - pocket:
      binder: B              # Ligand chain
      contacts: [[A, 10], [A, 11], [A, 12]]  # Binding site residues
      max_distance: 6.0      # Angstroms (4-20A, default 6A)
      force: false           # Use potential to enforce (default: false)

Contact Constraints:

constraints:
  - contact:
      token1: [A, 10]
      token2: [A, 50]
      max_distance: 8.0
      force: false

Bond Constraints (covalent link, e.g. a covalent inhibitor, glycosylation, or disulfide):

constraints:
  - bond:
      atom1: [A, 10, SG]     # [chain, residue, atom]
      atom2: [B, 1, C1]      # SMILES ligand: element + count in SMILES order

OpenDDE + covalent bonds: OpenDDE honors a bond constraint between a protein residue and a ligand atom (the covalent-inhibitor case) or between two protein residues (a disulfide or crosslink). Both ride the same token_bonds machinery as Protenix-v2 and are honored in the output (device-verified against upstream OpenDDE within the reference's own seed noise floor); see examples/opendde_covalent_ligand.yaml and examples/opendde_covalent_bond.yaml.

Templates

Use experimental structures as templates:

templates:
  - cif: ./template.cif
    chain_id: A
    template_id: A
    force: true              # Enforce template alignment
    threshold: 2.0           # Max deviation in Angstroms

chain_id names the chains to template (default: every protein chain) and template_id the template's chains by mmCIF label_asym_id (default: the best match). Each chain is aligned to the template's sequence for you. This block works on boltz2, protenix-v2, opendde, opendde-abag, openfold3, openbind and rf3. force (with its threshold) and pdb files are Boltz-2 only; the other models refuse them, and RF3 takes one template per chain.

The same models except boltz2 also take a precomputed alignment .npz per protein chain (the format the upstream benchmark cache ships); Boltz-2 refuses it and takes the template as a cif instead. Its structures are fetched from RCSB, and a missing one is a hard error rather than a silently dropped template. See examples/7xi5_tmpl.yaml. There is no template search.

sequences:
  - protein:
      id: A
      sequence: MSSATPDPAEILT...
      templates: ./templates.npz

Understanding Results

Output Structure

<model>_results_prot/   # e.g. protenix_results_prot, boltz2_results_prot
├── structures/
│   ├── prot.cif                      # Best-ranked predicted structure
│   └── prot_model_1.cif              # Additional samples (if diffusion_samples > 1)
├── results.json                      # One entry per target with confidence/affinity metrics
├── power_profile.csv                 # (optional, --report-energy)
├── power_profile.png                 # (optional, --report-energy)
├── prot_pae.npz                      # (optional, --write_pae)
├── prot_plddt.npz                    # (optional, --write_pae, Boltz-2)
├── prot_pde.npz                      # (optional, --write_pde)
└── prot_embeddings.npz               # (optional, --write_embeddings)

MSA results are cached in <out_dir>/msa/ (default ./msa/), keyed by sequence hash. The same protein sequence is never searched twice, even across different input files or runs. The MSA search uses all available CPU threads and keeps the database index memory-mapped for maximum speed.

Confidence Scores

Each target entry in results.json contains confidence metrics. The fields below are Boltz-2's; Protenix-v2 and OpenFold3 report the same confidence_score / ptm / iptm / plddt (and all_runs when --diffusion_samples > 1, ranked best-first), while an ESMFold2 entry instead carries plddt (mean, 0-1), ptm when available, and n_residues / n_chains. Every model reports its complex mean pLDDT under plddt. Boltz-2, Protenix-v2 and OpenDDE also report the two per-chain fields below on a multi-chain target, in all_runs as well as for the best sample.

{
    "id": "prot",
    "status": "ok",
    "confidence_score": 0.84,
    "ptm": 0.84,
    "iptm": 0.82,
    "complex_plddt": 0.84,
    "plddt": 0.84,
    "chains_ptm": {
        "0": 0.85,
        "1": 0.83
    },
    "pair_chains_iptm": {
        "0": {"0": 0.85, "1": 0.72},
        "1": {"0": 0.82, "1": 0.83}
    }
}

• confidence_score: Overall confidence (0-1, higher is better). Models are ranked by it and the best sample is written as <name>.cif. Boltz-2 and BoltzGen use their own published rule, 0.8 × complex_plddt + 0.2 × iptm, or 0.2 × ptm on a single chain where there is no interface and iptm is 0; it is not a pLDDT and can sit either side of one, on CDK2 0.008 above complex_plddt at 298 aa and 0.050 below it at 512 aa. The AlphaFold3-family models (OpenFold3, OpenBind-0, RF3, Protenix-v1/v2, OpenDDE) share one rule: AlphaFold3's 0.8 × iptm + 0.2 × ptm + 0.5 × disorder − 100 × clash, with plddt taking ipTM's weight on a single chain, where ipTM is zero by construction. OpenFold3 and OpenBind-0 are the only two that compute the disorder and clash terms; the others pass 0 for them. RF3 reports the same number as ranking_score. So the family's scores are comparable to each other but not to Boltz-2's
• ptm: Predicted TM-score for complex (0-1)
• iptm: Interface TM-score (0-1)
• complex_plddt, plddt: Mean confidence (0-1), the same value under both names. It is the mean of the B-factor column of the structure file the same fold wrote, so averaging that column reproduces it. Boltz-2 writes one pLDDT per residue, so average over one atom per residue (CA); Protenix-v2, OpenFold3, OpenBind-0 and OpenDDE write one per atom, so average over all of them
• chains_ptm: Per-chain TM-scores (0-1)
• pair_chains_iptm: Per-chain-pair interface TM-scores (0-1), with each chain's own chains_ptm on the diagonal. Read pair_chains_iptm[binder][target] to score one named interface of a complex; the global iptm is the whole-interface number and on a two-chain target the two agree. Like every other confidence value, these are comparable between targets of the same model, not between models

Interface Scores

With --write_pae, a multi-chain Boltz-2 entry also carries interface_scores: ipSAE (both directions, their max and their min), ipTM, interface pAE, pDockQ, pDockQ2 and LIS for every chain pair. The definitions match the script Adaptyv scored its Nipah binder competition with, at the same 15 A cutoffs. interface_score_distribution gives the same scores for every diffusion sample, with mean, standard deviation and range, so --diffusion_samples 5 shows how stable a design's score is as well as its value. To score a fold you already have, from tt-bio or from upstream Boltz, without a device:

tt-bio score prot_model_0.cif pae_prot_model_0.npz --plddt plddt_prot_model_0.npz

Tenstorrent arithmetic differs from a GPU's, so the same inputs fold to slightly different numbers than they do on CUDA. See docs/interface-scores.md for the definitions, the choices made where the reference is ambiguous, and how this relates to the ipSAE BoltzGen reports while designing.

Affinity Predictions

For affinity targets, the same results.json entry also contains:

{
    "affinity_pred_value": 2.47,
    "affinity_probability_binary": 0.41,
    "affinity_pred_value1": 2.55,
    "affinity_pred_value2": 2.19,
    "affinity_probability_binary1": 0.50,
    "affinity_probability_binary2": 0.42
}

• affinity_probability_binary: Probability of binding (0-1). Use for hit discovery (higher = more likely to bind)
• affinity_pred_value: Predicted binding affinity as log10(IC50) in μM. Use for ligand optimization (lower = stronger binding). Only compare between known active molecules
• affinity_pred_value1, affinity_pred_value2: Individual model predictions for binding affinity
• affinity_probability_binary1, affinity_probability_binary2: Individual model predictions for binding probability
• runtime_s: Wall-clock seconds for the whole target, structure and affinity together. Affinity targets also carry structure_runtime_s and affinity_runtime_s; the affinity leg is normally the larger of the two by several times, so read the split before pricing a screen

Weights and MSA

Weights

Weights download on first use, so nothing here is required. tt-bio weights is for when you want to see or move them:

tt-bio preflight protenix-v1         # can this machine run it right now?
tt-bio weights                       # every artifact: status, size, resolved path
tt-bio weights --download            # prefetch everything (e.g. before going offline)
tt-bio weights --download boltz2     # or just one model's set
tt-bio weights --prune               # reclaim superseded revisions and leftovers

tt-bio preflight answers before you submit a job, and exits non-zero when something is missing, so it works in a script. When weights are missing it also measures whether the hosts they come from can be reached from this machine.

A full set is about 65 GiB. It lands in ~/.boltz and the Hugging Face cache; set TT_BIO_CACHE to put both somewhere with more room. Each artifact also takes its own override, so TT_BIO_BOLTZ2_CONF=/mnt/weights/boltz2_conf.ckpt loads that file instead of downloading. Rows show as corrupt if a download was interrupted, and are re-fetched rather than loaded. No download waits forever: a source that sends nothing is dropped for the next one, and the error names every host tried. See docs/weights.md.

Offline MSA (Optional)

A local database avoids the online MSA server and is faster for repeated runs, if you have the disk and RAM for it.

tt-bio msa
tt-bio predict examples/prot.yaml --model boltz2 --override

tt-bio msa downloads UniRef30 to ~/.boltz/msa_db (~100GB download, ~500GB on disk after indexing). predict auto-detects this path.

EnvDB can improve MSA coverage when UniRef30 hits are weak, at higher disk and RAM cost. To add it and use it in prediction:

tt-bio msa --db all
tt-bio predict examples/prot.yaml --model boltz2 --use_envdb --override

Shared MSA Server (Optional)

Host the database on one machine and let others fetch MSAs from it over HTTP, so each prediction machine need not keep its own ~500GB copy.

# On the machine with the database:
tt-bio msa-server --listen 0.0.0.0:8765

# On any other machine (no local database needed):
tt-bio predict examples/prot.yaml --model protenix-v2 --msa_endpoint http://HOST:8765

The server runs the same offline colabfold_search and serves unpaired {hash}.a3m, with a shared cache and a search-concurrency cap (--max_concurrent). Add --token to require Authorization: Bearer <token>. --msa_endpoint applies to --model esmfold2, protenix-v1, protenix-v2, openfold3, opendde, and rf3.

MSA Server Authentication

For --use_msa_server:

Basic Authentication:

export BOLTZ_MSA_USERNAME=myuser
export BOLTZ_MSA_PASSWORD=mypassword
tt-bio predict ... --model boltz2 --use_msa_server

API Key Authentication:

export MSA_API_KEY_VALUE=your-api-key
tt-bio predict ... --model boltz2 --use_msa_server

Command-Line Options

Model-specific options are labelled below.

Common Options:

Affinity-Specific Options (Boltz-2):

MSA Options (Boltz-2, Protenix-v1, Protenix-v2, OpenFold3, OpenBind-0, OpenDDE and RF3 use an MSA by default; ESMFold2 only when requested):

MSA Database Setup Options:

Tuning Flags

The engine ships its device optimizations on. Each one is an environment variable you can set to 0 to fall back to the path it replaced, which is what you want if you are bisecting a result. "Bit for bit" means the same structure, byte for byte; a flag that is not bit-exact says how far it moves a structure, next to the seed-to-seed spread.

What each one is worth in seconds, at what clock, and how it was measured: docs/tuning-flags.md.

Running tt-bio on many machines

Each machine runs its own tt-bio controller, which listens on 127.0.0.1 and keeps a worker on every chip. Work reaches it through --controller, and whatever spreads jobs across machines sits on top: a platform, a cluster scheduler, or the fifty-line examples/many_hosts.py, which folds a directory across several hosts over ssh. docs/multi-host.md is the contract a scheduler builds against: the endpoints, what a host advertises, the lease and how a result settles once.

Optional: Energy Measurement (Boltz-2)

Use --report-energy to profile energy during prediction:

tt-bio predict examples/686.yaml --model boltz2 --override --device_ids 0 --report-energy --energy-metric both --energy-sample-hz 5

Behavior:

• Select metric channel(s) with --energy-metric (tdp, input, both)
• Uses one sampling rate (--energy-sample-hz, default 20 Hz)
• Supports only Tenstorrent runs with one selected device
• Records two power channels when available:
  • power_w: tt-mgmt UMD telemetry power (TDP channel)
  • input_power_w: tt-mgmt UMD telemetry input power
• Requires optional tt-mgmt installation:
  • git clone --recursive https://github.com/aperezvicente-TT/tt-mgmt.git
  • pip install -e ./tt-mgmt
• Prints energy summary metrics for selected channels
• Always writes:
  • power_profile.csv
  • power_profile.png

Changing a Model

Your own output head runs on a Boltz-2 or ESMFold-2 fold's trunk outputs with --head, and writes what it returns next to the structure:

tt-bio predict target.yaml --model boltz2 --head my_heads.py:ContactHead

The head is ordinary PyTorch on the host, so on Boltz-2 it runs the same on a CPU and beside a Tenstorrent card, and it cannot change the structure. Which change takes which route (a head, a BindCraft 2 loss, fine-tuning, a changed layer) and where each stops: docs/extending.md, with a runnable contact-head example in examples/custom_head/. Porting a model tt-bio does not ship, and what it took for the ones it does: docs/porting-a-model.md.

Training

Train OpenFold3 with the same forward the inference path uses:

pip install 'tt-bio[tenstorrent,train]'
tt-bio train --model openfold3

The first run downloads the OpenFold3 weights (2.29 GB, no login). With no data given, it fetches upstream's 8-structure training sample (73 MB, from OpenFold3's public bucket, checked file by file against a shipped sha256 list), trains the model's own weights at a 384-token crop for one pass over it, and writes to runs/openfold3. A step takes about 42 s on one p300c chip. Run the same command again and it resumes from the last checkpoint; run it with different settings on the same --out and it refuses rather than mixing two runs.

Your own data is OpenFold3's training-set layout, a pdb_training_set/ directory beside one training_cache*.json, featurised on the fly by upstream's own pipeline:

tt-bio train data/ --model openfold3 --steps 2000
tt-bio train data/ --model openfold3 --steps 2000 --chips 1,2 --global-batch 2
tt-bio train data/ --model openfold3 --dry-run      # does it fit, and how long a step takes

A run ends by writing <out>/weights.pt, the trained weights in the same format as the shipped checkpoint, whether it ran on one chip or several. Fold with them by passing that file to predict:

tt-bio predict target.yaml --model openfold3 --checkpoint runs/openfold3/weights.pt

Without --checkpoint, predict folds with the shipped weights exactly as before.

--help lists the next layer: --steps, --chips, --global-batch, --lr, --warmup-steps, --checkpoint-every, --tokens, --seed. --help-all adds the expert ones: precision (--exact), the objective, the recipe, and --train with the LoRA shape. OpenFold3 trains its own weights only; --train adapters is refused for it with what to do instead. docs/training.md has the data layout and the rest.

For an agent, everything a run says it also writes, in JapanFold's vocabulary:

The exit code is 0 on success and 1 on failure, with a one-line reason on stderr. A step whose loss or gradient norm is not finite stops the run rather than training on it. After a resume the progress rows after the last checkpoint appear twice, once from each process.

Underneath the command are three more levels, and you pick the one that matches what you want to write:

Dropping a level is not a rewrite. tt-bio train --show-recipe prints the loop the command runs, the body of train.recipes.source("default"), written only in names the level below exports, and a test keeps it that way: if the recipe ever needed a private hook, the test fails and the hook becomes public. More chips is mesh=train.Mesh({"dp": [1, 2]}) one level down, and adapters instead of weights is train="adapters"; one loop body serves both, which the escape-hatch test checks instruction for instruction.

What works today: OpenFold3 end to end from the command line (data, featurisation, the weights, training its weights, checkpoints, resume, the agent files, folding with the result), the dry run, gradient checking, and data parallelism across the chips in one box. What does not: OpenFold3 is the only model that ships a training featuriser; --model offers only it, and another model registers its own with tt_bio.train.catalogue.register. LoRA adapters are not offered for OpenFold3: they have never been measured on it, and predict loads full weights only.

tt-bio train follows OpenFold3's optimizer setup rather than Adam's library defaults, which differ in three places that no loss curve shows: betas=(0.9, 0.95), no weight decay, and the AlphaFold 2 learning-rate schedule. Each is an argument, and the loop clips every sample separately, so a batch of 8 is 8 forwards per step. See docs/training.md for what each one costs if you get it wrong.

A training step runs entirely on the device, with the softmax backward in fp32. On OpenFold3 that keeps the gradient inside its accuracy bar against upstream. --exact computes softmax and layer norm in float64 on the host instead, a diagnostic reference that makes a step many times slower. Inference is unaffected. See docs/training.md.

ABodyBuilder3 wants its data staged first: data.tar.gz from Zenodo 10.5281/zenodo.11354577, extracted so that structures/structures/*.pt sits under the path you pass. Fine-tuning also wants their checkpoint beside it; without one the command refuses rather than adapting a random initialisation. To train from scratch instead, scripts/abb3_port/repro.py is the reproduction's own entry point.

Four things the API enforces rather than documents, because each is a bug we hit:

• plan() answers from measured numbers or returns UNMEASURED. It refuses a crop size whose forward is measured to run out of memory instead of estimating one, and it will not report a 4-chip step time from a 2-chip measurement. Protenix's own 384-token crop is one of the refusals.
• The optimizer refuses a bfloat16 master copy of the weights. An update accumulated at bfloat16 stops moving the weight while the gradient still looks healthy.
• opt.step() raises if you spread training over several chips and never gave it a way to combine their gradients. Otherwise you train one model per chip and see one loss curve.
• Every run records the clock it actually ran at, sampled during the work, plus the seed and the commit. A time without its clock is not a measurement on this hardware.

Global batch is always yours to set and is never derived from how many chips you have, so a recipe means the same thing on a bigger box.

Training on several chips is one flag, --chips 2, or one argument, mesh=train.Mesh({"dp": [0, 1]}). Under it a launcher runs your program once per chip and sums the gradients between them, the way torchrun does, so the program has to be re-runnable and must not open a card before the finetune call. Both are checked before anything starts. Two p150a chips on a QuietBox measured 1.96x at a 0.33 MB adapter gradient and 1.70x at 5.24 MB, both at 1350 MHz; the gap is host-side Adam contending between the two processes, not the exchange, which costs 2.3 % of the step. Training OpenFold3's weights at a 384-token crop, two p300c chips on a QuietBox take 46.5 s per two-sample step against 77.4 s on one, 1.66x, both at 1350 MHz; each step exchanges a 2.1 GB gradient through shared memory. ABodyBuilder3 on one QuietBox's four chips measures 3.69x at 92 %, 7.65 s a step against 28.19 s on one, with one master hash across the four ranks. It needs the host's cores split between the ranks, which the launcher does by default; left at torch's own width every rank claims all 16 cores and the same step takes 931 s. Two boxes work too and the link is what it costs: the same two chips split across two QuietBoxes give 1.554x where one box gives 1.887x, all of the difference being the 28.4 MB gradient exchange over the second box's WiFi. That box has no cable, so 77.7 % efficiency is close to the floor for this; a wired link would cut the exchange to about 2 % of the step. Name the other box on the mesh, train.Mesh({"dp": [0, 1]}, hosts=["ttuser@tt-quietbox2"]), and it composes the rendezvous the transport takes. Starting the ranks on both boxes is still your own launcher rather than --chips.

Every run carries the check that makes a multi-chip number mean something. The ranks' weights must stay identical, so the launcher compares every rank's master weights at the end and refuses a run where they differ, and it reads each rank's chip off that rank's own open file descriptors rather than trusting TT_VISIBLE_DEVICES, which names a different number than the device node. Both failures look like a healthy run otherwise.

Tiers, cut lines, the escape-hatch test and where plan() gets its numbers: docs/training.md. What one gradient step through a pairformer stack costs, how it scales with depth and width, and where the headroom is: docs/gradient-step-cost.md.

Cite

If you use this code or the models in your research, please cite the following papers:

@article{passaro2025boltz2,
  author = {Passaro, Saro and Corso, Gabriele and Wohlwend, Jeremy and Reveiz, Mateo and Thaler, Stephan and Somnath, Vignesh Ram and Getz, Noah and Portnoi, Tally and Roy, Julien and Stark, Hannes and Kwabi-Addo, David and Beaini, Dominique and Jaakkola, Tommi and Barzilay, Regina},
  title = {Boltz-2: Towards Accurate and Efficient Binding Affinity Prediction},
  year = {2025},
  doi = {10.1101/2025.06.14.659707},
  journal = {bioRxiv}
}

@article{stark2025boltzgen,
  author = {Stark, Hannes and Faltings, Felix and Choi, MinGyu and Xie, Yuxin and Hur, Eunsu and O'Donnell, Timothy John and Bushuiev, Anton and U{\c c}ar, Talip and Passaro, Saro and Mao, Weian and Reveiz, Mateo and Bushuiev, Roman and Pluskal, Tom{\'a}{\v s} and Sivic, Josef and Kreis, Karsten and Vahdat, Arash and Ray, Shamayeeta and Goldstein, Jonathan T. and Savinov, Andrew and Hambalek, Jacob A. and Gupta, Anshika and Taquiri-Diaz, Diego A. and Zhang, Yaotian and Hatstat, A. Katherine and Arada, Angelika and Kim, Nam Hyeong and Tackie-Yarboi, Ethel and Boselli, Dylan and Schnaider, Lee and Liu, Chang C. and Li, Gene-Wei and Hnisz, Denes and Sabatini, David M. and DeGrado, William F. and Wohlwend, Jeremy and Corso, Gabriele and Barzilay, Regina and Jaakkola, Tommi},
  title = {BoltzGen: Toward Universal Binder Design},
  year = {2025},
  doi = {10.1101/2025.11.20.689494},
  journal = {bioRxiv}
}

@article{wohlwend2024boltz1,
  author = {Wohlwend, Jeremy and Corso, Gabriele and Passaro, Saro and Getz, Noah and Reveiz, Mateo and Leidal, Ken and Swiderski, Wojtek and Atkinson, Liam and Portnoi, Tally and Chinn, Itamar and Silterra, Jacob and Jaakkola, Tommi and Barzilay, Regina},
  title = {Boltz-1: Democratizing Biomolecular Interaction Modeling},
  year = {2024},
  doi = {10.1101/2024.11.19.624167},
  journal = {bioRxiv}
}

@misc{candido2026language,
  author = {Candido, Salvatore and Hayes, Thomas and Derry, Alexander and Rao, Roshan and Lin, Zeming and Verkuil, Robert and others},
  title = {Language Modeling Materializes a World Model of Protein Biology},
  year = {2026},
  url = {https://biohub.ai/papers/esm_protein.pdf},
  note = {Preprint; ESMC / ESMFold2}
}

@misc{protenix2025,
  author = {{ByteDance AML AI4Science Team}},
  title = {Protenix: An AlphaFold3 Reproduction for Biomolecular Structure Prediction},
  year = {2025},
  url = {https://github.com/bytedance/Protenix}
}

@misc{openfold3,
  author = {{OpenFold Consortium}},
  title = {OpenFold3: An Open-Source Reproduction of AlphaFold3},
  year = {2026},
  url = {https://github.com/aqlaboratory/openfold-3}
}

@article{butcher2025rfdiffusion3,
  author = {Butcher, Jasper and Krishna, Rohith and Mitra, Raktim and Brent, Rafael Isaac and Li, Yanjing and Corley, Nathaniel and Kim, Paul T and Funk, Jonathan and Mathis, Simon Valentin and Salike, Saman and Muraishi, Aiko and Eisenach, Helen and Thompson, Tuscan Rock and Chen, Jie and Politanska, Yuliya and Sehgal, Enisha and Coventry, Brian and Zhang, Odin and Qiang, Bo and Didi, Kieran and Kazman, Maxwell and DiMaio, Frank and Baker, David},
  title = {De novo Design of All-atom Biomolecular Interactions with RFdiffusion3},
  year = {2025},
  doi = {10.1101/2025.09.18.676967},
  journal = {bioRxiv}
}

In addition if you use the automatic MSA generation, please cite:

@article{mirdita2022colabfold,
  title={ColabFold: making protein folding accessible to all},
  author={Mirdita, Milot and Sch{\"u}tze, Konstantin and Moriwaki, Yoshitaka and Heo, Lim and Ovchinnikov, Sergey and Steinegger, Martin},
  journal={Nature methods},
  year={2022}
}

License

tt-bio is released under the MIT License (see LICENSE) and is built on the MIT-licensed Boltz-2 / Boltz-1 code. It bundles third-party code, each under its upstream license: the ESMFold2 host-side reference under tt_bio/_vendor/ (the esm pipeline, MIT, © Chan Zuckerberg Biohub; and the HuggingFace ESMFold2 model definition, Apache-2.0), the OpenFold3 host-side data pipeline under tt_bio/_vendor/openfold3/ (Apache-2.0, OpenFold Consortium), the BoltzGen binder-design source under tt_bio/boltzgen/ (MIT, © Hannes Stärk), and RF3's host featurizer under tt_bio/_vendor/rf3/, tt_bio/_vendor/foundry/ and tt_bio/_vendor/atomworks/ (BSD-3-Clause, University of Washington / Institute for Protein Design). Protenix-v2, OpenFold3's on-device model, RFdiffusion3, RF3's on-device model, and PXDesign's are independent ttnn reimplementations (no upstream compute code is vendored); tt-bio does not download the Protenix-v2 weights, which upstream's v2 release note says may not be redistributed without the rights holder's consent, and Protenix-v1's v0.5.0 checkpoint downloads under Apache-2.0; RFdiffusion3's and RF3's checkpoints download directly from the Institute for Protein Design (BSD-3-Clause), PXDesign's generator checkpoint downloads from ByteDance's release host under Apache-2.0 and its AF2-IG selection stage reads DeepMind's AlphaFold2 monomer pTM parameters (CC BY 4.0, from storage.googleapis.com/alphafold/), and OpenFold3's of3-p2-155k.pt and OpenBind-0's of3-ob-2025-06-30-174k.pt are the consortium's ungated public parameter releases under Apache-2.0, downloaded from upstream's bucket. See NOTICE for sources, versions, and modifications.