rush-py 3.0.0

Python SDK for interacting with the QDX Rush API and modules

rush-py

Quickstart

This document will walk through executing jobs on the Rush platform, by demonstrating how to prepare a protein. For a comprehensive guide on the concepts and constructing a full workflow, see the full rush-py explainer document.

First, install the following modules via pip—we require Python ≥ 3.9:

pip install rush-py pdb-tools

0) Code Sample

See the detailed breakdown in sections.

# Get a pdb to work with - we use the pdb-tools cli here
# but you can download directly from rcsb.org
!pdb_fetch '1brs' | pdb_selchain -A | pdb_delhetatm > '1B39_A_nohet.pdb'

# ...import the dependencies and set your configuration
from pathlib import Path
import rush

os.environ["RUSH_TOKEN"] = YOUR_TOKEN

# 1.3 Build your client
client = rush.build_blocking_provider_with_functions()

# 2.1 Prepare the protein
prepared_protein_qdxf, prepared_protein_pdb = client.prepare_protein(
    Path("1B39_A_nohet.pdb"), None, None, tags=["example_prep"]
)

# 2.3 Return run values
print(prepared_protein_qdxf.download(overwrite=True).open().read()[0:50], "...")

2024-04-08 17:12:28,141 - rush - INFO - Not restoring by default via default
2024-04-08 17:12:29,451 - rush - INFO - Argument 10ef4d21-fac5-4962-973b-cf0f6e26d964 is now ModuleInstanceStatus.RESOLVING
2024-04-08 17:12:40,388 - rush - INFO - Argument 10ef4d21-fac5-4962-973b-cf0f6e26d964 is now ModuleInstanceStatus.ADMITTED
2024-04-08 17:12:44,734 - rush - INFO - Argument 10ef4d21-fac5-4962-973b-cf0f6e26d964 is now ModuleInstanceStatus.DISPATCHED
2024-04-08 17:12:45,825 - rush - INFO - Argument 10ef4d21-fac5-4962-973b-cf0f6e26d964 is now ModuleInstanceStatus.RUNNING
2024-04-08 17:12:59,570 - rush - INFO - Argument 10ef4d21-fac5-4962-973b-cf0f6e26d964 is now ModuleInstanceStatus.AWAITING_UPLOAD
[{"topology": {"version": "V1", "symbols": ["N", " ...

1) Setup

This is where we prepare the rush client, directories, and input data we’ll be working with.

1.0) Imports

import json
from pathlib import Path

from pdbtools import pdb_delhetatm, pdb_fetch, pdb_selchain

import rush

1.1) Credentials

Retrieve your API token from the Rush UI.

You can either set the RUSH_URL and RUSH_TOKEN environment variables or provide them as variables to the client directly.

To see how to set environment variables, Wikipedia has an extensive article.

os.environ["RUSH_TOKEN"] = YOUR_TOKEN

1.2) Configuration

Lets set some global variables that define our project. These are not required, but are good practice to help organize the jobs that will be persisted under your account.

Make sure you create a unique set of tags for each run. Good practice is to have at least each of the experiment name and system name as a tag.

EXPERIMENT = "rush-py-quickstart"
SYSTEM = "1B39"
TAGS = ["qdx", EXPERIMENT, SYSTEM]

1.3) Build your client

Get our client, which we’ll use for calling modules and generally for using the Rush API.

As mentioned earlier, url and access_token are optional if you have set the env variables RUSH_URL and RUSH_TOKEN respectively.

batch_tags will be applied to each run that is spawned by this client.

A folder called .rush will be created in your workspace directory (defaults to the current working directory, can be overridden by passing workspace= to the provider builder).

# By using the `build_provider_with_functions` method,
# we will also build helper functions calling each module
client = rush.build_blocking_provider_with_functions(batch_tags=TAGS)

2024-04-08 17:13:26,467 - rush - INFO - Not restoring by default via default

1.4) Input selection

Fetch a pdb from RCSB, stripping hetatoms and selecting a single chain to pass as input to the modules:

PROTEIN_PDB_PATH = client.workspace / f"{SYSTEM}_P.pdb"

complex = list(pdb_fetch.fetch_structure(SYSTEM))
protein = pdb_delhetatm.remove_hetatm(pdb_selchain.select_chain(complex, "A"))
with open(PROTEIN_PDB_PATH, "w") as f:
    for l in protein:
        f.write(str(l))

2) Running Rush Modules

You can view which modules are available, alongside their documentation, in the API Documentation.

2.0) Prep the protein

First we will run the protein preparation routine (using pdbfixer and pdb2pqr internally) to prepare the protein for a molecular dynamics simulation.

# we can check the arguments and outputs for prepare_protein with help()
help(client.prepare_protein)

Help on function prepare_protein in module rush.provider:

prepare_protein(*args: *tuple[RushObject[bytes], Optional[float], Optional[EnumValue]], target: 'Target | None' = None, resources: 'Resources | None' = None, tags: 'list[str] | None' = None, restore: 'bool | None' = None) -> tuple[RushObject[list[Record]], RushObject[bytes]]
    Prepare a PDB for downstream tasks: protonate, fill missing atoms, etc.

    Module version:
    `github:talo/prepare_protein/fbeca1ad893cd763b00dc275c43806c0edce03de#prepare_protein_tengu`

    QDX Type Description:

        input_pdb: Object[@$PDB];
        ph: f32?;
        naming_scheme: NamingScheme[Amber | Charmm]?
        ->
        output_qdxf: Object[[Conformer]];
        output_pdb: Object[@$PDB]


    :param input_pdb: An input protein as a file; one PDB file
    :param ph: The ph for determining protonation states; 0-14
    :param naming_scheme: \
                    The force field naming scheme to use; \
                    options are "amber" or "charmm"; \
                    None produces RCSB/IUPAC standard naming\

    :return output_qdxf: An output protein a vec: one qdxf per model in pdb
    :return output_pdb: An output protein as a file: one PDB file

# Here we run the function, it will return a Provider.Arg which you can use to
# fetch the results
# We set restore = True so that we can restore a previous run to the same path
# with the same tags
prepared_protein_qdxf, prepared_protein_pdb = client.prepare_protein(
    PROTEIN_PDB_PATH, None, None
)
# This initially only has the id of your result; we will show how to fetch the
# actual value later
prepared_protein_qdxf

2024-04-08 17:13:29,649 - rush - INFO - Trying to restore job with tags: ['qdx', 'rush-py-quickstart', '1B39'] and path: github:talo/prepare_protein/fbeca1ad893cd763b00dc275c43806c0edce03de#prepare_protein_tengu

Arg(id=37deb248-97fe-443d-b243-36ba172ca7be, value=None)

2.1) Run statuses

This will show the status of all of your runs. You can also view run statuses on the Rush UI.

client.status()

{'8e8357a0-3c37-4c23-bf98-567db98d74df': (<ModuleInstanceStatus.RESOLVING: 'RESOLVING'>,
  'prepare_protein',
  1)}

2.2) Run Values

This will return the “value” of the output from the function—for files you will recieve a url that you can download, otherwise you will recieve them as python types:

protein_qdxf_info = prepared_protein_qdxf.get()
protein_qdxf_info

2024-04-08 17:13:29,921 - rush - INFO - Argument 37deb248-97fe-443d-b243-36ba172ca7be is now ModuleInstanceStatus.RESOLVING
2024-04-08 17:13:36,501 - rush - INFO - Argument 37deb248-97fe-443d-b243-36ba172ca7be is now ModuleInstanceStatus.ADMITTED
2024-04-08 17:13:40,894 - rush - INFO - Argument 37deb248-97fe-443d-b243-36ba172ca7be is now ModuleInstanceStatus.DISPATCHED
2024-04-08 17:13:43,225 - rush - INFO - Argument 37deb248-97fe-443d-b243-36ba172ca7be is now ModuleInstanceStatus.RUNNING
2024-04-08 17:13:55,523 - rush - INFO - Argument 37deb248-97fe-443d-b243-36ba172ca7be is now ModuleInstanceStatus.AWAITING_UPLOAD

'https://storage.googleapis.com/qdx-store/4a4271de-5e14-4756-b115-9c034d7ab294?x-goog-signature=202cb9f86a4351fc30780d26713b4478012548544ff30ba6700f349ad1fcc63137dfb1d2db92fc9d39bc51ba9881c44b27e89e0781aec23d034ec61c4efe2c806b8d784d124863a37d16ce0df02880e272aa0d0dc8f6be23f7214f207183337eb80c9dd0d70757e2761d8f366eea997d066761fadd9ed33483aa8ec98e7fb00e62a78501fdbc2e1e3b9eb4f9e2374a68972937bcae562d114a7e705ca645cd7eb80e5df835c9e462aa34601c8e14691ad6bcbfafef751a7562d8ea3640e77e79f2bd2f6f3b2533df462e245fc4991729f44bee6aa7d9de31cb8a14615b37ba7950c40af54f5a9f146a1a1fb91b2a9f6692d1bca60a8fb2cecd98c2b33fdbd558&x-goog-algorithm=GOOG4-RSA-SHA256&x-goog-credential=qdx-store-user%40humming-bird-321603.iam.gserviceaccount.com%2F20240408%2Fasia-southeast1%2Fstorage%2Fgoog4_request&x-goog-date=20240408T091417Z&x-goog-expires=3600&x-goog-signedheaders=host'

2.3) Downloads

We provide a utility to download files into your workspace, you can either provide a filename, which will be saved in workspace/objects/[filename], or you can provide your own filepath which the client will use as-is:

protein_qdxf_file = prepared_protein_qdxf.download(overwrite=True)

# qdxf files can be loaded as json
with open(protein_qdxf_file) as f:
    protein_qdxf_data = json.load(f)[0]
protein_qdxf_data["amino_acid_seq"][:10]

['MET', 'GLU', 'ASN', 'PHE', 'GLN', 'LYS', 'VAL', 'GLU', 'LYS', 'ILE']

prepared_protein_pdb.download(filename="01_prepared_protein.pdb", overwrite=True)

PosixPath('/home/machineer/qdx/rush-py-quickstart/objects/01_prepared_protein.pdb')

# we can read our prepared protein pdb like this
with open(client.workspace / "objects" / "01_prepared_protein.pdb", "r") as f:
    print(f.readline(), "...")

REMARK   1 CREATED WITH OPENMM 8.0, 2024-04-08
 ...