xenosite.predict

Python 3.11+ RDKit + ONNX predictors for XenoSite. Dist name xenosite-predict; import xenosite.predict (PEP 420 namespace). Checkout as a sibling of xenosite-api and xenosite-legacy; origin is github.com/swamidasslab/xenosite-predict.

Publish sdist only (no wheel): ONNX weights stay local (weights/, gitignored). OpenBabel comes from PyPI (uv add openbabel, currently 3.2.x wheels). make test is Docker-free. Feature tests compare the installed OpenBabel to the committed tests/fixtures/ob_dumps.json.gz (Git LFS). make test-live skips if Docker, the legacy image, or ONNX files are missing. Do not commit model weights, pickles, or extracted libridass/ trees.

This package is not wired into xenosite-api yet.

User API

from xenosite.predict import predict, predict_many, apredict, apredict_many, list_models

mol = predict("O=C(C)Oc1ccccc1C(=O)O", model="epoxidation")  # version "1"
mol = predict("O=C(C)Oc1ccccc1C(=O)O", models=["epoxidation", ("ugt", "1")])
mol = predict("O=C(C)Oc1ccccc1C(=O)O", models=[("epoxidation", "0")])  # v0 / legacy params
mol = predict(mol, models=["quinone"])  # append
list_models()  # what this process can actually run (backend-aware)

# Many molecules (process pool for ONNX; sync API)
mols = predict_many(smiles_list, model="ugt", workers=4)

# Async (event-loop friendly; same workers under the hood)
mol = await apredict(smi, model="ugt")
mols = await apredict_many(smiles_list, model="ugt", workers=4)
mols = await asyncio.gather(*[apredict(s, model="ugt") for s in smiles_list])

• One molecule at a time for predict / apredict (no multi-mol batch inside a single call).
• Many molecules: predict_many / apredict_many run each input independently in parallel.
• Parse once when several models run on one molecule. Canonical SMILES is non-isomeric (isomericSmiles=False). Atom/bond indices and score arrays use canonical SMILES atom order, not the input order.
• detailed=True adds atom/bond properties (z, chrg, impHs, cipRank, bond order) and atoms.reordered (original input atom indices in canonical SMILES order).
• rdkit=True keeps the in-process RDKit mols already built during parse and forest enumeration on Molecule.rdkit and Metabolite.rdkit. No extra parse. Omitted from JSON.
• models= is a name (default scoring version "1") or (name, version) pairs. "1" uses updated scoring parameters (HTTP /v1). "0" uses legacy parameters that match golden fixtures and HTTP /v0. Do not pass one version string for a whole list.
• Indices are 0-based RDKit atom/bond indices. Scores are floats (atol=1e-4 in tests).
• Name lookup is omitted. Pass SMILES, not drug names.
• Import does not open ONNX, HTTP, or OpenBabel. OpenBabel stays a required install for v0/v1; bindings load only when a v0/v1 ONNX predictor runs descriptors (not on list_models or HTTP). Callers never import openbabel / pybel.
• First predict() downloads ONNX weights when XENOSITE_ONNX_URL is set and none are cached (an INFO line reports when they are found or downloaded). No separate download_weights() call is required.
• Workers: ONNX batch/async paths use a process pool (descriptor generation is CPU-bound; threads do not help). Defaults: workers=min(cpu_count, 32) and XENOSITE_ORT_INTRA_OP=1 (many processes × one ORT thread). Override with workers= / XENOSITE_WORKERS. Server deployments (gunicorn / Lambda) should keep XENOSITE_WORKERS=1 so request-level workers provide parallelism without a nested pool. ORT profiling is off (no :mem:.sess dumps). Set XENOSITE_ORT_PROFILE to a real file path to write an ORT profile.
• Scoring versions: predict(..., models=[("epoxidation", "0")]) is v0/legacy parameters; omit the version or pass "1" for the updated mapping. Same ONNX weights. Score impact summary: docs/legacy-vs-principled.md. Walkthrough: tests/v0_legacy/test_legacy_vs_principled_guide.py. _parameter overlays individual flags for tests.

predict_many / apredict / apredict_many

workers defaults to min(cpu_count, 32) (XENOSITE_WORKERS overrides); ORT defaults to 1 intra-op thread per process. New models reuse the existing predict / runner path — no per-model async code.

predict(inp, model=..., models=..., backend=..., backends=..., env=..., detailed=..., canonicalize=..., rdkit=...)

Backends always receive a canonical SMILES molecule with detailed topology (identical calls → better cache dedupe). canonicalize / detailed only change what is returned.

Return type (Molecule)

Ported from xenosite-api types.py: smiles, atoms, bonds, results. Result variants: MolBondResult, MolAtomResult, MolAtomPairResult, AtomResult, BondResult, AtomBondResult. Each result has model and model_version ("0" or "1", the scoring generation). Metabolite.smiles is canonical SMILES except for conjugation adducts (ugt, reactivity.*), which are CXSMILES (* with an atomLabel such as GlcA / GSH). The SMILES token before | is valid on its own and depicts as * if the CX block is ignored.

list_models()

Returns dicts {name, version, available, backend, reason, heads, two_stage, pipeline} for this process, not a fictional union of every backend.

ONNX weights

ONNX graphs are not in the sdist. Set XENOSITE_ONNX_URL to an https tarball or a local .tgz path (the URL is not stored in this repo). The first predict() (or list_models()) downloads into $XDG_CACHE_HOME/xenosite/onnx/v0 (or ~/.cache/xenosite/onnx/v0, or XENOSITE_MODELS_WEIGHTS if set) and prints an INFO line when weights are found or downloaded. Download logs and errors never echo the URL (so a private weight location does not leak via stderr or tracebacks). Tests that pass env={} never fetch. python -m xenosite.predict download and make download-onnx are optional pre-fetch helpers.

Errors

InvalidMolecule, UnknownModel, BackendNotConfigured, BackendRequestError, WeightsNotFound, WeightsDownloadError, ModelNotAvailable, OpenBabelNotAvailable.

Backends

Picker (explicit env wins; first match):

1. XENOSITE_BACKEND is an http:// / https:// URL → HTTP against that deployed xenosite-api. Optional XENOSITE_API_KEY as Bearer (also applied when backend= is a URL).
2. Else XENOSITE_MODELS_WEIGHTS → local ONNX directory.
3. Else auto-detect ./weights/onnx/v0 (or a flat ./weights/onnx tree) → local ONNX.
4. Else user cache ($XDG_CACHE_HOME/xenosite/onnx/v0) if *.onnx exist.
5. Else, when XENOSITE_ONNX_URL is set in the process env, download that archive into the cache (INFO on found/download).
6. Else raise BackendNotConfigured.

bioactivation is not available on HTTP (no route on xenosite-api). _parameter overlays are ONNX-only (error on HTTP/legacy).

Benchmark ONNX vs HTTP predict_many:

uv run python tools/bench_http_vs_onnx.py --n 32 --model ugt

Live parity compares ONNX vs the legacy test-API, not vs production HTTP. Tests must pass backend= and must not inherit a developer shell (XENOSITE_* are cleared in conftest.py).

Built-in models (version "0")

Makefile (tools are not in the sdist)

make extract-weights          # Docker image or fallback tarball → weights/legacy/
make convert-onnx             # pickle → ONNX; MODEL=epoxidation for one model
make pack-onnx                # weights/xenosite_onnx_v0.tgz (runtime graphs, no _dump)
make extract-onnx             # unpack that tarball into weights/onnx/v0/
make download-onnx            # fetch $XENOSITE_ONNX_URL into weights/onnx/v0/
make test                     # pytest -m "not live"  (no Docker)
make test-live                # pytest -m live; fixture skips if Docker/image missing
make py2-dump-image           # python:2.7-slim + numpy + Debian OpenBabel 2.4
make dump-ob                  # OpenBabel feature dump via that image (no WashU)
make changelog                # preview next CHANGELOG.md section (towncrier)
make changelog-release        # fold changelog.d/ into CHANGELOG.md
make legacy-test-api          # build/run derived test image
make legacy-test-api-down

Convert deps: uv run --group convert. Installed runtime: rdkit, openbabel (PyPI 3.2.x), numpy, onnxruntime, httpx, pydantic. OpenBabel is internal (not part of the public API). No TensorFlow, pandas, or pickle at inference.

The dump image remains the OpenBabel 2.4.1 feature oracle. Host inference uses the PyPI 3.2.x wheel; tests/test_ob_features.py reports 3.x vs 2.4 drift at atol 1e-4 / rtol 0. Do not vendor OpenBabel sources (GPL).

Populate pickles from dockerreg01.accounts.ad.wustl.edu/swamidass/xenosite-legacy:api (needs registry login) or the sibling tarball xenosite-legacy/data/xenosite_legacy_data_trimmed.tgz. make convert-onnx unpickles in a public python:2.7-slim dump image (tools/py2-dump/), not the WashU API image.

The same dump image is the OpenBabel feature oracle: Debian Buster python-openbabel 2.4.1 and python-rdkit from archive.debian.org, running as /usr/bin/python (the image's /usr/local CPython cannot load the multiarch SWIG module). make dump-ob feeds an RDKit molblock so 1-based OB indices align with 0-based RDKit, and dumps BondTD/AtomTD/UGT/Heuristic/Bond_and_LonePair rows from sibling xenosite-legacy/src. It is idempotent: molecule/model pairs already in the suite are skipped, and the JSON is checkpointed after each chunk. The gzipped suite tests/fixtures/ob_dumps.json.gz is committed via Git LFS so dump tests run without Docker; uncompressed JSON stays gitignored. Clone with Git LFS (git lfs pull).

Public parse/canonicalize stays RDKit. Feature graphs call OpenBabel internally (PyPI 3.2.x). tests/test_ob_features.py compares host OpenBabel 3.2 rows to 2.4 dumps at atol 1e-4 / rtol 0. Missing dumps fail. Hypothesis draws random finite matrices for ONNX heads (test_onnx_random_matrix_finite) and live /nn vs ONNX (test_random_vector_nn). The convert dump tests/fixtures/random_vectors.json is the Python-2 regression (ONNX == pickled numpy NN).

Layout

src/xenosite/predict/   # user API (installed)
  v1/                   # principled runners (scoring version "1")
  v0/                   # deprecated wrap of v1 with legacy defaults (warns)
  v2/                   # future RDKit-first / multitask (not implemented)
tools/                  # extract, convert, legacy-test-api (not in the wheel)
weights/                # local only — README + .gitignore committed
tests/                  # unit + @pytest.mark.live
docs/vendored-diffs.md  # NN/feature hashes, MOPAC/SmartCYP gate
docs/legacy-vs-principled.md  # production defaults vs golden legacy modes
docs/release.md               # cut a version: towncrier, tag, PyPI
CHANGELOG.md            # Keep a Changelog (compiled by towncrier)
changelog.d/            # news fragments for the next release

Development

uv sync --group dev
make test

Changelog (towncrier)

User-facing notes live as fragments in changelog.d/, not in git subjects:

make changelog-create TYPE=added NAME=rdkit-mols MSG="Keep RDKit mols when rdkit=True."
make changelog VERSION=0.3.3   # draft; does not write files

How to cut a release (tag vX.Y.Z; CI compiles CHANGELOG.md): docs/release.md. Fragments: changelog.d/README.md.

Publishing to PyPI (trusted publishing)

Releases use GitHub OIDC (.github/workflows/publish.yml). The v* tag is the package version (hatch-vcs). One-time publisher setup and the tag push steps are in docs/release.md.

Do not commit XENOSITE_ONNX_URL, API keys, or weight hostnames. Keep those in local env / deployment secrets only.

Vendored-tree comparison (sibling checkout, not committed):

uv run python tools/compare_vendored.py --root ../xenosite-legacy/src/libridass