SilkRoute

SilkRoute is a Python library and command-line tool for reproducible biological data retrieval.

It covers three main workflows:

• Direct API access through 20 database interfaces (UniProt, AlphaFold, ChEMBL, KEGG, and others), with config-driven parsing, on-disk caching, and export to CSV, JSON, XML, or Parquet
• Cross-database enrichment that attaches records from other databases to a result set
• Workflows described in YAML, where every run writes its own metadata and a machine-readable summary

Installation

SilkRoute supports Python 3.11 through 3.14.

pip install silkroute

Install optional extras as needed:

• gui for the NiceGUI descriptor editor
• dev for linting and type checking
• tests for the test suite

pip install 'silkroute[gui]'
pip install 'silkroute[dev,tests]'

Only the credentialed APIs and the experimental sequence search need setup beyond the install.

Quick start

Search UniProt and attach an AlphaFold cross-reference:

silkroute search uniprot by-query \
  --query "antimicrobial AND reviewed:true" \
  --fields accession,protein_name,gene_primary,sequence \
  --crossref-fields alphafold \
  --output-dir results/antimicrobial

Call a single endpoint directly:

silkroute fetch alphafold prediction P12345 -o out.csv

Run a workflow from a descriptor:

silkroute workflow validate-config examples/workflows/protein_query_first_minimal.yml
silkroute workflow run --config examples/workflows/protein_query_first_minimal.yml

Supported databases

Command-line interface

Four namespaces. Run silkroute --help or silkroute <namespace> --help for the full list.

Every command exports csv, json, xml, or parquet. fetch takes -f/--format and infers the format from the output extension when you omit it, while search (-ef) and workflow (-e) take --export-format and default to csv. fetch -o FILE also writes a FILE.provenance.json sidecar (the shared provenance/1 format, with the fetch metadata under extensions) unless you pass --no-metadata. search chemical writes one <db>_results.csv.provenance.json per database.

Exit codes follow the shared ecosystem CLI contract: 0 success, 2 invalid config or usage, 3 runtime failure (failed run, API or I/O error), 4 unexpected internal error (with the traceback on stderr).

silkroute search uniprot by-query \
  --query "(length:[50 TO 51]) AND antimicrobial AND reviewed:true" \
  --fields accession,protein_name,gene_primary,sequence,ec \
  --crossref-fields alphafold,pdb \
  --output-dir search_query_test

silkroute search uniprot by-ids \
  --input unknown_ids.csv \
  --column accession \
  --output-dir search_ids_test \
  --crossref-fields alphafold

silkroute search uniprot by-sequences \
  --database uniprotkb_reviewed \
  --seq-column sequence \
  --min-identity 100.0 \
  --input unknown_sequences.csv \
  --output-dir search_sequences_test

Workflows

Workflows run data acquisition with retries, multi-threaded API calls, optional enrichment, and machine-readable run records (metadata.json plus run_summary.yml). You drive them with CLI flags, a YAML descriptor, or both.

Compound workflows accept ChEMBL, PubChem, and ChEBI source-prefixed queries. PubChem and ChEBI are reachable only through those prefixes.

silkroute workflow run \
  -o workflow_test \
  -q "temperature:99=temp_99,temperature:98=temp_98" \
  --modality protein \
  --mode query_composition \
  --debug

Both labeled queries run, and each exported record carries its label (temp_99, temp_98) in the combined result file. The labeling syntax:

• query=label or query|label, where the last = or | is the delimiter
• Commas separate multiple pairs: query1=label1,query2=label2
• An internal = stays part of the query: chembl.molecule:name__iexact=Imatinib=imatinib

Options

A run needs -o/--output, -m/--modality, -d/--mode, and -q/--query unless a descriptor supplies them.

Outputs

The summary status is success, completed_with_errors (outputs exist but the metadata records errors), or failed (execution failed, or a primary fetch error left no real output); failed exits with code 3.

Primary result tables (uniprot_results, chembl_results, pubchem_results, chebi_results) start with a string id column copied from the table's native key (accession, activity_id / assay_chembl_id / molecule_chembl_id / target_chembl_id, pubchem_cid, chebi_id). harmonization.id_column only renames it. Ids must be unique and non-empty, otherwise the run exits with code 3. Enrichment and graph tables get no id column.

Scenario cheat sheet

YAML descriptors

Descriptors make a run reproducible. They declare schema_version: "workflow-v1" plus dataset, query, resources, execution, harmonization, export, and reporting sections. Canonical examples live in examples/workflows/, and docs/workflow_yaml.md documents the full schema, including every field, forbidden key, and limitation.

# CLI flags override YAML
silkroute workflow run --config examples/workflows/protein_query_first_minimal.yml -o result_override

silkroute workflow validate-config CONFIG [--json] reports every error at once, each with a dotted path, and exits 2 (0 when valid). workflow run rejects the same problems with exit code 2.

dataset.modality: Unsupported dataset.modality 'rna'. Supported modalities are: protein, compound, interaction.
export.format: Unsupported export format 'xlsx'. Supported formats are: csv, json, xml, parquet.

With --json the same problems go to stdout as [{"path": "export.format", "message": "..."}].

Validate a config from Python (offline; returns an empty list when valid, never raises for config problems):

from silkroute import ConfigIssue, validate_config

issues = validate_config("examples/workflows/protein_query_first_minimal.yml")  # or a mapping
for issue in issues:
    print(issue.path, issue.message)

Not every field executes. dataset.modality, dataset.mode, query.value, selected query and execution options, and the export options drive the run. query.builder, query.composition, query.description, and query.filtering_strategy are descriptive metadata: they are preserved in metadata.json and run_summary.yml but never executed. If query.composition is present it must match query.value.

execution.chembl_pages_to_fetch: -1 (the default) fetches all pages and a positive value caps them. limit is records per page, not a total or a page count.

IC50 units accept nM, uM, mM, and pM, and µM/μM normalize to uM. SilkRoute never converts values between units.

Credentials never go in YAML. They come from environment variables or .env only.

schema_version: "workflow-v1"

dataset:
  name: antimicrobial_reviewed_proteins
  description: Reviewed UniProt protein records retrieved with an antimicrobial query.
  modality: protein
  mode: query_first
  primary_data_source: uniprot

query:
  value: "antimicrobial AND reviewed:true"
  description: Retrieve reviewed UniProt protein entries matching an antimicrobial query.

execution:
  enrich: false
  max_workers: 5
  total_retries: 3

harmonization:
  id_column: "_id"

export:
  output_dir: "results/protein_antimicrobial_reviewed"
  format: csv
  include_metadata: true
  include_summary: true

Tools that generate descriptors can read the schema definition programmatically:

from silkroute.core.workflow.schema import get_workflow_v1_schema_definition

GUI (optional)

A web-based form (created with NiceGUI) that makes preparing workflow descriptors easier.

pip install 'silkroute[gui]'
silkroute-gui

It serves http://localhost:8080 and opens a browser tab. Use --host, --port, and --no-browser to change that.

The form offers two query modes. Manual writes query.value directly, and the Advanced builder offers UniProt, ChEMBL, PubChem, and ChEBI builders filtered by the selected modality and interaction type. Either way query.value is the executable output.

Existing files round-trip. Loading a workflow-v1 YAML populates the supported fields, and query-builder-v1 metadata restores editable builder rows. Metadata the GUI cannot read falls back to manual or read-only handling and leaves query.value intact. GUI labels map to exact schema values, and query.fields stores UniProt field IDs rather than the visible labels.

docs/workflow_yaml.md carries the full GUI reference: per-source builder semantics, connector versus match mode, return-field selection, harmonization controls, and smoke tests.

Python API

Fetch a batch through a database interface:

import polars as pl
from silkroute import UniprotInterface

df = pl.DataFrame(
    {
        "id": [1, 2, 3],
        "accession": ["A1L3X0", "A0JNC4", "A2RUC4"],
    }
)

uniprot = UniprotInterface()
results, _ = uniprot.download_batch(
    df,
    id_column="accession",
    auto_db=False,
    from_db="UniProtKB_AC-ID",
    to_db="UniProtKB",
    batch_size=100,
)
results_df, _ = uniprot.parse(results, None)
print(results_df)

Then enrich those results with cross-referenced records from other databases:

from silkroute.core.crossref_enricher import CrossRefEnricher, EndpointSpec

specs = [
    EndpointSpec(database="alphafold", endpoint="prediction"),
    EndpointSpec(database="pdb", endpoint="entry"),
]

enricher = CrossRefEnricher(specs)
concat_df, _ = enricher.enrich(results_df, concat_results=True)

Configuration

Credentials

Credentials resolve in this order:

1. Explicit CLI arguments or constructor parameters
2. Environment variables, including values loaded from a .env file

SilkRoute looks for .env at SILKROUTE_ENV_FILE, then in the working directory, then at ~/.config/silkroute/.env or a per-interface directory such as ~/.config/silkroute/biogrid/.env. silkroute/config/.env.example is the template.

Cache and config locations

By default SilkRoute keeps its cache and its per-API config in the platform directories:

• Linux: ~/.cache/silkroute and ~/.config/silkroute
• macOS: ~/Library/Caches/silkroute and ~/Library/Application Support/silkroute
• Windows: %LOCALAPPDATA%\silkroute\Cache and %LOCALAPPDATA%\silkroute

Two environment variables override them:

• SILKROUTE_CACHE_DIR for the cache root, which also relocates the BLAST database directory (<cache root>/blast_db)
• SILKROUTE_CONFIG_DIR for the config root

Use silkroute cache list and silkroute cache clear to inspect or purge the cache.

Packaged configuration

Per-API parsing config ships inside the package under silkroute/config/<api>/ and loads automatically. The fields.yml files map API responses to output columns, keyed by endpoint name as output_column: api.response.path:

# silkroute/config/alphafold/fields.yml
prediction:
  entry: entryId
  gene: gene
  tax_id: taxId
  organism: organismScientificName

These files are library internals: editing them breaks parsing. Each call picks its own download location through the interface output_dir argument.

Learn more

• docs/workflow_yaml.md for the full workflow-v1 schema and GUI reference
• examples/ for runnable scripts, notebooks, and descriptor examples
• DEVELOPMENT.md for local setup, tests, and how the API fixtures are regenerated

Roadmap

• Automatic caching and offline mode
• Integration with external ML workflows
• Improve the API example notebooks

Contributing

Issues and pull requests should keep the docs aligned with what is implemented and tested. DEVELOPMENT.md covers development setup and testing conventions.

Citation

If you use SilkRoute in your research, please cite:

Fernández Villegas, D.; García - Vinuesa, J. A.; Alvarez Saravia, D.; Medina-Ortiz, D. SilkRoute: Reproducible Biological Data Retrieval. Version 0.1.1, 2026. Zenodo. https://doi.org/10.5281/zenodo.21930739

@software{silkroute2026,
  author    = {Fernández Villegas, Diego and García - Vinuesa, Julián Alfonso and Alvarez Saravia, Diego and Medina-Ortiz, David},
  title     = {{SilkRoute}: Reproducible Biological Data Retrieval},
  year      = {2026},
  version   = {0.1.1},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21930739},
  url       = {https://doi.org/10.5281/zenodo.21930739}
}

To cite this exact release, use the version DOI 10.5281/zenodo.21930740. Machine-readable metadata lives in CITATION.cff.

License

MIT. See LICENSE.

Acknowledgements

Built with polars, Biopython, Typer, niquests, and zeep. Some modules are based on the UniProt API, UniProt ID Mapping, and the AlphaFold Database.

Developed by KREN AI Lab at Universidad de Magallanes, Chile.