PanGBank API

This repository contains the API used to manage the PanGBank database, which stores collections of pangenomes built with PPanGGOLiN.

The API is built with FastAPI and uses SQLModel as its ORM. It provides a RESTful interface for querying and exploring pangenome collections. Alongside the API, a command-line tool pangbank_db is included to manage the database.

Installation

PanGBank-api is organized into two main components:

• Core package: Database models, CRUD operations, and CLI tools (pangbank_db)
• API server: FastAPI-based REST API (optional)

Option 1: Install Core Package Only

For database management and CLI tools without the API server:

pip install pangbank-api

This installs:

• Database models (pangbank_api.models)
• Database utilities (pangbank_api.database, pangbank_api.config)
• CRUD operations (pangbank_api.crud)
• CLI tool pangbank_db for database management

Option 2: Install with FastAPI (Full API Server)

For running the REST API server:

pip install pangbank-api[fastapi]

This additionally installs:

• FastAPI framework
• API routers (pangbank_api.routers)
• API server (pangbank_api.main)

Option 3: Install the Python SDK Client

pip install pangbank-api[sdk]

This additionally installs:

• httpx
• The pangbank_api.sdk client (PanGBankClient, AsyncPanGBankClient)

Local Development Setup

1. Clone the repository:

   git clone https://github.com/labgem/PanGBank-api.git
   cd PanGBank-api
   

2. Create a virtual environment and install with FastAPI:

   python -m venv venv
   source venv/bin/activate
   pip install .[fastapi]
   

3. Run the API in development mode:

   export PANGBANK_DB_PATH="<path/to/database.sqlite>"
   export PANGBANK_DATA_DIR="<path/to/pangenome_directory>"
   fastapi dev pangbank_api/main.py
   

PANGBANK_DB_PATH is the path to your SQLite database file. PANGBANK_DATA_DIR is the root directory containing your pangenome data and mash files.

Python SDK Client

pangbank_api.sdk wraps every route on the API behind plain python. Both a sync and an async client are available:

from pangbank_api.sdk import PanGBankClient

with PanGBankClient() as client:
    pangenomes = client.pangenomes.list(collection_name="GTDB_refseq", limit=10)
    pangenome = client.pangenomes.get(pangenomes[0].id)
    client.pangenomes.download_file(pangenome.id, dest="pangenome.h5")

import asyncio

from pangbank_api.sdk import AsyncPanGBankClient


async def main() -> None:
    async with AsyncPanGBankClient() as client:
        genomes = await client.genomes.list(taxon_name="Bacteria", limit=10)
        print([genome.name for genome in genomes])


asyncio.run(main())

See pangbank_api/sdk/README.md for complete SDK documentation.

Managing the Database with pangbank_db

All CLI commands require the PANGBANK_DB_PATH environment variable to be set.

export PANGBANK_DB_PATH="<path/to/database.sqlite>"

Add a Collection Release

To add a new collection of pangenomes in the database, use:

pangbank_db add-collection-release <collection_release.json>

{
  "collection": {
    "name": "GTDB_all_sampled",
    "description": "GTDB all is a collection of pangenomes made of GTDB species that have at least 15 genomes."
  },
  "release": {
    "version": "1.0.0",
    "ppanggolin_version": "2.2.4",
    "pangbank_wf_version": "0.0.2",
    "pangenomes_directory": "GTDB_refseq/release_v1.0.0/data/pangenomes/", // relative to PANGBANK_DATA_DIR
    "release_note": "",
    "date": "2025-07-10",
    "mash_sketch": "GTDB_refseq/release_v1.0.0/data/mash_sketch/families_persistent_all.msh", // relative to PANGBANK_DATA_DIR
    "mash_version": "2.3"
  },
  "taxonomy": {
    "name": "GTDB",
    "version": "10-RS226",
    "ranks": "Domain; Phylum; Class; Order; Family; Genus; Species",
    "file": "/absolute/path/to/taxonomy.tsv"
  },
  "genome_sources": [
    {
      "name": "RefSeq",
      "file": "/absolute/path/to/genomes.tsv",
      "version": "",
      "description": "",
      "source": "",
      "url": ""
    }
  ],
  "genome_metadata": {
    "file": "/absolute/path/to/genome_metadata.tsv"
  },
  "genome_statuses": [
    {
      "status_type": "representative",
      "origin": "GTDB",
      "file": "/absolute/path/to/gtdb_representatives.txt"
    },
    {
      "status_type": "reference",
      "origin": "NCBI_RefSeq",
      "file": "/absolute/path/to/ncbi_references.txt"
    }
  ]
}

List Existing Collections

pangbank_db list-collections

Delete a Collection Release

pangbank_db delete-collection <collection_name> --release-version <version>

Add Genome Statuses to an Existing Release

Add genome status information (representative, reference, type strain, etc.) to an existing collection release without re-importing the entire collection:

pangbank_db add-genome-statuses \
  --collection-name <collection_name> \
  --release-version <release_version> \
  --status-type <status_type> \
  --origin <origin> \
  --file <file>

Example:

pangbank_db add-genome-statuses \
  --collection-name "GTDB_all_sampled" \
  --release-version "1.0.0" \
  --status-type "representative" \
  --origin "GTDB" \
  --file /path/to/gtdb_representatives.txt

This command is useful for:

• Adding genome statuses to releases that were imported without them
• Updating status information when new representative/reference genomes are announced
• Adding multiple status types incrementally (e.g., first representatives, then type strains)

The file should contain one genome name per line. Duplicate statuses are automatically skipped.

Add Quality Metrics to Existing Genomes

Add or update genome quality metrics (CheckM completeness, contamination, genome size, etc.) for genomes already in the database:

pangbank_db add-quality-metrics <genome_metadata.tsv>

# Force overwrite existing values (with warnings)
pangbank_db add-quality-metrics <genome_metadata.tsv> --force

Example:

# Add new quality metrics (fails if trying to change existing values)
pangbank_db add-quality-metrics /path/to/gtdb_genome_metadata.tsv

# Intentionally overwrite existing metrics (logs warnings)
pangbank_db add-quality-metrics /path/to/updated_metrics.tsv --force

The TSV file should have:

• A genomes column with genome names
• Quality metric columns matching the Genome model fields (e.g., checkm2_completeness, checkm2_contamination, genome_size, gc_percentage)

Important Notes:

• Only columns that match optional Genome fields will be imported; unknown columns are automatically filtered out
• Quality metrics are immutable by default - attempting to change existing values raises an error
• Use --force flag to intentionally overwrite existing values (warnings will be logged for each change)
• If a genome already has a value for a field:
  • Identical values are skipped (idempotent operation)
  • Different values raise an error unless --force is used
• Only fields with None (no existing value) are updated without restriction

This command is useful for:

• Adding quality metrics after initial data import
• Importing metrics for newly added genomes
• Safely re-running imports without data corruption risk

Example TSV format:

genomes	checkm2_completeness	checkm2_contamination	genome_size	gc_percentage
GenomeA	98.5	0.2	5000000	45.5
GenomeB	95.0	1.5	4500000	42.0

Database Migrations with Alembic

We use Alembic to manage schema changes in the PanGBank database.

Create a new migration

Generate a migration after updating your SQLModel models (e.g., adding or changing columns):

alembic revision --autogenerate -m "Describe your change here"

Apply migrations to the database

This applies all pending migrations:

alembic upgrade head

Roll back the last migration (use with caution)

If something went wrong, you can revert the last migration:

alembic downgrade -1

Or go back to the base (empty schema):

alembic downgrade base

Contributing

1. Fork the repository.
2. Create a feature branch (git checkout -b feature-name).
3. Commit your changes (git commit -m 'Add new feature').
4. Push to the branch (git push origin feature-name).
5. Open a pull request.

Contact

For any inquiries or issues, open an issue on the GitHub repository.