Skip to main content

abcupload

ABCOMICS Genomic Data Upload Tool

A command-line tool to upload sequencing and genomic data files to the ABCOMICS platform. It handles the full upload process automatically: requesting a signed URL from the API, then streaming each file directly to secure cloud storage with a live progress bar.


Installation

pip install abcupload

Requires Python 3.8 or later. Dependencies (requests, rich) are installed automatically.


After submitting your runs and analyses metadata in the ABCOMICS dashboard, download the upload manifest (a TSV file) for your project. The manifest maps each local filename to its ID-based destination name in ABCOMICS storage.

# 1. Get your personal API key from the ABCOMICS dashboard:
#    Dashboard → Settings → abcupload API Key → Copy

# 2. Set it once in your terminal session
export ABCOMICS_API_KEY="your_key_here"

# 3. Upload all files listed in the manifest
abcupload -d /path/to/your/files/ -m PRJAB00001_upload_manifest.tsv

# Overwrite without asking if a file already exists
abcupload -d /path/to/your/files/ -m PRJAB00001_upload_manifest.tsv -o

Where is my API key?
Log into the ABCOMICS dashboard → Settings → abcupload API Key.
Each user has a unique personal key — never share it.

The manifest is a tab-separated file with four columns:

# entity_id    original_filename       new_gcs_filename            project_id
RUNAB00001      sample_R1.fastq.gz      RUNAB00001_1.fastq.gz       PRJAB00001
RUNAB00001      sample_R2.fastq.gz      RUNAB00001_2.fastq.gz       PRJAB00001
RUNAB00002      reads.bam               RUNAB00002.bam              PRJAB00001
ANZAB00001      result.tsv              ANZAB00001.tsv              PRJAB00001

abcupload reads the manifest to rename files to their ABCOMICS IDs on the way up — your local files are never renamed.


Legacy mode

# Upload a single file
abcupload -u abc-000001 -p PRJAB00001 -d sample_R1.fastq.gz

# Upload all FASTQ files in the current directory
abcupload -u abc-000001 -p PRJAB00001 -d '*.fastq.gz'

# Upload an entire directory (scans for all accepted formats)
abcupload -u abc-000001 -p PRJAB00001 -d /data/project/

# Test mode — files are deleted after 24 hours
abcupload -u abc-000001 -p PRJAB00001 -d '*.fastq.gz' -t

Full usage

manifest mode (recommended):
  abcupload -d DIRECTORY -m MANIFEST_TSV [options]

  -d DIRECTORY      Directory containing the files to upload
  -m MANIFEST_TSV   TSV manifest downloaded from the ABCOMICS dashboard

legacy mode:
  abcupload -u USERNAME -p PROJECT_ID -d FILES [options]

  -u USERNAME       ABCOMICS username          (e.g. abc-000001)
  -p PROJECT_ID     Project ID                 (e.g. PRJAB00001)
  -d FILES          Single file, comma list, glob pattern, or directory

options:
  -k API_KEY        API key  (alternative to ABCOMICS_API_KEY env var)
  -t                Test mode — files deleted after 24 h
  -o                Overwrite existing files without asking
  -h, --help        Show this help and exit
  -V, --version     Show version and exit

Environment variable

# Add to ~/.bashrc or ~/.zshrc for persistence
export ABCOMICS_API_KEY=your_key_here

Accepted file formats

Format Extensions
FASTQ .fastq, .fq, .fastq.gz, .fq.gz, .fastq.bz2, .fq.bz2
FASTA .fasta, .fa, .fna, .faa, .ffn, .frn, .fasta.gz, .fa.gz, .fna.gz, .faa.gz
BAM / SAM / CRAM .bam, .sam, .cram
Index files .bai, .crai, .csi, .tbi
VCF / BCF .vcf, .vcf.gz, .bcf, .bcf.gz
Annotation .gff, .gff3, .gtf, .bed (and .gz variants)
Tabular .csv, .tsv

Maximum file size: 500 GB.


Changelog

1.2.0

  • API keys are now per-user: each user has a unique personal key available in the ABCOMICS dashboard under Settings → abcupload API Key.
  • No changes required to the CLI — just update your ABCOMICS_API_KEY value.

1.1.0

  • New manifest mode (-d DIR -m manifest.tsv): reads an ABCOMICS upload manifest to rename files to their ID-based GCS names on upload without touching local files.
  • Files now upload directly to public/{project_id}/ in GCS; no private→public move step needed.
  • -u (username) and -p (project_id) are no longer required in manifest mode; all info comes from the TSV.
  • Legacy mode (-u USER -p PRJ -d files) still fully supported.

1.0.1

  • Initial public release.

Author

Khadim Gueye — African Bioinformatics Center (ABCOMICS)
Contact: contact@abcomics.org


License

MIT

Metadata

Release files for abcupload 1.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for abcupload 1.2.1
File Size Uploaded Provenance Actions
abcupload-1.2.1.tar.gz 11.5 kB Without provenance Details for abcupload-1.2.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for abcupload 1.2.1
File Interpreter ABI Platform Actions
abcupload-1.2.1-py3-none-any.whl Python 3 none any Details for abcupload-1.2.1-py3-none-any.whl

Total release size: 22.0 kB

Release files / abcupload-1.2.1.tar.gz

Download URL abcupload-1.2.1.tar.gz
Size 11.5 kB
Tags Source
SHA-256 checksum
How to use checksums
35e8ca329996a5dd6f3a3427460b772dce3fdf9d30cea07c9f187ec167b034db
BLAKE2b-256 checksum
How to use checksums
c36a1548056dc78c5fbddafbea193943ccfe16cc90f11e60cf43fa4a52a724c1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.11

Release files / abcupload-1.2.1-py3-none-any.whl

Download URL abcupload-1.2.1-py3-none-any.whl
Size 10.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6b6a5286eb567b71d1a829688bbc8bcf644fa99429c6c349b0e2db821136d5e4
BLAKE2b-256 checksum
How to use checksums
6fbae314ff6fe7b3ac8ef0fa7abc77d8a176012a3cfc4161a57c9fe40eaf52b5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.11

Security for abcupload 1.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page