abcupload
ABCOMICS Genomic Data Upload Tool
A command-line tool to upload sequencing and genomic data files to the ABCOMICS platform. It handles the full upload process automatically: requesting a signed URL from the API, then streaming each file directly to secure cloud storage with a live progress bar.
Installation
pip install abcupload
Requires Python 3.8 or later. Dependencies (requests, rich) are installed
automatically.
Quick start — manifest mode (recommended)
After submitting your runs and analyses metadata in the ABCOMICS dashboard, download the upload manifest (a TSV file) for your project. The manifest maps each local filename to its ID-based destination name in ABCOMICS storage.
# 1. Get your personal API key from the ABCOMICS dashboard:
# Dashboard → Settings → abcupload API Key → Copy
# 2. Set it once in your terminal session
export ABCOMICS_API_KEY="your_key_here"
# 3. Upload all files listed in the manifest
abcupload -d /path/to/your/files/ -m PRJAB00001_upload_manifest.tsv
# Overwrite without asking if a file already exists
abcupload -d /path/to/your/files/ -m PRJAB00001_upload_manifest.tsv -o
Where is my API key?
Log into the ABCOMICS dashboard → Settings → abcupload API Key.
Each user has a unique personal key — never share it.
The manifest is a tab-separated file with four columns:
# entity_id original_filename new_gcs_filename project_id
RUNAB00001 sample_R1.fastq.gz RUNAB00001_1.fastq.gz PRJAB00001
RUNAB00001 sample_R2.fastq.gz RUNAB00001_2.fastq.gz PRJAB00001
RUNAB00002 reads.bam RUNAB00002.bam PRJAB00001
ANZAB00001 result.tsv ANZAB00001.tsv PRJAB00001
abcupload reads the manifest to rename files to their ABCOMICS IDs on the
way up — your local files are never renamed.
Legacy mode
# Upload a single file
abcupload -u abc-000001 -p PRJAB00001 -d sample_R1.fastq.gz
# Upload all FASTQ files in the current directory
abcupload -u abc-000001 -p PRJAB00001 -d '*.fastq.gz'
# Upload an entire directory (scans for all accepted formats)
abcupload -u abc-000001 -p PRJAB00001 -d /data/project/
# Test mode — files are deleted after 24 hours
abcupload -u abc-000001 -p PRJAB00001 -d '*.fastq.gz' -t
Full usage
manifest mode (recommended):
abcupload -d DIRECTORY -m MANIFEST_TSV [options]
-d DIRECTORY Directory containing the files to upload
-m MANIFEST_TSV TSV manifest downloaded from the ABCOMICS dashboard
legacy mode:
abcupload -u USERNAME -p PROJECT_ID -d FILES [options]
-u USERNAME ABCOMICS username (e.g. abc-000001)
-p PROJECT_ID Project ID (e.g. PRJAB00001)
-d FILES Single file, comma list, glob pattern, or directory
options:
-k API_KEY API key (alternative to ABCOMICS_API_KEY env var)
-t Test mode — files deleted after 24 h
-o Overwrite existing files without asking
-h, --help Show this help and exit
-V, --version Show version and exit
Environment variable
# Add to ~/.bashrc or ~/.zshrc for persistence
export ABCOMICS_API_KEY=your_key_here
Accepted file formats
| Format | Extensions |
|---|---|
| FASTQ | .fastq, .fq, .fastq.gz, .fq.gz, .fastq.bz2, .fq.bz2 |
| FASTA | .fasta, .fa, .fna, .faa, .ffn, .frn, .fasta.gz, .fa.gz, .fna.gz, .faa.gz |
| BAM / SAM / CRAM | .bam, .sam, .cram |
| Index files | .bai, .crai, .csi, .tbi |
| VCF / BCF | .vcf, .vcf.gz, .bcf, .bcf.gz |
| Annotation | .gff, .gff3, .gtf, .bed (and .gz variants) |
| Tabular | .csv, .tsv |
Maximum file size: 500 GB.
Changelog
1.2.0
- API keys are now per-user: each user has a unique personal key available in the ABCOMICS dashboard under Settings → abcupload API Key.
- No changes required to the CLI — just update your
ABCOMICS_API_KEYvalue.
1.1.0
- New manifest mode (
-d DIR -m manifest.tsv): reads an ABCOMICS upload manifest to rename files to their ID-based GCS names on upload without touching local files. - Files now upload directly to
public/{project_id}/in GCS; no private→public move step needed. -u(username) and-p(project_id) are no longer required in manifest mode; all info comes from the TSV.- Legacy mode (
-u USER -p PRJ -d files) still fully supported.
1.0.1
- Initial public release.
Author
Khadim Gueye — African Bioinformatics Center (ABCOMICS)
Contact: contact@abcomics.org
License
Metadata
Release files for abcupload 1.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| abcupload-1.2.1.tar.gz | 11.5 kB | Without provenance | Details for abcupload-1.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| abcupload-1.2.1-py3-none-any.whl | Python 3 | none | any | Details for abcupload-1.2.1-py3-none-any.whl |
Total release size: 22.0 kB
Release files / abcupload-1.2.1.tar.gz
| Download URL | abcupload-1.2.1.tar.gz |
|---|---|
| Size | 11.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
35e8ca329996a5dd6f3a3427460b772dce3fdf9d30cea07c9f187ec167b034db
|
|
BLAKE2b-256 checksum How to use checksums |
c36a1548056dc78c5fbddafbea193943ccfe16cc90f11e60cf43fa4a52a724c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Release files / abcupload-1.2.1-py3-none-any.whl
| Download URL | abcupload-1.2.1-py3-none-any.whl |
|---|---|
| Size | 10.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6b6a5286eb567b71d1a829688bbc8bcf644fa99429c6c349b0e2db821136d5e4
|
|
BLAKE2b-256 checksum How to use checksums |
6fbae314ff6fe7b3ac8ef0fa7abc77d8a176012a3cfc4161a57c9fe40eaf52b5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.11
|
Security for abcupload 1.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.