Skip to main content

Python utility libraries on genome assembly, annotation and comparative genomics

Project description

JCVI: A Versatile Toolkit for Comparative Genomics Analysis

Latest PyPI version bioconda Github Actions Downloads

Collection of Python libraries to parse bioinformatics files, or perform computation related to assembly, annotation, and comparative genomics.

Authors Haibao Tang (tanghaibao)
Vivek Krishnakumar (vivekkrish)
Adam Taranto (Adamtaranto)
Xingtan Zhang (tangerzhang)
Won Cheol Yim (wyim-pgl)
Email tanghaibao@gmail.com
License BSD

How to cite

[!TIP] JCVI is now published in iMeta!

Tang et al. (2024) JCVI: A Versatile Toolkit for Comparative Genomics Analysis. iMeta

MCSCAN example

ALLMAPS animation

GRABSEEDS example

Contents

Following modules are available as generic Bioinformatics handling methods.

  • algorithms

    • Linear programming solver with SCIP and GLPK.
    • Supermap: find set of non-overlapping anchors in BLAST or NUCMER output.
    • Longest or heaviest increasing subsequence.
    • Matrix operations.
  • apps

    • GenBank entrez accession, Phytozome, Ensembl and SRA downloader.
    • Calculate (non)synonymous substitution rate between gene pairs.
    • Basic phylogenetic tree construction using PHYLIP, PhyML, or RAxML, and viualization.
    • Wrapper for BLAST+, LASTZ, LAST, BWA, BOWTIE2, CLC, CDHIT, CAP3, etc.
  • formats

    Currently supports .ace format (phrap, cap3, etc.), .agp (goldenpath), .bed format, .blast output, .btab format, .coords format (nucmer output), .fasta format, .fastq format, .fpc format, .gff format, obo format (ontology), .psl format (UCSC blat, GMAP, etc.), .posmap format (Celera assembler output), .sam format (read mapping), .contig format (TIGR assembly format), etc.

  • graphics

    • BLAST or synteny dot plot.
    • Histogram using R and ASCII art.
    • Paint regions on set of chromosomes.
    • Macro-synteny and micro-synteny plots.
    • Ribbon plots from whole genome alignments.
  • utils

    • Grouper can be used as disjoint set data structure.
    • range contains common range operations, like overlap and chaining.
    • Miscellaneous cookbook recipes, iterators decorators, table utilities.

Then there are modules that contain domain-specific methods.

  • assembly

    • K-mer histogram analysis.
    • Preparation and validation of tiling path for clone-based assemblies.
    • Scaffolding through ALLMAPS, optical map and genetic map.
    • Pre-assembly and post-assembly QC procedures.
  • annotation

    • Training of ab initio gene predictors.
    • Calculate gene, exon and intron statistics.
    • Wrapper for PASA and EVM.
    • Launch multiple MAKER processes.
  • compara

    • C-score based BLAST filter.
    • Synteny scan (de-novo) and lift over (find nearby anchors).
    • Ancestral genome reconstruction using Sankoff's and PAR method.
    • Ortholog and tandem gene duplicates finder.

Applications

Please visit wiki for full-fledged applications.

Dependencies

JCVI requires Python3 between v3.8 and v3.12.

Some graphics modules require the ImageMagick library.

On MacOS this can be installed using Conda (see next section). If you are using a linux system (i.e. Ubuntu) you can install ImageMagick using apt-get:

sudo apt-get update
sudo apt-get install libmagickwand-dev

See the Wand docs for instructions on installing ImageMagick on other systems.

A few modules may ask for locations of external programs, if the executable cannot be found in your PATH.

The external programs that are often used are:

Managing dependencies with Conda

You can use the the YAML files in this repo to create an environment with basic JCVI dependencies.

If you are new to Conda, we recommend the Miniforge distribution.

conda env create -f environment.yml

conda activate jcvi

Note: If you are using a Mac with an ARM64 (Apple Silicon) processor, some dependencies are not currently available from Bioconda for this architecture.

You can instead create a virtual OSX64 (intel) env like this:

conda env create -f env_osx64.yml

conda activate jcvi-osx64

After activating the Conda environment install JCVI using one of the following options.

Installation

Installation options

  1. Use pip to install the latest development version directly from this repo.
pip install git+git://github.com/tanghaibao/jcvi.git
  1. Install latest release from PyPi.
pip install jcvi
  1. Alternatively, if you want to install in development mode.
git clone git://github.com/tanghaibao/jcvi.git && cd jcvi
pip install -e '.[tests]'

Test Installation

If installed successfully, you can check the version with:

jcvi --version

Usage

Use python -m to call any of the modules installed with JCVI.

Most of the modules in this package contains multiple actions. To use the fasta example:

Usage:
    python -m jcvi.formats.fasta ACTION


Available ACTIONs:
          clean | Remove irregular chars in FASTA seqs
           diff | Check if two fasta records contain same information
        extract | Given fasta file and seq id, retrieve the sequence in fasta format
          fastq | Combine fasta and qual to create fastq file
         filter | Filter the records by size
         format | Trim accession id to the first space or switch id based on 2-column mapping file
        fromtab | Convert 2-column sequence file to FASTA format
           gaps | Print out a list of gap sizes within sequences
             gc | Plot G+C content distribution
      identical | Given 2 fasta files, find all exactly identical records
            ids | Generate a list of headers
           info | Run `sequence_info` on fasta files
          ispcr | Reformat paired primers into isPcr query format
           join | Concatenate a list of seqs and add gaps in between
     longestorf | Find longest orf for CDS fasta
           pair | Sort paired reads to .pairs, rest to .fragments
    pairinplace | Starting from fragment.fasta, find if adjacent records can form pairs
           pool | Pool a bunch of fastafiles together and add prefix
           qual | Generate dummy .qual file based on FASTA file
         random | Randomly take some records
         sequin | Generate a gapped fasta file for sequin submission
       simulate | Simulate random fasta file for testing
           some | Include or exclude a list of records (also performs on .qual file if available)
           sort | Sort the records by IDs, sizes, etc.
        summary | Report the real no of bases and N's in fasta files
           tidy | Normalize gap sizes and remove small components in fasta
      translate | Translate CDS to proteins
           trim | Given a cross_match screened fasta, trim the sequence
      trimsplit | Split sequences at lower-cased letters
           uniq | Remove records that are the same

Then you need to use one action, you can just do:

python -m jcvi.formats.fasta extract

This will tell you the options and arguments it expects.

Feel free to check out other scripts in the package, it is not just for FASTA.

Star History

Star History Chart

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jcvi-1.5.4.tar.gz (785.2 kB view details)

Uploaded Source

Built Distribution

jcvi-1.5.4-cp310-cp310-macosx_11_0_arm64.whl (1.1 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

File details

Details for the file jcvi-1.5.4.tar.gz.

File metadata

  • Download URL: jcvi-1.5.4.tar.gz
  • Upload date:
  • Size: 785.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.10.14

File hashes

Hashes for jcvi-1.5.4.tar.gz
Algorithm Hash digest
SHA256 865974660e0cebfabe79e86bdb82d8454f01830b4bfd0f6ff2c8da6d33b0d756
MD5 7b17731b59151a7b44faac5f9ff58f81
BLAKE2b-256 db594583ddfc5403e752d271a1e5ff7ce6629da294bb1209aa15c3d7885b347b

See more details on using hashes here.

File details

Details for the file jcvi-1.5.4-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for jcvi-1.5.4-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e2844c938f2b62cb35d0a5aa2d3a64d23a4be7596b6f4c7e05534e6ef9935f1d
MD5 1650d7d0c8338fe20346fbd5a255ec8d
BLAKE2b-256 c0f6ea0625ceb66ccaa5797bd871c40fcb3d9a4b23717b3ddf30226e1f0a429b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page