Bioinformatics tool outputs converter to JSON or YAML.
Project description
crimson
crimson converts non-standard bioinformatics tool outputs to JSON or YAML.
Currently it accepts outputs of the following programs:
FastQC (fastqc)
FusionCatcher (fusioncatcher)
samtools flagstat (flagstat)
Picard metrics tools (picard)
STAR log file (star)
STAR-Fusion hits table (star-fusion)
Variant Effect Predictor plain text output (vep)
From those, you can convert the respective output files into JSON (the default) or YAML. You can also use crimson in your scripts by importing the parser functions themselves.
Installation
crimson is available via the Python Package Index and you can install it via pip:
$ pip install crimson
It is tested on the following Python versions:
3.8
3.7
3.6
and against the following bioinformatics tools:
FastQC (version 0.10.1)
FusionCatcher (version 0.99.5a)
samtools (version 0.19.1, 1.1)
Picard (version 1.124)
STAR (version 2.3.0)
STAR-Fusion (version 0.6.0, 1.6.0)
VEP (version 77)
Usage
Command-line
The general command is crimson {program_name} and by default the output is written to stdout. For example, to use the picard parser, you would execute:
$ crimson picard /path/to/a/picard.metrics
You can also specify a file name directly to write to a file. The following command will write the output to a file named converted.json:
$ crimson picard /path/to/a/picard.metrics converted.json
Some parsers may also accept additional input format. The FastQC parser, for example, also works if you give it a path to the FastQC output directory:
$ crimson fastqc /path/to/a/fastqc/dir
or path to a zipped result:
$ crimson fastqc /path/to/a/fastqc_result.zip
When in doubt, use the --help flag:
$ crimson --help # for the general help
$ crimson fastqc --help # for parser-specific (FastQC) help
Python Module
The function to import is located at crimson.{program_name}.parser. So to use the picard parser in your script, you can do this:
from crimson import picard
# You can supply the file name as string ...
parsed = picard.parse("/path/to/a/picard.metrics")
# ... or a file handle directly
with open("/path/to/a/picard.metrics") as src:
parsed = picard.parse(src)
Why?
Not enough tools use standard output formats.
Writing and re-writing the same parsers across different scripts is not a productive way to spend the day.
Contributing
If you are interested, crimson accepts the following types contribution:
Documentation additions (if anything seems unclear, feel free to open an issue)
Bug reports
Support for tools’ outputs which can be converted to JSON or YAML.
For any of these, feel free to open an issue in the issue tracker or submitt a pull request.
Local Development
Setting up a local development requires any of the supported Python version. It is ideal if you have support Python 2.x and 3.x versions installed, as that will allow you to run the full tests suite against all versions using tox.
In any case, the following steps can be your guide for setting up your local development environment:
# Clone the repository and cd into it
$ git clone {repo-url}
$ cd crimson
# Create your virtualenv, using pyenv for example (recommended, https://github.com/pyenv/pyenv)
$ pyenv virtualenv 3.7.0 crimson-dev
# or using virtualenvwrapper (https://virtualenvwrapper.readthedocs.io/en/latest/)
$ mkvirtualenv -p /usr/bin/python3.7 crimson-dev
# From within the root directory and with an active virtualenv, install the dependencies and package itself
$ pip install -e .[dev]
License
crimson is BSD-licensed. Refer to the LICENSE file for the full license.
Changelog
This format is based on Keep a Changelog and this project adheres to Semantic Versioning.
Unreleased
[0.5.2] - 2020-06-30
Fixed
FusionCatcher v1.20 output without any result rows is now parsed properly (see #11).
Sections in VEP files containing no data is now parsed properly (see #13).
[0.5.1] - 2020-02-27
Changed
Relaxed Click and YAML requirements. Now crimon requires only minimum versions of these dependencies instead of exact ones.
[0.5.0] - 2020-02-04
Added
Support for parsing output of STAR-Fusion v1.6.0 under the same star-fusion parser. Thank you @Redmar-van-den-Berg!
Removed
Support for Python 2.7, 3.3, 3.4, and 3.5
[0.4.0] - 2018-07-25
Added
Support for parsing zipped FastQC result.
Changed
Improved detection of zipped FastQC input.
Set configurable file-size limits for flagstat, Picard, and FastQC.
[0.3.0] - 2016-05-19
Added
Support for parsing FusionCatcher final fusion genes file.
[0.2.0] - 2016-04-13
Added
Support for parsing STAR-Fusion hits table output.
Support for parsing STAR alignment log output.
Support for parsing VEP plain text output.
[0.1.1] - 2016-02-02
Changed
Test and build dependencies.
[0.1.0] - 2015-07-27
Added
First release.
Support for parsing FastQC, samtools flagstat, and Picard.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.