Skip to main content

GWAS summary statistics file validator

Project description

Summary Statistics TSV file Validator

A file validator for validating GWAS summary statistics TSV files prior to and post harmonisation using pandas_schema. The purpose is to validate files before their conversion to HDF5.


  • python3


  • pip install ss-validate

Running the validator

To run the validator on a file:

  • ss-validate -f <file_to_validate.tsv> --logfile <logfile_name>

Information and errors are logged to the console and errors logged to the file specified. A console output might look like:

(INFO): Filename is good!
(INFO): Validating file...
(ERROR): Length of row 7 is: 16 instead of 15
(ERROR): Please fix the table. Some rows have different numbers of columns to the header
(INFO): Rows with different numbers of columns to the header are not validated
(ERROR): {row: 1, column: "p_value"}: "-99" was not in the range [0, 1)

The errors from the output tell us that row seven has too many columns and row one does not have a valid pvalue.

Addional options

  • --linelimit : int, default 1000

    Once this number of erroneous rows has been reached, stop looking for more.

  • --minrows : int, default 100000

    The minimum number of rows the file is required to have in order to validate sucZZcessfully.

  • --drop-bad-lines : bool, default False

    Drops the the lines with errors from the file and writes it to a new file called <file_to_validate.tsv.valid>

  • --stage : {'standard', 'harmonised', 'curated'}, default 'standard'

    The stage the file is in. It is either standard format ('standard'), harmonised ('harmonised') or pre-standard in the custom curated format ('curated'). Recommended to leave as default.

Import ss-validate to another python script

  • Install as above
  • Import and use in your python file
import validate.validator as ssv

# initialise a validator object for your summary statistics and settings 
validator = ssv.Validator(file='sumstats.tsv.gz', filetype='gwas-upload', error_limit=1, logfile='logfile.log')

# validate the headers

# validate the squareness

# validate the data

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ss-validate-0.4.8.tar.gz (11.6 kB view hashes)

Uploaded Source

Built Distribution

ss_validate-0.4.8-py3-none-any.whl (12.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page