Skip to main content

Summarize LSF job properties by parsing log files.

Project description


PyPI Tests

Summarize LSF job properties by parsing log files of workflows executed by Snakemake.


$ pip install lsf_stats


$ lsf_stats --help
Usage: lsf_stats [OPTIONS] COMMAND [ARGS]...

  Summarize LSF job properties by parsing log files.

  --version  Show the version and exit.
  --help     Show this message and exit.

  gather     Aggregate information from log files in single dataframe.
  summarize  Summarize and visualize aggregated information.


Assume that you executed your Snakemake workflow using the lsf-profile and all generated log files are stored in the directory ./logs/:

$ snakemake --profile lsf

You can then quickly aggregate resource, rule and other types of information about the workflow execution into a single dataframe:

$ lsf_stats gather -o workflow_stats.csv.gz ./logs/

This dataframe can then be summarized in various ways:

$ lsf_stats summarize \
    --query 'status == "Successfully completed."' \
    --split-wildcards \
    --grouping-variable category \

For example, the following plots will be generated:

Job execution Job resources
Job execution Job resources

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lsf_stats-0.3.0.tar.gz (5.2 kB view hashes)

Uploaded Source

Built Distribution

lsf_stats-0.3.0-py3-none-any.whl (5.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page