ugbio_omics

This module includes Python scripts for working with AWS HealthOmics.

Log in with SSH to the desired AWS account before running tools in this package.

List of tools:

  1. Compare Cromwell vs. Omics - Process cost and performance on both platforms and save results into comparable CSV files. Also saves metadata and intermediate cost and performance files. Additionally, you can find a plots folder with a variety of HTML plots. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_cromwell_omics --help for more details.

  2. Compare Omics Runs - Process cost and performance and compare multiple Omics runs. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_omics_runs --help for more details.

  3. Get Run Logs - Download logs of an Omics run. For FAILED run by default, you'll get the logs of all failed tasks or the run's engine log if there are no failed tasks. For successful runs by default, you'll get all tasks' logs. You can use --task-id to get the log for a specific task.

    Run uv run get_omics_logs --help for more details.

  4. Get Omics Cache Path - Get S3 uri of the cache of an Omics run. You can choose to copy the indexes files using --copy-indexes. Use --task-id to get the cache uri / copy indexes for a specific task.

    Run uv run omics_cache_path --help for more details.

  5. Manifest Log - Download and parse the manifest log of an Omics run. The manifest log contains a lot of information about storage usage, CPU and memory usage per task, Docker images, inputs, and general information about the run.

  6. Performance - Process CPU, memory, and I/O metrics from the monitor log running in each task. Get usage plots over time for a better understanding of your code utilization. Note that task logs must be accessible for this.

    Run uv run performance --help for more details.

Admin tools:

  1. Report Run Completion - Manually triggers the OmicsEndRunHandler AWS Lambda to report mongodb on end of omics run/s. Run uv run report_run_completion --help for more details.
  2. Report Missing runs - Manually triggers the start and end AWS Lambda handlers to report mongodb on omics run/s that are missing from the db. Run uv run report_missing_runs --help for more details.
  3. Cost Distribution by Version - Analyze cost distribution per workflow for a specific version or version prefix. Queries the Papyrus database for runs matching a version (exact or prefix match), groups by workflow name, and generates histograms with cost statistics. Supports caching, outlier filtering, and tarball creation. Run uv run cost_distribution_by_version --help for more details.

Run with Docker

A fast and easy way to run the above-mentioned tools is by running a container:

docker run -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} -v <local_output>:<output> 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest <tool>

Breakdown:

  1. A simple docker run command.

  2. -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} - this will allow the Docker container to use the SSH profile you are logged into (customer/dev/prod).

    Remember to use alog before running the container.

  3. -v <local_output>:<output> - add a volume mapping and use it in the output path of the tool to get the outputs available outside the container.

  4. 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest - Docker image.

  5. <tool> - available tools are:

    • get_omics_logs
    • omics_cache_path
    • manifest_log
    • compare_omics_runs
    • compare_cromwell_omics

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.30.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ultimagen from gravatar.com ultimagen

Credits

Author: Inbal Zelig

Requires

Python >=3.11

Provides Extra

db

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ugbio_omics-1.30.0.tar.gz (30.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ugbio_omics-1.30.0-py3-none-any.whl (35.8 kB view details)

Uploaded Python 3

File details

Details for the file ugbio_omics-1.30.0.tar.gz.

File metadata

  • Download URL: ugbio_omics-1.30.0.tar.gz
  • Upload date:
  • Size: 30.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.30.0.tar.gz
Algorithm Hash digest
SHA256 df17e69f5a33fb60ef2c63e7e553d5a1e90c32900b80ea6be376730fc1263d63
MD5 542624bfbc0ca45efd3b73dfd1e1a5b9
BLAKE2b-256 fbc5440290db708171f0a69fda3d0793b46b5e6cf5fe7c07ddb7f31d9288df95

See more details on using hashes here.

File details

Details for the file ugbio_omics-1.30.0-py3-none-any.whl.

File metadata

  • Download URL: ugbio_omics-1.30.0-py3-none-any.whl
  • Upload date:
  • Size: 35.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.30.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c07dea761bfcdf81845fbf40325b31e9457ab9166fc4b062fd15aa9c901dca36
MD5 2cd11b59791578d034509e94bad2e77a
BLAKE2b-256 2583d743d06b48d69d890f71a9eef27c9e44e636ff5a8691ac70b1138be9fb50

See more details on using hashes here.

Release history Release notifications | RSS feed

1.30.2

2 files

1.30.1

2 files

This release

1.30.0 This release

2 files

1.29.0

2 files

1.28.1

2 files

1.28.0

2 files

1.27.1

2 files

1.27.0

2 files

1.26.0

2 files

1.25.0

2 files

1.24.2

2 files

1.24.1

2 files

1.24.0

2 files

1.23.0

2 files

1.22.2

2 files

1.22.1

2 files

1.22.0

2 files

1.21.4

2 files

1.21.3

2 files

1.21.2

2 files

1.21.1

2 files

1.21.0

2 files

1.20.0

2 files

1.19.0

2 files

1.18.0

2 files

1.17.2

2 files

1.17.1

2 files

1.17.0

2 files

1.16.2

2 files

1.16.1

2 files

1.16.0

2 files

1.15.0

2 files

1.14.0

2 files

1.13.2

2 files

1.13.1

2 files

1.13.0

2 files

1.12.0

2 files

1.11.0

2 files

1.10.2

2 files

1.10.1

2 files

1.10.0

2 files

1.8.0

2 files

1.7.0

2 files

1.6.1

2 files

1.6.0

2 files

1.5.5

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.4

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

1.3.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page