ugbio_omics

This module includes Python scripts for working with AWS HealthOmics.

Log in with SSH to the desired AWS account before running tools in this package.

List of tools:

  1. Compare Cromwell vs. Omics - Process cost and performance on both platforms and save results into comparable CSV files. Also saves metadata and intermediate cost and performance files. Additionally, you can find a plots folder with a variety of HTML plots. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_cromwell_omics --help for more details.

  2. Compare Omics Runs - Process cost and performance and compare multiple Omics runs. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_omics_runs --help for more details.

  3. Get Run Logs - Download logs of an Omics run. For FAILED run by default, you'll get the logs of all failed tasks or the run's engine log if there are no failed tasks. For successful runs by default, you'll get all tasks' logs. You can use --task-id to get the log for a specific task.

    Run uv run get_omics_logs --help for more details.

  4. Get Omics Cache Path - Get S3 uri of the cache of an Omics run. You can choose to copy the indexes files using --copy-indexes. Use --task-id to get the cache uri / copy indexes for a specific task.

    Run uv run omics_cache_path --help for more details.

  5. Manifest Log - Download and parse the manifest log of an Omics run. The manifest log contains a lot of information about storage usage, CPU and memory usage per task, Docker images, inputs, and general information about the run.

  6. Performance - Process CPU, memory, and I/O metrics from the monitor log running in each task. Get usage plots over time for a better understanding of your code utilization. Note that task logs must be accessible for this.

    Run uv run performance --help for more details.

Admin tools:

  1. Report Run Completion - Manually triggers the OmicsEndRunHandler AWS Lambda to report mongodb on end of omics run/s. Run uv run report_run_completion --help for more details.
  2. Report Missing runs - Manually triggers the start and end AWS Lambda handlers to report mongodb on omics run/s that are missing from the db. Run uv run report_missing_runs --help for more details.
  3. Cost Distribution by Version - Analyze cost distribution per workflow for a specific version or version prefix. Queries the Papyrus database for runs matching a version (exact or prefix match), groups by workflow name, and generates histograms with cost statistics. Supports caching, outlier filtering, and tarball creation. Run uv run cost_distribution_by_version --help for more details.

Run with Docker

A fast and easy way to run the above-mentioned tools is by running a container:

docker run -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} -v <local_output>:<output> 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest <tool>

Breakdown:

  1. A simple docker run command.

  2. -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} - this will allow the Docker container to use the SSH profile you are logged into (customer/dev/prod).

    Remember to use alog before running the container.

  3. -v <local_output>:<output> - add a volume mapping and use it in the output path of the tool to get the outputs available outside the container.

  4. 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest - Docker image.

  5. <tool> - available tools are:

    • get_omics_logs
    • omics_cache_path
    • manifest_log
    • compare_omics_runs
    • compare_cromwell_omics

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.30.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ultimagen from gravatar.com ultimagen

Credits

Author: Inbal Zelig

Requires

Python >=3.11

Provides Extra

db

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ugbio_omics-1.29.0.tar.gz (30.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ugbio_omics-1.29.0-py3-none-any.whl (35.8 kB view details)

Uploaded Python 3

File details

Details for the file ugbio_omics-1.29.0.tar.gz.

File metadata

  • Download URL: ugbio_omics-1.29.0.tar.gz
  • Upload date:
  • Size: 30.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.29.0.tar.gz
Algorithm Hash digest
SHA256 5037ca344e1dd0a4e18a7204061c79261ae64cdb9024e72c85cd69c654d0d2b8
MD5 b664d034cfecd1227282261b0ad2cb0e
BLAKE2b-256 17fa86cbd45b70198f4162f93218b46cbfdaec44913b41e685e7fdbb12f426df

See more details on using hashes here.

File details

Details for the file ugbio_omics-1.29.0-py3-none-any.whl.

File metadata

  • Download URL: ugbio_omics-1.29.0-py3-none-any.whl
  • Upload date:
  • Size: 35.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.29.0-py3-none-any.whl
Algorithm Hash digest
SHA256 87e48ac20cfbf3995128ade0ebeab13e2d79918e28a5b8e253a8ee62d5e2bb0e
MD5 52a5d2e890a1f4443b46185c6328415e
BLAKE2b-256 fbbe67fab729b534a519b0ac6473db1c4e99718819356559e769505d40fba02d

See more details on using hashes here.

Release history Release notifications | RSS feed

1.30.2

2 files

1.30.1

2 files

1.30.0

2 files

This release

1.29.0 This release

2 files

1.28.1

2 files

1.28.0

2 files

1.27.1

2 files

1.27.0

2 files

1.26.0

2 files

1.25.0

2 files

1.24.2

2 files

1.24.1

2 files

1.24.0

2 files

1.23.0

2 files

1.22.2

2 files

1.22.1

2 files

1.22.0

2 files

1.21.4

2 files

1.21.3

2 files

1.21.2

2 files

1.21.1

2 files

1.21.0

2 files

1.20.0

2 files

1.19.0

2 files

1.18.0

2 files

1.17.2

2 files

1.17.1

2 files

1.17.0

2 files

1.16.2

2 files

1.16.1

2 files

1.16.0

2 files

1.15.0

2 files

1.14.0

2 files

1.13.2

2 files

1.13.1

2 files

1.13.0

2 files

1.12.0

2 files

1.11.0

2 files

1.10.2

2 files

1.10.1

2 files

1.10.0

2 files

1.8.0

2 files

1.7.0

2 files

1.6.1

2 files

1.6.0

2 files

1.5.5

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.4

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

1.3.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page