ugbio_omics

This module includes Python scripts for working with AWS HealthOmics.

Log in with SSH to the desired AWS account before running tools in this package.

List of tools:

  1. Compare Cromwell vs. Omics - Process cost and performance on both platforms and save results into comparable CSV files. Also saves metadata and intermediate cost and performance files. Additionally, you can find a plots folder with a variety of HTML plots. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_cromwell_omics --help for more details.

  2. Compare Omics Runs - Process cost and performance and compare multiple Omics runs. Omics cost information is fetched from cost.csv in S3 run output bucket.

    Run uv run compare_omics_runs --help for more details.

  3. Get Run Logs - Download logs of an Omics run. For FAILED run by default, you'll get the logs of all failed tasks or the run's engine log if there are no failed tasks. For successful runs by default, you'll get all tasks' logs. You can use --task-id to get the log for a specific task.

    Run uv run get_omics_logs --help for more details.

  4. Get Omics Cache Path - Get S3 uri of the cache of an Omics run. You can choose to copy the indexes files using --copy-indexes. Use --task-id to get the cache uri / copy indexes for a specific task.

    Run uv run omics_cache_path --help for more details.

  5. Manifest Log - Download and parse the manifest log of an Omics run. The manifest log contains a lot of information about storage usage, CPU and memory usage per task, Docker images, inputs, and general information about the run.

  6. Performance - Process CPU, memory, and I/O metrics from the monitor log running in each task. Get usage plots over time for a better understanding of your code utilization. Note that task logs must be accessible for this.

    Run uv run performance --help for more details.

Admin tools:

  1. Report Run Completion - Manually triggers the OmicsEndRunHandler AWS Lambda to report mongodb on end of omics run/s. Run uv run report_run_completion --help for more details.
  2. Report Missing runs - Manually triggers the start and end AWS Lambda handlers to report mongodb on omics run/s that are missing from the db. Run uv run report_missing_runs --help for more details.
  3. Cost Distribution by Version - Analyze cost distribution per workflow for a specific version or version prefix. Queries the Papyrus database for runs matching a version (exact or prefix match), groups by workflow name, and generates histograms with cost statistics. Supports caching, outlier filtering, and tarball creation. Run uv run cost_distribution_by_version --help for more details.

Run with Docker

A fast and easy way to run the above-mentioned tools is by running a container:

docker run -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} -v <local_output>:<output> 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest <tool>

Breakdown:

  1. A simple docker run command.

  2. -v ~/.aws:/root/.aws -e AWS_PROFILE=${AWS_PROFILE} - this will allow the Docker container to use the SSH profile you are logged into (customer/dev/prod).

    Remember to use alog before running the container.

  3. -v <local_output>:<output> - add a volume mapping and use it in the output path of the tool to get the outputs available outside the container.

  4. 337532070941.dkr.ecr.us-east-1.amazonaws.com/ugbio_omics:latest - Docker image.

  5. <tool> - available tools are:

    • get_omics_logs
    • omics_cache_path
    • manifest_log
    • compare_omics_runs
    • compare_cromwell_omics

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.30.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ultimagen from gravatar.com ultimagen

Credits

Author: Inbal Zelig

Requires

Python >=3.11

Provides Extra

db

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ugbio_omics-1.28.1.tar.gz (30.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ugbio_omics-1.28.1-py3-none-any.whl (35.8 kB view details)

Uploaded Python 3

File details

Details for the file ugbio_omics-1.28.1.tar.gz.

File metadata

  • Download URL: ugbio_omics-1.28.1.tar.gz
  • Upload date:
  • Size: 30.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.28.1.tar.gz
Algorithm Hash digest
SHA256 d48692cfaa4f509487a8dc1bb13a8974636248dda5d72b9d14243e0dc84d08ed
MD5 8a10744b93c0528bca77df4fc3c0e3db
BLAKE2b-256 7108eed0ec51aa83664f27eb249575f026713320a2ea80129c33c773af633991

See more details on using hashes here.

File details

Details for the file ugbio_omics-1.28.1-py3-none-any.whl.

File metadata

  • Download URL: ugbio_omics-1.28.1-py3-none-any.whl
  • Upload date:
  • Size: 35.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ugbio_omics-1.28.1-py3-none-any.whl
Algorithm Hash digest
SHA256 37c5486c96fa4a69fdd5105d73db2c075f8f494e322bcbae37230942c864ea76
MD5 6ebf58236d113a850734658bfc8c8c6e
BLAKE2b-256 3d6f2596dd84956e897297b94ad43e97aa47ed1b61d5b95768d8382e2521627c

See more details on using hashes here.

Release history Release notifications | RSS feed

1.30.2

2 files

1.30.1

2 files

1.30.0

2 files

1.29.0

2 files

This release

1.28.1 This release

2 files

1.28.0

2 files

1.27.1

2 files

1.27.0

2 files

1.26.0

2 files

1.25.0

2 files

1.24.2

2 files

1.24.1

2 files

1.24.0

2 files

1.23.0

2 files

1.22.2

2 files

1.22.1

2 files

1.22.0

2 files

1.21.4

2 files

1.21.3

2 files

1.21.2

2 files

1.21.1

2 files

1.21.0

2 files

1.20.0

2 files

1.19.0

2 files

1.18.0

2 files

1.17.2

2 files

1.17.1

2 files

1.17.0

2 files

1.16.2

2 files

1.16.1

2 files

1.16.0

2 files

1.15.0

2 files

1.14.0

2 files

1.13.2

2 files

1.13.1

2 files

1.13.0

2 files

1.12.0

2 files

1.11.0

2 files

1.10.2

2 files

1.10.1

2 files

1.10.0

2 files

1.8.0

2 files

1.7.0

2 files

1.6.1

2 files

1.6.0

2 files

1.5.5

2 files

1.5.4

2 files

1.5.3

2 files

1.5.2

2 files

1.5.1

2 files

1.5.0

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.4

2 files

1.3.3

2 files

1.3.2

2 files

1.3.1

2 files

1.3.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page