The repo contains a benchmarking script for some of the CWL workflows used by 4DN-DCIC (https://github.com/4dn-dcic/pipelines-cwl), that returns total space, mem and CPUs required per given input size and a recommended AWS EC2 instance type.
Example usage of benchmarking script
- importing the module
from Benchmark import run as B
- md5
app_name = 'md5'
input_json = {'input_size_in_bytes': {'input_file': 20000}}
B.benchmark(app_name, input_json)
{'aws': {'recommended_instance_type': 't2.xlarge', 'EBS_optimized': False, 'cost_in_usd': 0.188, 'EBS_optimization_surcharge': None, 'mem_in_gb': 16.0, 'cpu': 4}, 'total_size_in_GB': 14.855186462402344, 'total_mem_in_MB': 13142.84375, 'min_CPU': 4}
- fastqc-0-11-4-1
app_name = 'fastqc-0-11-4-1'
input_json = {'input_size_in_bytes': {'input_fastq':20000},
'parameters': {'threads': 2}}
B.benchmark(app_name, input_json)
{'recommended_instance_type': 't2.nano', 'EBS_optimized': False, 'cost_in_usd': 0.006, 'EBS_optimization_surcharge': None, 'mem_in_gb': 0.5, 'cpu': 1}
- bwa-mem
app_name = 'bwa-mem'
input_json = {'input_size_in_bytes': {'fastq1':93520000,
'fastq2':97604000,
'bwa_index':3364568000},
'parameters': {'nThreads': 4}}
B.benchmark(app_name, input_json)
{'aws': {'cost_in_usd': 0.188, 'EBS_optimization_surcharge': None, 'EBS_optimized': False, 'cpu': 4, 'mem_in_gb': 16.0, 'recommended_instance_type': 't2.xlarge'}, 'total_mem_in_MB': 12834.808349609375, 'total_size_in_GB': 15.502477258443832, 'min_CPU': 4}
To use Benchmark in from other places, install it as below.
pip install Benchmark-4dn
or
pip install git+git://github.com/SooLee/Benchmark.git
Note: From 0.5.3 we have a new function that takes in cpu and memory and returns a sorted list of instance dictionaries.
get_instance_types(cpu=1, mem_in_gb=0.5, instances=instance_list(), top=10, rank='cost_in_usd')
Keys in each instance dictionary:
'cost_in_usd', 'mem_in_gb', 'cpu', 'instance_type', 'EBS_optimized', 'EBS_optimization_surcharge'
Release files for benchmark-4dn 0.5.28
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| benchmark_4dn-0.5.28.tar.gz | 31.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| benchmark_4dn-0.5.28-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 63.8 kB
Release files / benchmark_4dn-0.5.28.tar.gz
| Download URL | benchmark_4dn-0.5.28.tar.gz |
|---|---|
| Size | 31.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
71a938af987dfcd7099b29d4b37e445b03c2809e3d90c269df901e01eedce680
|
|
BLAKE2b-256 checksum How to use checksums |
e431325c40dbb5f7489576309f8aa6337e24a465cc927173a147b0a34fed5e4e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.4.2 CPython/3.11.13 Darwin/23.6.0
|
Release files / benchmark_4dn-0.5.28-py3-none-any.whl
| Download URL | benchmark_4dn-0.5.28-py3-none-any.whl |
|---|---|
| Size | 32.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5e23ceab52c874d68cfdd86d5beee015a6f3b2b07a5c0c55f31c10bfe5b7c4f5
|
|
BLAKE2b-256 checksum How to use checksums |
8f4084de2ec04265ed9f5aed24465ce6dd7e9344a839c17817b8d354cf5f3e1b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.4.2 CPython/3.11.13 Darwin/23.6.0
|
