Skip to main content

Utility package to calculate cost of an AWS EMR cluster

Project description

EMR Cost Calculator

This project is a fork of [https://github.com/marko-bast/aws-emr-cost-calculator].

Original features:

  • Calculates exact costs of an EMR cluster (EMR + EC2 costs)
  • Multiple EMR clusters cost calculation for a given period
  • Spot prices and all other prices are exact and retrieved every time from AWS Pricing API
  • If a cluster is still running, costs incurred up to current time are displayed

In addition to the original project:

  • calculate cost for EMR Instance Fleets
  • compute cost of EBS

Why the need for this script

Given that Amazon doesn't provide a straightforward solution to calculate the cost of an EMR workflow, this module aims to calculate the cost of an EMR workflow given a period of days, or the cost of a single cluster given the cluster id. The simple way to do that would be to use the information given by the JobFLow method of the boto.emr module. However, this method doesn't return any information about the Task nodes of a cluster, and whether or not spot instances were used. This cost calculator takes care of both. OnDemand instance prices are retrieved using the AWS pricing API. In case spot instances were used, the price is retrieved using the AWS EC2 API.

EBS cost is fixed to 0.1 $ for month.

Install

To install or upgrade the package it's best to use pip: pip install -U aws-emr-cost-calculator2

How it works

This module is using docopt to parse command line arguments.

It currently supports two operations:

  1. Get the cost of an EMR cluster given the cluster id
  • aws-emr-cost-calculator2 cluster --cluster_id=<j-xxxxxxxxxxxx>

Authentication to AWS API is done using credentials of AWS CLI which are configured by executing aws configure

For example (if you want to use a different profile):

  • aws-emr-cost-calculator2 cluster --cluster_id=<j-xxxxxxxxxxxx> --profile=<your profile>

Remember to include your default region in the default section of your ./aws/config file.

Example run

$ aws-emr-cost-calculator cluster --cluster_id=j-K3C155R34111 --profile=myorg
CORE.EC2    :   0.40
CORE.EMR    :   0.11
CORE.EBS    :   0.01
MASTER.EC2  :   0.40
MASTER.EBS  :   0.01
MASTER.EMR  :   0.11
TASK.EC2    :   2.80
TASK.EMR    :   3.12
TOTAL       :   6.95

How release a new module

To release a new module you have to use:

  • pypi
  • twine

For example, to deploy the 0.1.1 version:

python setup.py sdist
twine upload dist/*0.1.1* -r pypi

License

Distributed under the MIT license. See LICENSE for more information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aws-emr-cost-calculator2-0.1.6.tar.gz (14.0 kB view details)

Uploaded Source

File details

Details for the file aws-emr-cost-calculator2-0.1.6.tar.gz.

File metadata

  • Download URL: aws-emr-cost-calculator2-0.1.6.tar.gz
  • Upload date:
  • Size: 14.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.7.1 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.2

File hashes

Hashes for aws-emr-cost-calculator2-0.1.6.tar.gz
Algorithm Hash digest
SHA256 9f472a3d7366599a636005f7a587fbe400c702e54a05e227c5563e95e8841f8f
MD5 32a0346817be9dc4bd41712672904a72
BLAKE2b-256 f0eb8b4e913e9b8d852571f6bfc7f670c51dafb96e4e6da87c46c8a418d32995

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page