Skip to main content

Utility package to calculate cost of an AWS EMR cluster

Project description

EMR Cost Calculator

Features at a glance

  • Calculates exact costs of an EMR cluster (EMR + EC2 costs)
  • Multiple EMR clusters cost calculation for a given period
  • Spot prices and all other prices are exact and retrieved every time from AWS Pricing API
  • If a cluster is still running, costs incurred up to current time are displayed

Why the need for this script

Given that Amazon doesn't provide a straightforward solution to calculate the cost of an EMR workflow, this module aims to calculate the cost of an EMR workflow given a period of days, or the cost of a single cluster given the cluster id. The simple way to do that would be to use the information given by the JobFLow method of the boto.emr module. However, this method doesn't return any information about the Task nodes of a cluster, and whether or not spot instances were used. This cost calculator takes care of both. OnDemand instance prices are retrieved using the AWS pricing API. In case spot instances were used, the price is retrieved using the AWS EC2 API.

How it works

This module is using docopt to parse command line arguments.

It currently supports two operations:

  1. Get the total cost of an EMR workflow for a given period of days
  • aws-emr-cost-calculator total --created_after=<YYYY-MM-DD> --created_before=<YYYY-MM-DD>
  1. Get the cost of an EMR cluster given the cluster id
  • aws-emr-cost-calculator cluster --cluster_id=<j-xxxxxxxxxxxx>

Authentication to AWS API is done using credentials of AWS CLI which are configured by executing aws configure

Install

To install all requirements it's best to use pip install -r requirements.txt

Users with python<2.7.9 won't be able to run the code if requests[security] isn't installed (which is listed in requirements.txt)
Python 3.7 is tested, lower 3x versions will probably work though.

License

Distributed under the MIT license. See LICENSE for more information.

Project details


Release history Release notifications

This version

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aws-emr-cost-calculator, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size aws_emr_cost_calculator-0.0.1-py3-none-any.whl (7.7 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size aws-emr-cost-calculator-0.0.1.tar.gz (6.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page