Use linear programming to allocate applications to cloud infrastructure
Project description
Malloovia
Use linear programming to allocate applications to cloud infrastructure.
- Free software: MIT license
- Documentation: https://malloovia.readthedocs.io.
Introduction
Malloovia is a Python package to solve virtual machine (VM) allocation problems in Infrastructure as a Service (IaaS) clouds from the point of view of the cloud customer. It was first presented in the paper "Cost Minimization of Virtual Machine Allocation in Public Clouds Considering Multiple Applications" presented at GECON 2017.
The problem to solve is: given a cloud infrastructure composed of different virtual machine types, each one with its own hardware characteristics, and prices, some of them with different pricing schemas, such as discounts for reservation over long periods, and given a set of applications to run on that infrastructure, each one with a different performance on each different VM type, and with a different workload over time, find the number of VMs of each type to activate at each timeslot for each application, so that the expected workload is fulfilled for all applications, the cloud provider limits are not exceeded and the total cost is minimized.
It works in two phases: first, it computes the number of reserved VMs using a Long Term Workload Prediction (LTWP) and then, it computes the number of on-demand for each time slot using a Short Term Workload Prediction (STWP).
Malloovia can be directly used in Python or by a CLI interface. The problems and the solutions can be saved using a YAML format.
This is an example that assumes that the problem definition is in problems.yaml
, with problem1
describing the LTWP and problem2
describing the STWP:
$ malloovia solve problems.yaml --phase-i-id=problem1 --phase-ii-id=problem2
Reading problems.yaml...(0.004s)
Solving phase I...(0.020s)
Solving Phase II |███████████████████████████████████| 100.0% - ETA: 0:00:00
(0.101s)
Writing solutions in problems-sol.yaml...(0.006s)
This is an example in Python (explained in more detail in the Usage section of the documentation):
from malloovia import *
# Infrastructure definition
region1 = LimitingSet("r1", name="us.east", max_vms=20)
zone1 = LimitingSet("r1_z1", name="us.east_a", max_vms=20)
m3large_z1 = InstanceClass(
"m3large_r1_z1", name="reserved m3.large in us.east_a",
limiting_sets=(zone1,), is_reserved=True,
price=7, max_vms=20)
m4xlarge_r1 = InstanceClass(
"m4xlarge_r1", name="ondemand m4.xlarge in us.east",
limiting_sets=(region1,), is_reserved=False,
price=10, max_vms=10)
# Performances
app0 = App("a0", "Web server")
app1 = App("a1", "Database")
performances = PerformanceSet(
id="example_perfs",
values=PerformanceValues({
m3large_z1: {app0: 12, app1: 500},
m4xlarge_r1: {app0: 44, app1: 1800}
})
)
# Workload
# Long term workload prediction of each app, for Phase I
ltwp_app0 = Workload(
"ltwp0", description="rph to the web server", app=app0,
values=(201, 203, 180, 220, 190, 211, 199, 204, 500, 200)
)
ltwp_app1 = Workload(
"ltwp1", description="rph to the database", app=app1,
values=(2010, 2035, 1807, 2202, 1910, 2110, 1985, 2033, 5050, 1992)
)
# Building the problem for phase I and solving
problem = Problem(
id="example1",
name="Example problem",
workloads=(ltwp_app0, ltwp_app1),
instance_classes=(m3large_z1, m4xlarge_r1),
performances=performances
)
phase_i_solution = PhaseI(problem).solve()
# Building the problem for a timeslot in phase II and solving
phase_ii = PhaseII(problem, phase_i_solution)
timeslot_solution = phase_ii.solve_timeslot(
workloads=(Workload("stwp0", app=app0, description=None, values=(315,)),
Workload("stwp1", app=app1, description=None, values=(1950,))
)
)
# Showing the cost and the allocation
print("Cost:", timeslot_solution.solving_stats.optimal_cost)
print(timeslot_solution.allocation._inspect())
You can find example problems and solutions in YAML format in the test data directory and in the GECON 2017 data repository, where you can find a notebook that shows how to compute the solutions from the problems.
Please, refer to the documentation and the he paper "Cost Minimization of Virtual Machine Allocation in Public Clouds Considering Multiple Applications" for more details.
History
0.1.0 (2017-07-24)
- First release on PyPI. .. _section-1:
0.2.0 (2017-07-27)
from malloovia import *
imports all relevant classes and methods.read_problems_from_github()
added.- Integration with Travis-CI and ReadTheDocs.
- Working on the documentation.
- Modified YAML schema of the Solutions.
.. _section-2:
0.3.0 (2017-07-31)
- Much improved documentation. Windows installation covered.
- Command-line interface
- Changed from PyYAML to ruamel.yaml, much faster
- Read from YAML now accepts gzipped files too
- Some bugs fixed in the schema
.. _section-3:
1.0.0 (2017-11-01)
- Incompatible API change: it is required to specify
time_unit
inInstanceClass
,PerformanceSet
andWorkload
classes, in order to clarify the time unit for price, performance and workload timeslots. - Added utility function to read solutions from yaml files.
- Revised documentation and code quality. Improved README for github.
- Minor bugfixes.
.. _section-4:
1.0.1 (2018-01-12)
- Bugfix to make all malloovia classes pickable, allowing for multiprocessing.
.. _section-5: 1.1.0 (2018-03-16)
- New class
PhaseIIGuided
which allows to solve a single timeslot using a given allocation which specifies the minimum number of VMs to keep running.
.. _section-6:
2.1.1 (2019-06-20)
- Internal refactorization of Malloovia Model’s classes, which are now
based on
typing.NamedTuple
instead ofcollections.namedtuple
, which allows for proper type checking and documentation of the fields. - Several typing bugs related to YAML export and import fixed.
- This version introduces backwards incompatibility, since it requires python 3.6+ to run. However the API and usage is the same.
.. _section-7:
2.2.0 (2020-03-04)
- Updated to work with PuLP 2.0 (and fix that version in setup.py)
- Fixed problem with LP variable names too long wen the number of apps in the problem is large.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for malloovia-2.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39e125639c04a9898e932822739dc37878e70a52cfbbc7c643e1c27bd9ebb0d7 |
|
MD5 | e9e916bd5474eb63def2b992c8673350 |
|
BLAKE2b-256 | bcc1ba8b856e9678e375c7002fb8c11f9bd39e7a1106bafe4354009909a3e5bf |