Skip to main content

A query predictor pipeline and service to predict resource usages of Presto queries

Project description


presto-query-predictor is a Python module introducing machine learning techniques to the Presto ecosystem. It contains a machine learning pipeline for the model training/evaluation and a query predictor web service to predict CPU and memory usages of Presto queries.


After cloning the GitHub repository,

pip3 install -e .  # Installs the presto-query-predictor package locally
pip3 install -r requirements.txt  # Installs dependencies

An alternative way is to install the package from PyPi,

pip3 install presto-query-predictor

We recommend installing the package in a Python virtual environment instead of installing it globally.


The query_predictor/ folder contains the core of the package. We have prepared some examples in the example/ folder, including

  • - An example to load the embedded fake TPCH-based dataset.
  • - An example to transform datasets for further training.
  • - An example to train CPU and memory models.
  • - An example to tune classification algorithms.
  • - An example to create a query predictor web service.


A simple way to get a sense of the CPU and memory model training is running the examples in the example/ folder.

cd examples

The presto-query-predictor package can only be executed in a Python 3 environment. It does not support Python 2.

Afterward, the trained models should be generated in the models folder, including


By default, the vectorizers are trained from the TF-IDF algorithm, and the models are trained from XGBoost classifiers. The dataset used for training is a faked dataset based on the TPC-H benchmark with only 22 samples.


After running


A Flask web application should be created at There is a web UI for the application where you can fill in the form with a query for resources prediction.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for presto-query-predictor, version 0.1.4
Filename, size File type Python version Upload date Hashes
Filename, size presto_query_predictor-0.1.4-py3.7.egg (99.7 kB) File type Egg Python version 3.7 Upload date Hashes View
Filename, size presto-query-predictor-0.1.4.tar.gz (373.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page