Skip to main content

Python interface to Hive

Project description

PyHive is a collection of Python DB-API and SQLAlchemy interfaces for Presto and Hive.



from pyhive import presto  # or import hive
cursor = presto.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
print cursor.fetchone()
print cursor.fetchall()

DB-API (asynchronous)

from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)

status = cursor.poll().operationState
while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
    logs = cursor.fetch_logs()
    for message in logs:
        print message

    # If needed, an asynchronous query can be cancelled at any time with:
    # cursor.cancel()

    status = cursor.poll().operationState

print cursor.fetchall()


First install this package to register it with SQLAlchemy (see

from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
# Presto
engine = create_engine('presto://localhost:8080/hive/default')
# Hive
engine = create_engine('hive://localhost:10000/default')
logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
print select([func.count('*')], from_obj=logs).scalar()

Note: query generation functionality is not exhaustive or fully tested, but there should be no problem with raw SQL.

Passing session configuration

hive.connect('localhost', configuration={'hive.exec.reducers.max': '123'})
presto.connect('localhost', session_props={'query_max_run_time': '1234m'})
# SQLAlchemy
    connect_args={'protocol': 'https',
                  'session_props': {'query_max_run_time': '1234m'}}
    connect_args={'configuration': {'hive.exec.reducers.max': '123'}},
# SQLAlchemy with LDAP
    connect_args={'auth': 'LDAP'},


Install using

  • pip install pyhive[hive] for the Hive interface and

  • pip install pyhive[presto] for the Presto interface.

PyHive works with

  • Python 2.7 / Python 3

  • For Presto: Presto install

  • For Hive: HiveServer2 daemon

  • For Python 3 + Hive + SASL, you currently need to install an unreleased version of thrift_sasl (pip install git+ At the time of writing, the latest version of thrift_sasl was 0.2.1.




  • Please fill out the Dropbox Contributor License Agreement at and note this in your pull request.

  • Changes must come with tests, with the exception of trivial things like fixing comments. See .travis.yml for the test environment setup.

  • Notes on project scope:

    • This project is intended to be a minimal Hive/Presto client that does that one thing and nothing else. Features that can be implemented on top of PyHive, such integration with your favorite data analysis library, are likely out of scope.

    • We prefer having a small number of generic features over a large number of specialized, inflexible features. For example, the Presto code takes an arbitrary requests_session argument for customizing HTTP calls, as opposed to having a separate parameter/branch for each requests option.


Run the following in an environment with Hive/Presto:

virtualenv --no-site-packages env
source env/bin/activate
pip install -e .
pip install -r dev_requirements.txt

WARNING: This drops/creates tables named one_row, one_row_complex, and many_rows, plus a database called pyhive_test_database.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

PyHive-0.5.1.tar.gz (41.2 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page