Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

PyHive is a collection of Python DB-API and SQLAlchemy interfaces for Presto and Hive.

Usage

DB-API

from pyhive import presto
cursor = presto.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
print cursor.fetchone()
print cursor.fetchall()

DB-API (asynchronous)

from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)

status = cursor.poll().operationState
while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
    logs = cursor.fetch_logs()
    for message in logs:
        print message

    # If needed, an asynchronous query can be cancelled at any time with:
    # cursor.cancel()

    status = cursor.poll().operationState

print cursor.fetchall()

SQLAlchemy

First install this package to register it with SQLAlchemy (see setup.py).

from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
engine = create_engine('presto://localhost:8080/hive/default')
logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
print select([func.count('*')], from_obj=logs).scalar()

Note: query generation functionality is not exhaustive or fully tested, but there should be no problem with raw SQL.

Passing session configuration

# DB-API
hive.connect('localhost', configuration={'hive.exec.reducers.max': '123'})
presto.connect('localhost', session_props={'query_max_run_time': '1234m'})
# SQLAlchemy
create_engine(
    'hive://user@host:10000/database',
    connect_args={'configuration': {'hive.exec.reducers.max': '123'}},
)

Requirements

Install using

  • pip install pyhive[hive] for the Hive interface and

  • pip install pyhive[presto] for the Presto interface.

PyHive works with

  • Python 2.7

  • For Presto: Presto install

  • For Hive: HiveServer2 daemon

There’s also a third party Conda package.

Changelog

See https://github.com/dropbox/PyHive/releases.

Testing

https://travis-ci.org/dropbox/PyHive.svg http://codecov.io/github/dropbox/PyHive/coverage.svg?branch=master

Run the following in an environment with Hive/Presto:

./scripts/make_test_tables.sh
virtualenv --no-site-packages env
source env/bin/activate
pip install -e .
pip install -r dev_requirements.txt
py.test

WARNING: This drops/creates tables named one_row, one_row_complex, and many_rows, plus a database called pyhive_test_database.

Release files for PyHive-Hack 0.2.1b1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for PyHive-Hack 0.2.1b1
File Size Uploaded
PyHive-Hack-0.2.1b1.tar.gz 38.7 kB Details

Release files / PyHive-Hack-0.2.1b1.tar.gz

Download URL PyHive-Hack-0.2.1b1.tar.gz
Size 38.7 kB
Tags Source
SHA-256 checksum
How to use checksums
07210d1acf8ee35ecaa465c8fa89853675971b786b28645f966562b6464a0b5d
BLAKE2b-256 checksum
How to use checksums
7e2e457c3678b4d82d69b1bb257dbd722d06202b5e0ab5a1a7e1f5476a2c202d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via Python-urllib/2.7
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page