Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

PyHive is a collection of Python DB-API and SQLAlchemy interfaces for Presto and Hive.

Usage

DB-API

from pyhive import presto
cursor = presto.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
print cursor.fetchone()
print cursor.fetchall()

DB-API (asynchronous)

from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)

status = cursor.poll().operationState
while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
    logs = cursor.fetch_logs()
    for message in logs:
        print message

    # If needed, an asynchronous query can be cancelled at any time with:
    # cursor.cancel()

    status = cursor.poll().operationState

print cursor.fetchall()

SQLAlchemy

First install this package to register it with SQLAlchemy (see setup.py).

from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
engine = create_engine('presto://localhost:8080/hive/default')
logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
print select([func.count('*')], from_obj=logs).scalar()

Note: query generation functionality is not exhaustive or fully tested, but there should be no problem with raw SQL.

Passing session configuration

# DB-API
hive.connect('localhost', configuration={'hive.exec.reducers.max': '123'})
presto.connect('localhost', session_props={'query_max_run_time': '1234m'})
# SQLAlchemy
create_engine(
    'hive://user@host:10000/database',
    connect_args={'configuration': {'hive.exec.reducers.max': '123'}},
)

Requirements

Install using

  • pip install pyhive[hive] for the Hive interface and

  • pip install pyhive[presto] for the Presto interface.

PyHive works with

  • Python 2.7

  • For Presto: Presto install

  • For Hive: HiveServer2 daemon

There’s also a third party Conda package.

Changelog

See https://github.com/dropbox/PyHive/releases.

Testing

https://travis-ci.org/dropbox/PyHive.svg http://codecov.io/github/dropbox/PyHive/coverage.svg?branch=master

Run the following in an environment with Hive/Presto:

./scripts/make_test_tables.sh
virtualenv --no-site-packages env
source env/bin/activate
pip install -e .
pip install -r dev_requirements.txt
py.test

WARNING: This drops/creates tables named one_row, one_row_complex, and many_rows, plus a database called pyhive_test_database.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

PyHive-Hack-0.2.1b1.tar.gz (38.7 kB view details)

Uploaded Source

File details

Details for the file PyHive-Hack-0.2.1b1.tar.gz.

File metadata

  • Download URL: PyHive-Hack-0.2.1b1.tar.gz
  • Upload date:
  • Size: 38.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: Python-urllib/2.7

File hashes

Hashes for PyHive-Hack-0.2.1b1.tar.gz
Algorithm Hash digest
SHA256 07210d1acf8ee35ecaa465c8fa89853675971b786b28645f966562b6464a0b5d
MD5 f212d1ca8d5fc820d369aa3462fa641a
BLAKE2b-256 7e2e457c3678b4d82d69b1bb257dbd722d06202b5e0ab5a1a7e1f5476a2c202d

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

0.2.1b1 This release

1 file

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page