This release is a pre-release and may not be stable for production use.
PyHive is a collection of Python DB-API and SQLAlchemy interfaces for Presto and Hive.
Usage
DB-API
from pyhive import presto
cursor = presto.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
print cursor.fetchone()
print cursor.fetchall()
DB-API (asynchronous)
from pyhive import hive
from TCLIService.ttypes import TOperationState
cursor = hive.connect('localhost').cursor()
cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)
status = cursor.poll().operationState
while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
logs = cursor.fetch_logs()
for message in logs:
print message
# If needed, an asynchronous query can be cancelled at any time with:
# cursor.cancel()
status = cursor.poll().operationState
print cursor.fetchall()
SQLAlchemy
First install this package to register it with SQLAlchemy (see setup.py).
from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
engine = create_engine('presto://localhost:8080/hive/default')
logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
print select([func.count('*')], from_obj=logs).scalar()
Note: query generation functionality is not exhaustive or fully tested, but there should be no problem with raw SQL.
Passing session configuration
# DB-API
hive.connect('localhost', configuration={'hive.exec.reducers.max': '123'})
presto.connect('localhost', session_props={'query_max_run_time': '1234m'})
# SQLAlchemy
create_engine(
'hive://user@host:10000/database',
connect_args={'configuration': {'hive.exec.reducers.max': '123'}},
)
Requirements
Install using
pip install pyhive[hive] for the Hive interface and
pip install pyhive[presto] for the Presto interface.
PyHive works with
Python 2.7
For Presto: Presto install
For Hive: HiveServer2 daemon
There’s also a third party Conda package.
Changelog
Testing
Run the following in an environment with Hive/Presto:
./scripts/make_test_tables.sh virtualenv --no-site-packages env source env/bin/activate pip install -e . pip install -r dev_requirements.txt py.test
WARNING: This drops/creates tables named one_row, one_row_complex, and many_rows, plus a database called pyhive_test_database.
Release files for PyHive-Hack 0.2.1b1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| PyHive-Hack-0.2.1b1.tar.gz | 38.7 kB | Details |
Release files / PyHive-Hack-0.2.1b1.tar.gz
| Download URL | PyHive-Hack-0.2.1b1.tar.gz |
|---|---|
| Size | 38.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
07210d1acf8ee35ecaa465c8fa89853675971b786b28645f966562b6464a0b5d
|
|
BLAKE2b-256 checksum How to use checksums |
7e2e457c3678b4d82d69b1bb257dbd722d06202b5e0ab5a1a7e1f5476a2c202d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
Python-urllib/2.7
|