Select ClickHouse data, convert to pandas dataframes
Project description
clickhouse2pandas
Select ClickHouse data, convert to pandas dataframes and various other formats, by using the ClickHouse HTTP interface.
Features
- The transmitting data is compressed by default, which reduces network traffic and thus reduces the time for downloading data.
- Converts the ClickHouse query result into proper pandas data types, e.g., ClickHouse DateTime -> pandas datetime64.
- Minimum dependencies, 3 standard python libraries (urllib, gzip, json) and 1 external library (pandas).
Installation
pip install clickhouse2pandas
Usage
import clickhouse2pandas as ch2pd
connection_url = 'http://user:password@clickhouse_host:8123'
query = """
select *
from system.numbers
limit 100
"""
df = ch2pd.select_data(connection_url, query)
# df is a pandas dataframe converted from ClickHouse query result
API Reference
clickhouse2pandas.select_data(connection_url, query = None, convert_to = 'DataFrame', settings = None)
Return a formatted query result specified by "convert_to" parameter.
Parameters:
- connection_url: the connection url to the ClickHouse HTTP interface, e.g.,
http://user:password@clickhouse_host:8123
- query: the SQL query
- convert_to: convert the query result into specific format, could be one of the following: 'DataFrame', 'TabSeparated', 'TabSeparatedRaw', 'TabSeparatedWithNames', 'TabSeparatedWithNamesAndTypes', 'CSV', 'CSVWithNames', 'Values', 'Vertical', 'JSON', 'JSONCompact', 'JSONEachRow', 'TSKV', 'Pretty', 'PrettyCompact', 'PrettyCompactMonoBlock', 'PrettyNoEscapes', 'PrettySpace', 'XML'. Refer to ClickHouse Input and Output Formats
- settings: a dict containing the setting key-values, default settings are {'enable_http_compression': 1, 'send_progress_in_http_headers': 0,'log_queries': 1, 'connect_timeout': 10, 'receive_timeout': 300, 'send_timeout': 300, 'output_format_json_quote_64bit_integers': 0, 'wait_end_of_query': 0}. Refer to ClickHouse Settings
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for clickhouse2pandas-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0f321cc7a2b00be7ca1859eeb4d41a050831c17d5d255e348cfc17e87ef37c05 |
|
MD5 | c0e6d2fd3aafe8c4212254895e73142a |
|
BLAKE2b-256 | 392ae99e361f81e55f9e132ff1ffd1355de8ea776e531b9443342d6adbb96c8b |