Package codes to execute queries on BigQuery and save to local machine
Project description
Credit
This repository is forked from https://github.com/IcarusSO/bigQueryExporter
Change Log (Compare with original version)
query_to_local()
- Create a temp table with a random hash on BQ as output table, so that simultaneous execution of the function will not overwrite each other.
- Remove the temp table after the execution. (May also set keep_temp_table=True if you wish to keep them).
bigQueryExporter
Export query data from google bigquery to local machine
Installation
pip install bigQueryExporterEnhanced
Prepare for the connection
from bigQueryExport import BigQueryExporter
bigQueryExporter = BigQueryExporter(project_name, dataset_name, bucket_name)
Query To Table
bigQueryExporter.query_to_table(query, job_name, dataset_name)
Table To GS
bigQueryExporter.table_to_gs(destination_table, job_name)
GS To Local
bigQueryExporter.gs_to_local(bucket, job_name, data_dir_path)
Query To GS (Query to Table + Table to GS)
bigQueryExporter.query_to_gs(query, job_name)
Query To Local (Query to Table + Table to GS + GS to Local)
export_path = bigQueryExporter.query_to_local(query)
# or with the options
export_path = bigQueryExporter.query_to_local(query,
job_name='simple_query',
data_dir_path='out/',
keep_temp_table=False,
overwrite_output_folder=True)
Requirement
-
Your server/ local machine should have the right to access the project
-
Right should be granted following the insturction on Google SDK
-
Execute the following command
gcloud auth application-default login
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for bigQueryExporterEnhanced-1.0.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0bda24c5c2783949e5b5c4cf1abdb4a8eb28ca286c74176ba3fcaf196e25d49 |
|
MD5 | 7a74bafc862e2ae677a05246cc511545 |
|
BLAKE2b-256 | 2630a17c63d029f68d6b1db3d187229bd9163c9f588b6d2f12c4e8b1d3ff38f3 |
Close
Hashes for bigQueryExporterEnhanced-1.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 572980f71e8b700481efd759dd70f0b2aa7ead94a68e6c7fe892f3646548b387 |
|
MD5 | 6ff5846e695e3ebe0abef25cf69fce8d |
|
BLAKE2b-256 | e076a7cb9ff5c330c8db9b1aa1f64d86430f5ac7514852062506c6c5449c682f |