Easily integrate data in BigQuery
Project description
PyGBQ
Easily integrate data in BigQuery
Example
The following snippet
from pygbq import Client
import requests
client = Client(default_dataset='Finance')
def transactions(start_date):
data = requests.get(url='some/api/endpoint', headers={'start_date': start_date}).json()
client.update_table_using_temp(data=data, table_id='transactions', how=['id'])
return {'status': 200}
if __name__ == "__main__":
transactions("2020-11-25")
will upsert data to table Finance.transactions
by id
.
Install and set up
pip install pygbq
Set up the authentication.
Documentation
- Client - you can set
default_dataset
,save_dir
,path_to_key
- Client.update_table_using_temp - the main update function, use
how='insert'
to insert data andhow=['field1', 'field2']
to upsert (merge) byfield1
andfield2
- read_jsonl - read newline delimited json
- Client.get_secret - get a secret version from Secret Manager
- Client.add_secret - add a secret version to Secret Manager
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pygbq-0.24.tar.gz
(9.4 kB
view hashes)
Built Distribution
pygbq-0.24-py3-none-any.whl
(10.7 kB
view hashes)