A local client for connecting to and working with Google BigQuery
Project description
shipyard-bigquery
The shipyard-bigquery
Python package is a local client for connecting to BigQuery to perform common operations such as loading and unloading data.
Installation
python3 -m pip install shipyard-bigquery
Usage
To initialize the client, provide the service account (in JSON format) as a string:
import os
from shipyard_bigquery import BigQueryClient
service_account = os.getenv('GOOGLE_APPLICATION_CREDENTIALS') # in this case, service account is stored as an environment variable
client = BigQueryClient(service_account)
client.connect()
Upload a file to a table in BigQuery
The upload method allows for the loading of a local CSV file to BigQuery and supports two different types of upload: overwrite
and append
.
client.upload(file = "my_file.csv", dataset = "my_dataset", table = "my_table", upload_type = 'overwrite')
Additionally, a the schema of the file to be loaded can be decalred ahead of time, otherwise they will be infered
schema = '[{"name": "string_col", "type": "string"}, {"name": "char_col", "type": "string"}, {"name": "int_col", "type": "Int64"}, {"name": "float_col", "type": "Float64"}, {"name": "bool_col", "type": "Bool"}, {"name": "date_col", "type": "Date"},{"name": "datetime_col", "type": "Datetime"}]'
client.upload(file = "my_file.csv", dataset = "my_dataset", table = "my_table", upload_type = 'overwrite',schema = schema)
Fetch Query Results
The fetch method allows for the downloading of a SQL query to a pandas dataframe
df = client.fetch('select * from my_table limit 10')
Execute Remote Queries
The execute method allows for you to execute queries without return values in BigQuery. Common operations would be dropping tables, deleting tables, or altering tables
client.execute_query('drop table if exists my_table')
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file shipyard_bigquery-0.1.4.tar.gz
.
File metadata
- Download URL: shipyard_bigquery-0.1.4.tar.gz
- Upload date:
- Size: 7.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.4.2 CPython/3.9.16 Darwin/22.5.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33e0c6a1b68699978bff5d3a1ac19714e69323811a5e8bfc28c7a8a450817669 |
|
MD5 | 91dbf3db788a1b88923c76ae325ab85a |
|
BLAKE2b-256 | 6beb9ea41334b592af8aa3001debfdfb0ae61632838b9ad7a30aaafb63c12f44 |
File details
Details for the file shipyard_bigquery-0.1.4-py3-none-any.whl
.
File metadata
- Download URL: shipyard_bigquery-0.1.4-py3-none-any.whl
- Upload date:
- Size: 9.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.4.2 CPython/3.9.16 Darwin/22.5.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 410fe071201dc20c6ad1735480bcd2ce1534886d25898ce88a95a5d020e91716 |
|
MD5 | 10916ac3726b425c1c2cf143dd362b31 |
|
BLAKE2b-256 | e3fb84c9c9d963481fad86de38cee07ddbdaaabe25f971f383959835e628d469 |