Skip to main content pypi package

Project description


Package for working with dataclasses and Google cloud instances as Bigquery and Storage. For this to work the dataclasses need to contain only basic python variables as: str, int, dict, list etc.


These modules dependant on the environment variable GOOGLE_CLOUD_PROJECT a.k.a. gcp project id.



Execute query and log errors

returns query_job

stream_to_bigquery(objects: list, table_ref):

Cast python objects to json and stream them to GBQ Note: this is more expensive compared to using buckets but also quicker

returns table ref

write_disposition_bucket(table_ref, blob_name, write_disposition):

Get data from bucket to GBQ using a write_disposition method

Requires: bucket with name storage_to_bigquery-[project-id]

  • WRITE_EMPTY This job should only be writing to empty tables.
  • WRITE_TRUNCATE This job will truncate table data and write from the beginning.
  • WRITE_APPEND This job will append to a table.
returns load_job.result 


storage_to_bigquery(objects: list, table_ref, write_disposition):

Function that stores data into multiple storage blobs. Afterwards these wil be composed into one storage blob The reason for this process is to downsize the sie of the data send to Google Cloud Storage.

Requires: bucket with name storage_to_bigquery-[project-id]

get_bucket(name_bucket: str):

returns name_bucket


returns blobs:list

upload_from_string(blob, string):

returns blob.public_url

Secrets (beta)


returns json

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for vaknl-gcp, version 1.1.7
Filename, size File type Python version Upload date Hashes
Filename, size vaknl_gcp-1.1.7-py3-none-any.whl (6.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size vaknl-gcp-1.1.7.tar.gz (4.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page