Generate BigQuery tables, load and extract data, based on JSON Table Schema descriptors.
Project description
Generate and load BigQuery tables based on JSON Table Schema descriptors.
Version v0.3 contains breaking changes:
- renamed Storage.tables to Storage.buckets
- changed Storage.read to read into memory
- added Storage.iter to yield row by row
Getting Started
Installation
pip install jsontableschema-bigquery
Storage
Package implements Tabular Storage interface.
To start using Google BigQuery service:
- Create a new project - link
- Create a service key - link
- Download json credentials and set GOOGLE_APPLICATION_CREDENTIALS environment variable
We can get storage this way:
import io import os import json from apiclient.discovery import build from oauth2client.client import GoogleCredentials from jsontableschema_bigquery import Storage os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '.credentials.json' credentials = GoogleCredentials.get_application_default() service = build('bigquery', 'v2', credentials=credentials) project = json.load(io.open('.credentials.json', encoding='utf-8'))['project_id'] storage = Storage(service, project, 'dataset', prefix='prefix')
Then we could interact with storage:
storage.buckets storage.create('bucket', descriptor) storage.delete('bucket') storage.describe('bucket') # return descriptor storage.iter('bucket') # yields rows storage.read('bucket') # return rows storage.write('bucket', rows)
Mappings
schema.json -> bigquery table schema data.csv -> bigquery talbe data
Drivers
Default Google BigQuery client is used - docs.
API Reference
Detailed
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for jsontableschema-bigquery-0.5.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2aa5db666d81c652aa3a0cc6ffc7d5a668518a2455b9901d44f82af0f7d77197 |
|
MD5 | d545c1d3c8dc1f029b66bc541f8b8983 |
|
BLAKE2-256 | 07e6acc1ca3ef200c44967705c16f7c2db0c8dd9a8c274786e3b74f4eafa393c |
Close
Hashes for jsontableschema_bigquery-0.5.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab26f4b081130675b7ec0b2f55ecf047ad840a8493a990a93d3356f3ad6a053e |
|
MD5 | a00d2925498046f38be74fe091497aa8 |
|
BLAKE2-256 | 5808efcf7dab5af14aa683d67c519a0f6235c3c940d8d7dbd877093d891f5dc2 |