tableschema-bigquery-py

Generate and load BigQuery tables based on Table Schema descriptors.

Features

• implements tableschema.Storage interface

Contents

• Getting Started
  • Installation
  • Prepare BigQuery
• Documentation
• API Reference
  • Storage
• Contributing
• Changelog

Getting Started

Installation

The package use semantic versioning. It means that major versions could include breaking changes. It's highly recommended to specify package version range in your setup/requirements file e.g. package>=1.0,<2.0.

pip install tableschema-bigquery

Prepare BigQuery

To start using Google BigQuery service:

• Create a new project - link
• Create a service key - link
• Download json credentials and set GOOGLE_APPLICATION_CREDENTIALS environment variable

Documentation

import io
import os
import json
from datapackage import Package
from apiclient.discovery import build
from oauth2client.client import GoogleCredentials

# Prepare BigQuery credentials
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '.credentials.json'
credentials = GoogleCredentials.get_application_default()
service = build('bigquery', 'v2', credentials=credentials)
project = json.load(io.open('.credentials.json', encoding='utf-8'))['project_id']

# Save package to BigQuery
package = Package('datapackage.json')
package.save(storage='bigquery', service=service, project=project, dataset='dataset')

# Load package from BigQuery
package = Package(storage='bigquery', service=service, project=project, dataset='dataset')
package.resources

API Reference

Storage

Storage(self, service, project, dataset, prefix='')

BigQuery storage

Package implements Tabular Storage interface (see full documentation on the link):

Only additional API is documented

Arguments

• service (object): BigQuery Service object
• project (str): BigQuery project name
• dataset (str): BigQuery dataset name
• prefix (str): prefix for all buckets

Contributing

The project follows the Open Knowledge International coding standards.

Recommended way to get started is to create and activate a project virtual environment. To install package and development dependencies into active environment:

$ make install

To run tests with linting and coverage:

$ make test

Changelog

Here described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted commit history.

v1.0

• Initial driver realease