Pypi Build Status codecov MIT licensed
bigquery_fdw is a BigQuery foreign data wrapper for PostgreSQL using Multicorn.
It allows to write queries in PostgreSQL SQL syntax using a foreign table. It supports most of BigQuery’s data types and operators.
Features and limitations
Table partitioning is supported. You can use partitions in your SQL queries.
Queries are parameterized when sent to BigQuery
BigQuery’s standard SQL support (legacy SQL is not supported)
Authentication works with a “Service Account” Json private key
Requirements
PostgreSQL >= 9.5
Python 3
Get started
Using docker
Installation on Debian/Ubuntu
Dependencies required to install bigquery_fdw:
You need to install the following dependencies:
# Install required packages
apt-get update
apt-get install --yes postgresql-server-dev-10 python3-setuptools python3-dev make gcc git
# Install pip3
easy_install3 pipFor PostgresSQL 9.X, install postgresql-server-dev-9.X instead of postgresql-server-dev-10.
Installation
# Install `setuptools` if necessary
pip3 install --upgrade setuptools
# Install Multicorn
git clone git://github.com/Kozea/Multicorn.git && cd Multicorn
export PYTHON_OVERRIDE=python3
make && make install
# Install bigquery_fdw
pip3 install bigquery-fdwMajor dependencies installed automatically during the installation process:
Usage
We recommend testing the BigQuery client connectivity before trying to use the FDW.
With psql:
CREATE EXTENSION multicorn;
CREATE SERVER bigquery_srv FOREIGN DATA WRAPPER multicorn
OPTIONS (
wrapper 'bigquery_fdw.fdw.ConstantForeignDataWrapper'
);
CREATE FOREIGN TABLE my_bigquery_table (
column1 text,
column2 bigint
) SERVER bigquery_srv
OPTIONS (
fdw_dataset 'my_dataset',
fdw_table 'my_table',
fdw_key '/opt/bigquery_fdw/key.json'
);Options
List of options implemented in CREATE FOREIGN TABLE syntax:
Option |
Default |
Description |
|---|---|---|
fdw_dataset |
BigQuery dataset name |
|
fdw_table |
BigQuery table name |
|
fdw_key |
Path to private Json key (See Key storage recommendations <do cs/key_storage.md> __) |
|
fdw_convert_tz |
Convert BigQuery time zone for dates and timestamps to selected time zone. Example: 'US/Eastern'. |
|
fdw_group |
'false' |
|
fdw_casting |
See Casting. |
|
fdw_verbose |
'false' |
Set to 'true' to output debug information in PostrgeSQL’s logs |
fdw_sql_dialect |
'standard' |
BigQuery SQL dialect. Currently only standard is supported. |
More documentation
1 maintainer
Metadata
1 maintainer
Release files for bigquery-fdw 1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| bigquery-fdw-1.5.tar.gz | 9.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bigquery_fdw-1.5-py2.py3-none-any.whl | Python 2, Python 3 | none | any | Details |
Total release size: 19.1 kB
Release files / bigquery-fdw-1.5.tar.gz
| Download URL | bigquery-fdw-1.5.tar.gz |
|---|---|
| Size | 9.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
85fdba73a6e5d1252340c8e582d29a3f3869202bd52517192679d7be54ae656f
|
|
BLAKE2b-256 checksum How to use checksums |
ee2437b2b46fcfe9d98930c3f9bbc3d5ef5c79008271235094904b389794d9ac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / bigquery_fdw-1.5-py2.py3-none-any.whl
| Download URL | bigquery_fdw-1.5-py2.py3-none-any.whl |
|---|---|
| Size | 9.8 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
2bb885816e6629687ad69e22c26ff89746135dd180a9cb24d82383ce1df396b0
|
|
BLAKE2b-256 checksum How to use checksums |
54bd39123a00a17117bc29b3e5d04bdf027aae76e3ccff61973aa2ea52aa4d7f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
