wj-google 0.1.1

No project description provided

wj_google

Getting started

Poetry

To make more easy the instalation, is recomended use Poetry, follow the instructions, and then use the command:

poetry install

To activate the vierual enviroment use:

poetry shell

Virtual Enviroment

Or you can use your own virtual enviroment, and install the requirements:

python -m venv <env_name>
<env_name>/scripts/activate
pip install -r requirements.txt

Connection to GCP

Is important consult the documentation of GCP, is required obtain the json credentials with the permission to use the resource that is going to use. ***

Supported services

The currently supported services are:

• Google Drive
• Buckets
• BigQuery (just sql consults)***

Google Drive

The GCP account must have Google Drive API enabled, and make sure that you have a OAuth 2.0 application created, in Google Cloud console, you can see the documentation.

The following methods are currently supported:

• Upload files
• Download files
• Create directory You must The package must be instantiated as follows:

from wj_google.google_api.google_api import GoogleApi

googleApi = GoogleApi(
    client_secret_file=CLIENT_SECRET_FILE, -> Json file must be configured in GCP
    api_name=API_NAME, -> drive in this case
    api_version=API_VERSION, -> Current version, 3 in this case
    scopes=GOOGLE_DRIVE_SCOPES, -> Scope of drive see doc
)

There are diferent scopes, as you can see in the documentation.

Google Drive

The GCP account must have Google Drive API enabled, and make sure that you have a OAuth 2.0 application created, in Google Cloud console, you can see the documentation.

The following methods are currently supported:

• Upload files
• Download files
• Create directory

To use the library, it is necessary to prepare the credentials, and then instantiate the service that will be used.

from wj_google.google_api.google_api import GoogleApi

googleApi = GoogleApi(
    client_secret_file=CLIENT_SECRET_FILE, -> Json file must be configured in GCP
    api_name=API_NAME, -> drive in this case
    api_version=API_VERSION, -> Current version, 3 in this case
    scopes=GOOGLE_DRIVE_SCOPES, -> Scope of drive see doc
)

There are diferent scopes, as you can see in the documentation.

Uploading a file:

googleApi.upload_files_to_drive(
    folder_id, -> Id from the drive of the directory 
    file_paths, -> Path of yur file
    mime_types -> Type of file
)

Downloading a file:

googleApi.download_files_drive(
    file_ids -> Google Drive id file, you can find it in the uri
    file_names -> Name given in your local
)

Creating a directory:

googleApi.create_directory_drive(
    directory_name
)

Google storage buckets

This resource can be used with the credentials of a service user, you see de documentation.

The following methods are currently supported:

• Upload files
• Download files
• Create directory

As in the previous case, credentials are required. After which the resource is instantiated.

from wj_google.google_cloud.cloud_connector import GoogleStorage
googleStorageClient = GoogleStorage(
    service -> STORAGE in this case and for now***
    credentials
)

Uploading a file:

googleStorageClient.upload_file(
    bucket_name -> Destination bucket 
    source_file_name -> Path of the file to upload
    destination_file_name Name of the file in bucket
)

Downloading a file:

googleStorageClient.download_file(
    bucket_name -> bucket file
    file_name -> name and path in the bucket
    destination_file -> name and path in your local of the file
)

Creating a bucket:

googleStorageClient.create_bucket(
    bucket_name,
    storage_class, -> Optional, to define the type of bucket to use
    location, -> Optional, to define in which region the bucket is hosted 
)

Listing bucket's files:

googleStorageClient.list_files(bucket_name)

Google BigQuery

This resource can be used with the credentials of a service user, you see de documentation.

With the credentials are required. After which the resource is instantiated, in this case just execute the query.

from google_cloud.cloud_connector import GoogleBigQuery

bigquery = GoogleBigQuery(
    service=BIGQUERY,
    credentials=GOOGLE_BIGQUEY_CREDENTIALS,
    scopes=scopes -> To use services like Google Drive
)

Do a consult:

query = 'YOUR_SQL_QUERY'
sql_job = bigquery.sql_query(query=query)

You can iterate over the sql_job:

for item in sql_job:
    print(item.name)

Add label to bigquery table:

labels = {
    "type": "social_media",
    "category": "cloud",
}
bigquery = GoogleBigQuery(service=BIGQUERY, credentials=GOOGLE_BIGQUEY_CREDENTIALS)
bigquery.add_table_labels(
    labels=labels, dataset_name=dataset_name, table_name=table_name
)

Delete label to bigquery table:

delete_labels_in_table(dataset_name, table_name)

Export queries to excel(Depreciated!):

You can use multiple queries and add it in a excel document, every query will be converted in a worksheet.

query1 = """
SELECT * FROM `test.tw` LIMIT 10
"""
query2 = """
SELECT * FROM `reports.reports` LIMIT 10
"""
queries = [query1, query2]
export_queries_to_excel(queries=queries)