pycanvasdata 0.0.3

Module Usage

This module can be used programatically in other scripts and software. An example of creating a canvas_data object is found below.:

from canvas_data_utils.canvas_data_auth import CanvasData

canvas_data_object = CanvasData(
    API_KEY=YOUR_API_KEY ,
    API_SECRET=YOUR_API_SECRET,
    base_folder = YOUR_BASE_DIR,
    data_folder = YOUR_DATA_DIR)

Once you have that object created, you can…

generate mysql table creation statements

mysql_table_creation_statement = canvas_data_object.table_creation_statement('mysql')

generate sqlite table creation statements

sqlite_table_creation_statement = canvas_data_object.table_creation_statement('sqlite')

generate postgres table creation statements

postgres_table_creation_statement = canvas_data_object.table_creation_statement('postgres')

create tables in a database given by a connection string

canvas_data_object.create_tables('sqlite:///{}'.format(db_filename))

fetch the current schema (as json)

schema = canvas_data_object.fetch_schema()

get a list of columns in a table

user_dim_columns = canvas_data_object.get_schema_columns( 'user_dim')

convert an text file download from TSV (Tab Separated Values) to CSV

canvas_data_object.convert_tsv_to_csv(tsv_filepath)

list all the tables in the schema

table_list = canvas_data_object.table_list()

download and convert all files to CSV

canvas_data_object.convert_all_to_csv()

list all downloadable files for a table

file_list = canvas_data_object.list_all_files('user_dim')

Config File

You need to create a config file somewhere. This config file is a typical .INI file. It should look something like the following example.

[config]
API_SECRET = replace_with_api_secret_from_canvas_data
API_KEY = replace_with_api_key_from_canvas_data

base_folder = /path/to/base/folder/for/downloads/
data_folder = %(base_folder)s/test2

connection_string = sqlite:///%(base_folder)s/sample.db

Note: The connection_string configuration follows the connection pattern needed by SQLAlchemy at http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html.

This library supports any database type than SQLAlchemy does.

Command-line Tool

This library includes a command line utility called canvasdata.

Usage

canvasdata [-h] [--config CONFIG] [-t TABLE] [--offline OFFLINE]
                {convert_to_csv,import,create_tables,reset,sql_create_statement,list_files,download,sample_queries,schema}

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG       path to the configuration file
  -t TABLE              specify a specific table
  --offline OFFLINE     run in offline mode