faas-cli 1.0.4

Command Line Interface (CLI) to work with FaaS API

FaaS-CLI

Command Line Interface (CLI) for FaaS API https://api.sonra.io/documentation/

Usage

Help

To get a general help with usage examples use

faascli --help

To get a help for a specific command use

faascli source_data --help
faascli source_schema --help
faascli target_connection --help
faascli conversion --help
faascli webhook --help

Login

To login into FaaS using credentials provided by Sonra use

# for interactive email/password
faascli login

# for predefined email
faascli login --username [e-mail]

# for predefined email/password
faascli login --username [e-mail] --password "mypassword"

Source data/schema

Source Data is the data that you feed to FaaS for conversion. XML and JSON are currently supported.

The Source Schema is metadata for your Source Sata. It defines data types, relationships, constraints etc. XML schema (XSD) is currently the only supported Source Schema.

Once authenticated, you can create Source Data/Schema by the following commands:

faascli source_data create file --name <name> --type <xml|json> --file <path_to_file>

faascli source_schema create file --name <name> --file <path_to_file>

Examples:

xml data

# xml source data from local file system
faascli source_data create file --name customers_data1 --type xml --file ~/data/CustomersOrders.xml

# for multiple files under a single name use --file multiple times
faascli source_data create file --name customers_data1 --type xml --file ~/data/CustomersOrders_11112019.xml --file ~/data/CustomersOrders_12112019.xml

json data

faascli source_data create file --name weather_11112019 --type json --file ~/data/weather.json

xsd schema

faascli source_schema create file --name customers_schema --file ~/data/customersorders.xsd

Target connections

Target Connection is a resource containing connection details for thedata storage, where you want to save the processed data.

Note, that Target Connection is an optional resource. If you don't create Target Connection, you still can process data, and the output will be saved in our cloud and will be available for download.

Different connections require different input parameters, for more details about creating connections use the following commands to get a help/usage messages:

# Amazon S3:
faascli target_connection create s3 --help

# MySQL:
faascli target_connection create mysql --help

# PostGreSQL:
faascli target_connection create postgresql --help

# Snowflake:
faascli target_connection create snowflake --help

# Microsoft SQL Server:
faascli target_connection create sqlserver --help

# Oracle DB:
faascli target_connection create oracle --help

# Amazon Redshift:
faascli target_connection create redshift --help

Conversions

Conversion is a task of processing of Source Data, using a Source Schema (optionally), and saving the result to a Target Connection

To create a conversion use the following commands:

faascli conversion create --name <unique_name> --data <source_data_name> [--schema <schema_name>] --target <target_connection_name>

Examples

# to process XML without XSD to downloadable results
faascli conversion create --name testconv --data customers_data1 --target download_link

# to process XML with XSD to downloadable results
faascli conversion create --name testconv --data customers_data1 --schema customers_schema --target download_link

If download_link is selected, results will be available right after conversion is completed. You can track conversion's status by calling:

faascli conversion get --name testconv

{
    "create_date": "2019-11-11T14:14:57.007+0000",
    "credit_usage": null,
    "data_source_connection": {
        "create_date": "2019-11-06T10:05:00.806+0000",
        "name": "testurl2",
        "path": "https://clinicaltrials.gov/ct2/download_studies?lupd_s=11%2F11%2F2018&lupd_e=11%2F12%2F2018",
        "size": 0,
        "source_type": "http",
        "type": "xml"
    },
    "diagram_link": null,
    "download_link": null,
    "full_status": "SCHEDULED",
    "mapping_link": null,
    "name": "testconv",
    "schema_source_connection": null,
    "status": "S",
    "target_connection": null
}

The field full_status indicates the status of the conversion. Once the status is COMPLETED, the link to results will be available in the download_link field of the response. Each conversion also generates mapping or lineage (mapping_link) and ER Diagram (diagram_link).

{
    "create_date": "2019-11-11T14:14:57.007+0000",
    "credit_usage": 6,
    "data_source_connection": {
        "create_date": "2019-11-06T10:05:00.806+0000",
        "name": "testurl2",
        "path": "https://clinicaltrials.gov/ct2/download_studies?lupd_s=11%2F11%2F2018&lupd_e=11%2F12%2F2018",
        "size": 0,
        "source_type": "http",
        "type": "xml"
    },
    "diagram_link": "https://sonra-api.s3-eu-west-1.amazonaws.com/o...",
    "download_link": "https://sonra-api.s3.amazonaws.com/o...",
    "full_status": "COMPLETED",
    "mapping_link": "https://sonra-api.s3-eu-west-1.amazonaws.com/o...",
    "name": "testconv",
    "schema_source_connection": null,
    "status": "C",
    "target_connection": null
}