datazone 0.0.9

Datazone Client Package

Datazone Client Library 👋

Installation

You can install package via pip

pip install datazone

Configuration & Login

Run the following command in your terminal to create a new profile.

datazone profile create

When prompted, enter the following information:

Profile Name: <PROFILE_NAME>
Host [app.datazone.co]: <DATAZONE_ADDRESS>
Email: <EMAIL>
Password: <PASSWORD>
Repeat for confirmation: <PASSWORD_CONFIRM>

Then you can run following command to log in. (At the first profile creation, it will log in automatically)

datazone auth login

It will keep your tokens until refresh token expired. (Default 30min). Then you can re-run and login again.

Create first repository

datazone repository create hello-world
cd hello-world

It creates a new folder in your current directory. Thıs folder has two file initially. config.yaml and first_pipeline.py. After you modify your script you can deploy project via following command.

datazone repository deploy

After the deployment is complete, you can check your repository status via following command.

datazone repository summary

If repository has deployed, you can run your first execution with following command.

datazone execution run <pipeline_id>

Commands

Also you can check all command with help sub command. datazone auth --help or datazone repository summary --help

# Auth
datazone auth login

# Profile
datazone profile list
datazone profile create
datazone profile delete <profile_name>
datazone profile setdefault <profile_name>

# Repository
datazone repository list
datazone repository create
datazone repository deploy
datazone repository summary <file_name>
datazone repository clone <repository_id>
datazone repository pull

# Dataset
datazone dataset list
datazone dataset show <dataset_id> [<branch_name>] [<size>] [<transaction_id>]
datazone dataset transactions <dataset_id>

# Source
datazone source create
datazone source list
datazone source update <source_id>
datazone source delete <source_id>

# Extract
datazone extract create
datazone extract list
datazone extract update <extract_id>
datazone extract delete <extract_id>
datazone extract execute <extract_id>

# Schedule
datazone schedule create
datazone schedule list
datazone schedule delete <source_id>

# Execution
datazone execution run [--extract-id] [--pipeline-id] [<execution_type>] [<transform_selection>]
datazone execution list [--extract-id] [--pipeline-id]
datazone execution log <execution_id>

# Pipeline
datazone pipeline create

# Common
datazone version
datazone info