A CLI utility for Snowflake Snowpipe REST APIs
Project description
Snowpipe CLI
Snowpipe CLI provides access to the Snowpipe REST API via the CLI. The script uses the snowflake-ingest python package to call the REST endpoints.
In addition to calling the Snowpipe REST endpoints, you can use Snowpipe CLI to PUT local files in the stage used by the pipe and then ingest. A single invocation of the ingest command handles any combination of already staged files and local files that need to be staged.
There is also a JWT generator that you can use to simply generate a JWT for adhoc use.
Installation
You can install the latest version with pip
pip install snowpipe-cli
Note: If you are installing into a virtual environment, you may need to deactivate and activate again in order for
the snowpipe
command to work correctly from your shell.
Requirements
Snowpipe CLI requires Python 3.8+.
Configuration
Create a config file with the Snowflake credentials and account information. The config file must be YAML like below:
url: phdata.snowflakecomputing.com
account: PHDATA
user: USER
key_fp: SHA256:something_from_snowflake_user
key_file: /path/to/user/private/key.pem
key_password: optional_password
See also example config.
The account
value must not include the region and cloud if present in the Snowflake URL. The url
is the URL of the
Snowflake instance minus the scheme. For example, if your URL is https://tacos.us-east-2.azure.snowflakecomputing.com,
then the account
will be tacos
and the url
will be tacos.us-east-2.azure.snowflakecomputing.com
.
To generate a private key,
see Key Pair Authentication & Key Pair Rotation. When
verifying the fingerprint, record the value into the key_fp
YAML key.
Usage
The commands below are examples of calling the snowpipe
script from your shell after it has been installed with pip.
If you have cloned the repository, you can also invoke the script from its parent directory with ./snowpipe.py
, or you
can run from the src
directory with python -m snowpipe_cli
.
Show the script help, including the subcommands:
snowpipe -h
Show subcommand help:
snowpipe <subcommand> -h
Run a subcommand:
snowpipe <subcommand> <config_file> <other_args>...
Enable debug logging:
snowpipe -d <subcommand> <config_file>
Contributing
Install the prerequisites:
pip install -r requirements.txt
Build the package in the dist
directory
python3 -m build
Upload to pypi
twine upload --repository pypi dist/*
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for snowpipe_cli-0.0.2rc4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 927e963619050d187b7d1180db92ad0b87573d7343e16d0db5d3987f889a2e63 |
|
MD5 | 2437f043a1eb3b60b5f4a440daa9d938 |
|
BLAKE2b-256 | 654e2a3ffb96d3ed105fd8c04f5431ef37a6c95037c78764574647d22fc359e8 |