manonaid-helpers 0.0.4

A small example package

Installation

pip install manonaid_helpers

See PyPi for package index.

Requirements

Azure Storage connection string has to be set as environment variable AZURE_STORAGE_CONNECTION_STRING or the seperate environment variables AZURE_STORAGE_KEY and AZURE_STORAGE_NAME which will be used to create the connection string.

Usage

Download

1. Using the standard environment variables

Azure-batch-load automatically checks for environment variables: AZURE_STORAGE_CONNECTION_STRING, AZURE_STORAGE_KEYand AZURE_STORAGE_ACCOUNT. So if the connection_string or storage_key + storage_account are set as environment variables, we can leave the argument connection_string, account_key and account_name empty:

from manonaid_helpers.download import Download

df = Download(
    destination="../test_data", 
    source="cyi5", 
    extension=".csv", 
    folder="pack_test/"
).returnAsDataFrameDict()

2. Download a specific folder from a container

We can download a folder by setting the folder argument. This works both for single and batch.

from manonaid_helpers.download import Download

Download(
   destination="../test_data",
   source='cyi5',
   folder="pack_test/",
   extension='.csv',
).download()

3. Download a given list of files

We can give a list of files to download with the list_files argument.

from manonaid_helpers.download import Download

Download(
   destination="../test_data",
   source='cyi5',
   folder="pack_test/",
   extension='.csv',
   list_files=["20230301_ais_accuracy.csv"],
).download()