A small example package
Project description
Installation
pip install manonaid_helpers
See PyPi for package index.
Requirements
Azure Storage connection string has to be set as environment variable AZURE_STORAGE_CONNECTION_STRING
or
the seperate environment variables AZURE_STORAGE_KEY
and AZURE_STORAGE_NAME
which will be used to create the connection string.
Set variables like below: For Windows PowerShell $env:AZURE_STORAGE_CONNECTION_STRING = 'blahblahblah....'
For Command Prompt set AZURE_STORAGE_CONNECTION_STRING=blahblahblah....
Usage
Download
1. Fetch as pd.DataFrame
We can use the returnAsDataFrameDict()
method to fetch one or multiple blobs as DataFrames
While fetching multiple blobs the return object is a dictonary of DataFrames with the blob name(with the extension) as the key.
If type_correct_date_time_cols
is set as True
:
For all blobs that contain columns with names containing "date" or "time" are converted to datetime with format="%Y-%m-%dT%H:%M:%S"
delimitter
is set to','
by default
from manonaid_helpers.download import Download
df = Download(
source="cyi5",
extension=".csv",
folder="pack_test/"
).returnAsDataFrameDict()
2. Download a specific folder from a container
We can download a folder by setting the folder
argument.
from manonaid_helpers.download import Download
Download(
source='cyi5',
folder="pack_test/",
extension='.csv',
).returnAsDataFrameDict()
3. Download a given list of files
We can give a list of files to download with the list_files
argument.
from manonaid_helpers.download import Download
Download(
source='cyi5',
folder="pack_test/",
extension='.csv',
list_files=["20230301_ais_accuracy.csv"],
).returnAsDataFrameDict()
List blobs
With the Utils.list_blobs
method we can do advanced listing of blobs in a container or specific folder in a container.
We have several argument we can use to define our scope of information:
name_starts_with
: This can be used to filter files with certain prefix, or to select certain folders:name_starts_with=folder1/subfolder/lastfolder/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for manonaid_helpers-0.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5559b4e22394ee20e132040d3cd34800b5d78ec4dfe811185b822f03bf55f495 |
|
MD5 | cbb950e37accf5c2517d5b7a887c6a18 |
|
BLAKE2b-256 | 29fad544289dfbb2bbce66ca2c82f39a54a87f08950d057631e7a732d507a6ca |