azfs 0.2.0

AzFS is to provide convenient Python read/write functions for Azure Storage Account.

AzFS

AzFS is to provide convenient Python read/write functions for Azure Storage Account.

AzFS can

• list files in blob (also with wildcard *),
• check if file exists,
• read csv as pd.DataFrame, and json as dict from blob,
• write pd.DataFrame as csv, and dict as json to blob,
• and raise lots of exceptions ! (Thank you for your cooperation)

install

$ pip install azfs

usage

import azfs
from azure.identity import DefaultAzureCredential
import pandas as pd

# credential is not required if your environment is on AAD(Azure Active Directory)
azc = azfs.AzFileClient()

# credential is required if your environment is not on AAD
credential = "[your storage account credential]"
# or
credential = DefaultAzureCredential()
azc = azfs.AzFileClient(credential=credential)

# connection_string is also supported
connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net"
azc = azfs.AzFileClient(connection_string=connection_string)

# data paths
csv_path = "https://testazfs.blob.core.windows.net/test_caontainer/test_file.csv"

# read csv as pd.DataFrame
df = azc.read_csv(csv_path, index_col=0)
# or
with azc:
    df = pd.read_csv_az(csv_path, header=None)


# write csv
azc.write_csv(path=csv_path, df=df)
# or
with azc:
    df.to_csv_az(path=csv_path, index=False)

check more details in

types of authorization

Supported authentication types are

• Azure Active Directory (AAD) token credential.
• connection_string, like DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net

types of storage account kind

The table blow shows if AzFS provides read/write functions for the storage.

account kind	Blob	Data Lake	Queue	File	Table
StorageV2	O	O	O	X	X
StorageV1	O	O	O	X	X
BlobStorage	O	-	-	-	-

• O: provides basic functions
• X: not provides
• -: storage type unavailable

dependencies

pandas
azure-identity >= "1.3.1"
azure-storage-blob >= "12.3.0"
azure-storage-file-datalake >= "12.0.0"
azure-storage-queue >= "12.1.1"

references

• azure-sdk-for-python/storage
• filesystem_spec