Skip to main content

Access datastore uri with fsspec

Project description

Filesystem interface to Azure Machine Learning defined URIs

Getting started

This package can be installed using:

pip install azureml-fsspec

Examples

Accepted uri format is Azure Machcine Learning defined datastore uri: azureml://subscriptions/([^/]+)/resourcegroups/([^/]+)/workspaces/([^/]+)/datastores/([^/]+)/paths/([^/]+)

# load parquet file to pandas
import pandas
df = pandas.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to pandas
import pandas
df = pandas.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

# load parquet file to dask
import dask.dataframe as dd
df = dd.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to dask
import dask.dataframe as dd
df = dd.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

Release History

1.3.1 (02-05-2024)

Features Added

  • added support for using mlclient for authorization

1.3.0 (01-17-2024)

Features Added

  • added registry support for fsspec.

1.2.0 (08-14-2023)

Features Added

  • added auth dict support (SP and managedIdentity) for fsspec.
  • added user exception mapping

1.1.0 (06-01-2023)

Features Added

  • upload/download performance improvement

1.0.0 (04-07-2023)

Features Added

  • Added upload/download API
  • bugfix(root behavior)

0.1.0b1

Features Added

  • Initial public preview release of FileSystem for Azure Machine Learning defined datastore uri

0.1.0b2

Features Added

  • Fix release notes

0.1.0b3

Features Added

  • Add mode in open API
  • Add validation
  • Upgrade dataprep to 4.8

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page