Quick Start

Install base dependencies:

uv sync

The base install supports local filesystem configuration without installing cloud backend SDKs. Install provider extras only for the cloud protocols you need:

uv sync --extra s3
uv sync --extra gcs
uv sync --extra azure
uv sync --all-extras

Run local checks:

make lint
make test

Format code:

make format

Development

Packages in this monorepo use uv, ruff, pytest, and hatchling. Tox is not used.

Packages target Python 3.12 or newer. This package contributes to the shared unlib namespace:

from pydantic import TypeAdapter
from unlibcore.storage_provider import StorageProvider

storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "local",
        "root": "./data",
    }
)

Supported Providers

unlibcore-storage-provider keeps provider configuration models in the base package, while cloud filesystem backends are optional extras:

Install a single backend extra when only one cloud provider is needed:

pip install "unlibcore-storage-provider[s3]"
pip install "unlibcore-storage-provider[gcs]"
pip install "unlibcore-storage-provider[azure]"

Install all cloud backends when a deployment needs multiple protocols:

pip install "unlibcore-storage-provider[all]"

Example S3 configuration:

storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "s3",
        "bucket": "media-library",
        "prefix": "originals",
        "region_name": "ap-southeast-2",
    }
)

Example GCS configuration:

storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "gcs",
        "bucket": "media-library",
        "project": "google-project",
    }
)

Example Azure Blob configuration:

storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "az",
        "container": "application-data",
        "account_name": "productionstorage",
    }
)

CI/CD

Package CI runs with uv run pytest through the reusable .github/workflows/package-ci.yaml workflow. This package's PR workflow is .github/workflows/package-unlibcore-storage-provider-ci.yaml, which only runs when files under packages/unlibcore-storage-provider/ or its workflow files change.

Package publishing is handled by .github/workflows/package-cd.yaml. Run it manually and provide unlibcore-storage-provider as the package name, or publish a GitHub Release tagged packages/unlibcore-storage-provider@X.Y.Z.