storage-actions

Unified object storage helpers for Alibaba OSS and S3-compatible backends such as RustFS, MinIO, and generic S3 services.

Features

• One StorageClient API for Alibaba OSS and S3-compatible storage.
• Explicit StorageConfig / StorageProvider configuration with mapping, object, and environment factory helpers.
• Browser-facing public URL and presigned URL generation with separate internal and public endpoints.
• URL transfer helpers for common "fetch remote file then upload" and "promote by public URL" workflows.
• Aliyun provider extras for browser upload credentials that also support S3-compatible presigned uploads.
• Metadata-based ACL compatibility mode for S3-compatible storage that does not support object-level ACL semantics.
• Runtime token support for temporary S3 session credentials and Alibaba Cloud security tokens.
• Legacy aliyun_oss_x.Bucket-style adapter for gradual migrations.

Install

pip install storage-actions

The current release is storage-actions==0.2.6.

During local development:

uv sync --dev
uv run pytest -v
uv build

For Alibaba Function Compute Python runtimes, 0.2.2 also avoids eager STS imports during module initialization and forces the Alibaba OSS v2 SDK to use PyCryptodome's ctypes fallback, so a shared FC layer does not crash on partial cffi runtimes.

For offline delivery, build the wheel ahead of time and install it from the artifact bundle:

uv build
pip install dist/storage_actions-*.whl

Basic Usage

from storage_actions import StorageClient, StorageConfig, StorageProvider, client_from_env

config = StorageConfig(
    provider=StorageProvider.RUSTFS,
    bucket="private-bucket",
    endpoint="http://rustfs:9000",
    public_endpoint="https://storage.example.internal",
    region="us-east-1",
    addressing_style="path",
    access_key="access-key",
    secret_key="secret-key",
    acl_compat_mode="metadata-gateway",
)

storage = StorageClient(config)
storage.put_object("reports/hello.txt", b"hello")
download_url = storage.get("reports/hello.txt", sign=True)
public_url = storage.get("reports/hello.txt", sign=False)

env_storage = client_from_env(bucket="private-bucket")

Environment Variables

StorageConfig.from_env() reads these generic variables:

• OBJECT_STORAGE_PROVIDER: oss, rustfs, minio, or s3
• OBJECT_STORAGE_BUCKET
• OBJECT_STORAGE_ENDPOINT
• OBJECT_STORAGE_PUBLIC_ENDPOINT
• OBJECT_STORAGE_CDN_PUBLIC_ENDPOINT
• OBJECT_STORAGE_REGION
• OBJECT_STORAGE_ADDRESSING_STYLE: path or virtual
• OBJECT_STORAGE_ACCESS_KEY
• OBJECT_STORAGE_SECRET_KEY
• OBJECT_STORAGE_SESSION_TOKEN
• OBJECT_STORAGE_ACL_COMPAT_MODE

Temporary S3-compatible credentials should populate OBJECT_STORAGE_SESSION_TOKEN.

Temporary Alibaba Cloud credentials can also provide ALIBABA_CLOUD_SECURITY_TOKEN; settings-like objects may provide the same names and can be loaded with config_from_object() or client_from_object(). The runtime backends now forward those tokens to boto3 as aws_session_token and to Alibaba OSS as security_token.

URL Helpers

Use storage_actions.urls or the package-level exports to build public URLs, strip presign query parameters, and resolve path-style public URLs:

from storage_actions import resolve_bucket_key_from_url, strip_url_query

clean_url = strip_url_query(signed_url)
bucket_key = resolve_bucket_key_from_url("https://storage.example.internal/private-bucket/reports/hello.txt")

Encoded object keys round-trip safely through build_object_url(), extract_key_from_url(), and resolve_bucket_key_from_url().

For Alibaba OSS private bucket public domains, both storage.example.internal and https://storage.example.internal are accepted. Bare hostnames are normalized to https://... for browser-facing URLs.

StorageClient.get_object_meta() returns a normalized ObjectMeta object for both Alibaba OSS and S3-compatible head_object() responses. Use meta.metadata, meta.content_length, meta.storage_class, and meta.etag instead of depending on provider-specific response shapes.

Transfer Helpers

Use upload_from_url() to stream a remote file into a temporary file and upload it with the source Content-Type, or ensure_public_read_from_url() to promote an object to public-read when the URL resolves back to the current client bucket.

from storage_actions import ensure_public_read_from_url, upload_from_url

upload_from_url(storage, "reports/demo.txt", "https://example.com/demo.txt")
ensure_public_read_from_url(storage, "https://storage.example.internal/private-bucket/reports/demo.txt")

ACL Compatibility

For S3-compatible backends in metadata-gateway mode, put_object_acl("public-read") writes vv-acl=public-read into object metadata. Setting the ACL back to private or default removes the marker.

This mode is designed for deployments where the gateway decides whether anonymous GET/HEAD requests may read an object by checking metadata. Signed URLs and authenticated requests should continue to go directly to the storage backend.

The same compatibility mode is applied by generate_sts_upload_credential() for S3-compatible uploads: when object_acl="public-read" it emits x-amz-meta-vv-acl instead of a raw ACL header.

Aliyun Provider Extras

storage_actions.providers.aliyun.generate_sts_upload_credential() keeps the legacy browser-upload contract in a provider-specific module:

from storage_actions.providers.aliyun import generate_sts_upload_credential

credential = generate_sts_upload_credential(
    config=config,
    role_arn="acs:ram::1234567890123456:role/upload",
    role_session_name="upload-demo",
    allowed_buckets={"private-bucket", "cdn-bucket"},
    bucket="cdn-bucket",
    object_key="user-upload/demo.png",
    content_type="image/png",
    object_acl="public-read",
)

For s3, rustfs, and minio providers, it returns a presigned PUT upload payload using the public endpoint when available. Bucket overrides are denied by default; pass allowed_buckets explicitly when a caller may choose among a bounded set of buckets. For Alibaba OSS, it returns STS credentials plus provider, endpoint, public_endpoint, region, and addressing_style so backend, worker, and cloud-function migrations can keep their existing call sites stable. If the STS SDK call fails, the helper returns None to preserve the legacy migration contract.

Legacy Adapter

Some older code may still instantiate aliyun_oss_x.Bucket. storage_actions.compat.OssBucketAdapter implements the subset commonly used during migrations:

• get_object
• get_object_to_file
• put_object
• put_object_from_file
• put_object_acl
• sign_url
• copy_object
• delete_object
• head_object
• object_exists
• list_objects_v2
• get_bucket_info

The adapter also accepts common aliyun_oss_x keyword aliases such as key= and filename=.