byoc

Python SDK for BYOC (Bring Your Own Cloud): a provider-agnostic storage layer for building applications on top of cloud storage your users already own.

Part of the BYOC monorepo. Peer implementation of @byoc/core on npm.

Status: v0.2.0. Every adapter is validated against a live server. Not yet on PyPI.

Why

Instead of paying to host every user's files on your own S3 bucket, BYOC lets your application read and write storage the user already owns: their Google Drive, their company's Nextcloud, their own R2 bucket.

FastAPI / Django / Celery
   │
   ├── PostgreSQL  → users, jobs, metadata pointers
   │
   └── byoc        → the actual PDFs, images, audio, model artifacts
           │
           └── the user's own cloud

Use Postgres for your application data. Use BYOC for their files.

Install

pip install byoc-storage

The distribution is named byoc-storage because byoc on PyPI belongs to an unrelated CLI framework. The import name is still byoc:

import byoc

If you also depend on the unrelated byoc CLI framework, do not install both into the same environment: they share the byoc import name and will shadow each other.

Usage

import os
from byoc import AsyncBYOC
from byoc.providers.s3 import S3CompatibleProvider

storage = AsyncBYOC(
    provider=S3CompatibleProvider(
        endpoint="https://<account_id>.r2.cloudflarestorage.com",
        bucket="user-assets",
        region="auto",
        access_key_id=os.environ["R2_ACCESS_KEY_ID"],
        secret_access_key=os.environ["R2_SECRET_ACCESS_KEY"],
    )
)

async with storage:
    await storage.write_text("documents/welcome.md", "# Hello from BYOC!")
    content = await storage.read_text("documents/welcome.md")

Multiple providers and migration

storage = AsyncBYOC(providers=[drive, r2, nextcloud], default_provider_id="s3-compatible")

storage.use_provider("webdav")
await storage.write_bytes("images/banner.png", data)

report = await storage.migrate(
    source="s3-compatible",
    target="webdav",
    paths=["documents/report.pdf"],
    on_progress=lambda p: print(f"{p.current_file} ({p.percentage}%)"),
)

report.files_partial counts files that reached the target but whose source cleanup failed. The transfer is done, so retrying those would re-upload for nothing.

Async only for now. A synchronous facade for Celery, Django, and scripts is planned but not yet implemented; use asyncio.run() in the meantime.

Google Drive

Drive needs a Google Cloud OAuth client. See Google Drive OAuth Setup, then verify your setup with:

.venv/bin/python scripts/validate_gdrive_live.py

Cross-SDK compatibility

This SDK and the TypeScript SDK are peer implementations of the same contract. Both run against the shared conformance vectors in spec/fixtures, so a file written by a Next.js frontend can be read by a FastAPI backend, including client-side encrypted files, whose envelope format is byte-identical across both.

Development

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

pytest              # unit + conformance suite
mypy src            # strict type checking
ruff check .        # lint

Without activating, call the venv binaries directly: .venv/bin/pytest, .venv/bin/mypy src, .venv/bin/ruff check .

License

Apache-2.0