boxdrive 0.0.3

S3-compatible API with Abstract Object Store in Python (FastAPI).

BoxDrive

S3-compatible API with Abstract Object Store in Python (FastAPI). Work in progress.

Built-in stores

• InMemoryStore
• GitlabStore (with some limitations)

Quick Start

0. install boxdrive from pypi using your favorite package manager:

uv add boxdrive

1. create main.py:

from boxdrive import create_app
from boxdrive.stores import InMemoryStore

store = InMemoryStore()
app = create_app(store)

2. start API (in dev mode):

fastapi dev main.py

Swagger will start on http://localhost:8000/docs by default

API Endpoints

The API provides S3-compatible endpoints:

• GET / - List buckets
• PUT /{bucket} - Create a bucket
• DELETE /{bucket} - Delete a bucket
• GET /{bucket} - List objects in a bucket
• GET /{bucket}/{key} - Get an object
• PUT /{bucket}/{key} - Put an object
• DELETE /{bucket}/{key} - Delete an object

Creating Custom Object Stores

To create a custom object store implementation, inherit from ObjectStore:

from boxdrive import (
    BucketInfo,
    BucketName,
    ContentType,
    Key,
    ListObjectsInfo,
    ListObjectsV2Info,
    MaxKeys,
    Object,
    ObjectInfo,
    ObjectStore,
)

class MyCustomStore(ObjectStore):
    async def list_buckets(self) -> list[BucketInfo]: ...
    async def create_bucket(self, bucket_name: BucketName) -> None: ...
    async def delete_bucket(self, bucket_name: BucketName) -> None: ...
    async def get_object(self, bucket_name: BucketName, key: Key) -> Object: ...
    async def put_object(
        self, bucket_name: BucketName, key: Key, data: bytes, content_type: ContentType | None = None
    ) -> ObjectInfo: ...
    async def delete_object(self, bucket_name: BucketName, key: Key) -> None: ...
    async def head_object(self, bucket_name: BucketName, key: Key) -> ObjectInfo: ...
    async def list_objects(
        self,
        bucket_name: BucketName,
        *,
        prefix: Key | None = None,
        delimiter: str | None = None,
        max_keys: MaxKeys = 1000,
        marker: Key | None = None,
    ) -> ListObjectsInfo: ...
    async def list_objects_v2(
        self,
        bucket_name: BucketName,
        *,
        continuation_token: Key | None = None,
        delimiter: str | None = None,
        encoding_type: str | None = None,
        max_keys: MaxKeys = 1000,
        prefix: Key | None = None,
        start_after: Key | None = None,
    ) -> ListObjectsV2Info: ...

Use exceptions from boxdrive.exceptions (e.g., NoSuchBucket, NoSuchKey) in your custom store implementation.

Development

Start API

0. start monitoring pipeline:

docker compose -f docker-compose-monitoring.yaml up --detach --wait

1. start api:

uv run fastapi dev examples/inmemory.py

Run Tests

unit

uv run pytest tests/unit

e2e

API should run in the background.

export S3_ENDPOINT_URL=http://127.0.0.1:8000
uv run run pytest tests/e2e

third_party/s3-tests

API should run in the background.

cd tests/third_party/s3-tests
export S3TEST_CONF=s3tests.conf
uv run tox -- s3tests_boto3/functional/test_s3.py -m boxdrive

See tests/third_party/s3-tests/boxdrive.md for additional info.

Check Code Quality

uv run ruff format .
uv run ruff check . --fix
uv run mypy .

License

Apache 2.0 - see LICENSE file for details.