vinta-django-s3-direct

Upload files from Django forms, the Django admin and DRF straight from the browser to S3, with FilePond as the UI.

The bytes never pass through your Django process. Your web workers stay free, your request timeouts stop mattering, and a 2 GB video costs you one signature instead of an hour of streamed I/O.

from django.db import models
from vinta_s3_direct.fields import S3DirectImageField


class Profile(models.Model):
    avatar = S3DirectImageField(destination="avatars", blank=True)

That is the whole integration. profile.avatar.url, .name, .size, .delete() and every other FieldFile API behave exactly as they would for a server-side upload, because the field stores an ordinary storage-relative name and delegates to your Django storage.

Why not django-s3direct?

This package is a rewrite of the idea behind django-s3direct, which has not kept up with modern Django or modern S3. Three things are different:

It stores a storage name, not a URL. django-s3direct ships its own S3DirectField, a bare Field over a text column that holds the full object URL. Everything downstream — signed URLs, moving buckets, switching to a CDN, deleting a file — becomes string surgery, and the storage never learns the file exists. Here the fields are real FileField/ImageField subclasses backed by your configured Storage, so none of that is your problem.

The server decides, and signs, everything. In django-s3direct the widget is a TextInput carrying the object URL, and the form field stores whatever comes back. A user who can upload to any destination can therefore point any field at any key in the bucket. Here the browser posts an HMAC-signed token, and the form field verifies both the signature and the destination before writing anything.

It signs uploads the way S3 expects. django-s3direct reimplements AWS Signature V4 in JavaScript and calls back to Django to sign each chunk. This package uses presigned POST for small files — which lets S3 itself enforce the content type and an exact byte count — and presigned multipart part URLs for large ones.

Requirements

• Python 3.10+
• Django 5.2+
• An S3-compatible bucket (AWS S3, MinIO, Cloudflare R2, Ceph, …)

Install

pip install vinta-django-s3-direct

Optional extras: [drf] for the serializer field, [storages] to pull in django-storages.

Setup

1. Add the app and the URLs.

INSTALLED_APPS = [
    ...,
    "vinta_s3_direct",
]

urlpatterns = [
    ...,
    path("s3direct/", include("vinta_s3_direct.urls")),
]

2. Configure your storage as you normally would with django-storages:

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3boto3.S3Boto3Storage",
        "OPTIONS": {"bucket_name": "my-media-bucket", "region_name": "us-east-1"},
    },
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
}

3. Declare your destinations.

VINTA_S3_DIRECT = {
    "DESTINATIONS": {
        "avatars": {
            "key_prefix": "uploads/avatars",
            "auth": "vinta_s3_direct.auth.is_authenticated",
            "allowed_content_types": ["image/png", "image/jpeg", "image/webp"],
            "max_size": 5 * 1024 * 1024,
        },
        "recordings": {
            "key_prefix": "uploads/recordings",
            "auth": "myapp.policies.can_upload_recordings",
            "allowed_content_types": ["video/mp4"],
            "max_size": 5 * 1024 * 1024 * 1024,
            "multipart_threshold": 16 * 1024 * 1024,
        },
    },
}

4. Configure CORS on the bucket so the browser may PUT/POST to it and read the ETag header back (multipart needs that last part):

[
  {
    "AllowedOrigins": ["https://your-site.example"],
    "AllowedMethods": ["POST", "PUT"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Destination options

A destination is a named upload policy. Every request names one, and the server takes every security-relevant decision from it — never from the browser.

A "DEFAULTS" key applies to every destination:

VINTA_S3_DIRECT = {
    "DEFAULTS": {"server_side_encryption": "AES256", "token_max_age": 1800},
    "DESTINATIONS": {...},
}

Writing an auth callable

It receives the whole HttpRequest, so it can look at the user, the session, or a tenant resolved by middleware:

def can_upload_recordings(request):
    return request.user.is_authenticated and request.user.has_perm("media.add_recording")

Four ready-made ones live in vinta_s3_direct.auth: allow_any, deny_all, is_authenticated (the default) and is_staff.

Django admin

Nothing to do:

@admin.register(Profile)
class ProfileAdmin(admin.ModelAdmin):
    pass

That really is the whole integration, and it holds for every ModelAdmin, every custom AdminSite, and inline formsets. It is worth knowing why, because this is the part that most often breaks quietly: ModelAdmin.formfield_for_dbfield walks db_field.__class__.mro() and maps FileField/ImageField to AdminFileWidget. Our fields subclass both, so the admin hands them a plain file input — which cannot work here, since the value comes back in POST data rather than in request.FILES. The model field simply discards that widget.

The tidier-looking alternative — registering our fields in the admin's FORMFIELD_FOR_DBFIELD_DEFAULTS from AppConfig.ready() — is subtly broken: a ModelAdmin snapshots that table when it is constructed, and django.contrib.admin's own ready() may run autodiscover() before this package's app is ready. Handling it in the field sidesteps app-loading order altogether.

Inline formsets work too — the JavaScript listens for Django's formset:added event and initialises new rows.

Plain forms

from vinta_s3_direct.forms import S3DirectFormField


class UploadForm(forms.Form):
    document = S3DirectFormField(destination="documents")

Render {{ form.media }} in your <head>, and the form normally in the body. The form does not need enctype="multipart/form-data" — only a signed token is posted.

One nicety over a plain FileField: if the form fails validation on some other field, the upload survives the re-render, because the posted value is a token rather than the file itself.

Django REST Framework

from rest_framework import serializers
from vinta_s3_direct.drf import S3DirectSerializerMixin


class ProfileSerializer(S3DirectSerializerMixin, serializers.ModelSerializer):
    class Meta:
        model = Profile
        fields = ["id", "avatar"]

The field is asymmetric on purpose: clients write an upload token and read back a ready-to-use URL. They never see, and never choose, the key.

// PATCH /api/profiles/1/
{"avatar": "eyJkZXN0aW5hdGlvbiI6ImF2YXRhcnMi...:1t8Xk2:9c..."}

// 200 OK
{"id": 1, "avatar": "https://my-media-bucket.s3.amazonaws.com/uploads/avatars/photo_a1b2.png?X-Amz-..."}

Use it directly if you prefer:

from vinta_s3_direct.drf import S3DirectSerializerField

avatar = S3DirectSerializerField(destination="avatars", expire=900)

The upload flow

For a non-Django client — a React SPA, a mobile app — talk to the endpoints directly. All five are POST, JSON in and JSON out, and CSRF-protected.

POST /s3direct/begin/
  {"destination": "avatars", "filename": "me.png",
   "content_type": "image/png", "size": 51200}
→ {"transport": "post", "name": "uploads/avatars/me_a1b2.png",
   "session": "...", "url": "https://...", "fields": {...}}

Then POST the file to url with fields as form data, and finish:

POST /s3direct/complete/   {"session": "..."}
→ {"token": "...", "name": "uploads/avatars/me_a1b2.png", "size": 51200}

Post token as the field's value in your form or API request.

For a large file, begin answers {"transport": "multipart", "part_size": ..., "part_count": ...} instead. Ask POST /s3direct/sign-parts/ for presigned part URLs, PUT each slice, collect the ETag headers, and pass them to complete:

POST /s3direct/sign-parts/ {"session": "...", "part_numbers": [1, 2, 3]}
POST /s3direct/complete/   {"session": "...",
                            "parts": [{"part_number": 1, "etag": "\"abc\""}, ...]}

POST /s3direct/abort/ discards an in-flight upload and POST /s3direct/revert/ deletes a completed one that was never attached to a model.

Security model

• The client never picks the key. begin generates it, and it is sealed into the signed session; the part and complete endpoints read it from there.
• Limits are enforced by S3, not by the browser. A presigned POST pins the content type and an exact content-length-range, so a patched client cannot upload something larger or of a different type than was authorised.
• Multipart is checked afterwards. S3 will not enforce a size limit on a multipart upload, so complete issues a HeadObject, compares the real size against the destination, and deletes the object if it does not fit.
• Tokens are scoped and short-lived. A token records its destination, and the form field refuses one minted for a different destination — so a token from a permissive destination cannot be replayed into a stricter field.
• Every endpoint re-runs auth. Not just begin.

Signing uses Django's SECRET_KEY via django.core.signing, so SECRET_KEY_FALLBACKS works during a key rotation.

MinIO and other private endpoints

When your app reaches S3 at an address the browser cannot resolve — a MinIO container on a private Docker network, say — set both endpoints:

"uploads": {
    "endpoint_url": "http://minio:9000",           # what Django uses
    "public_endpoint_url": "http://localhost:9000",  # what the browser uses
}

A presigned SigV4 URL commits to its host, so the browser-facing URL is signed by a separate client configured with the public endpoint. Rewriting the host after signing — which is what you may have had to do with django-s3direct — would invalidate the signature.

Serving files back

This package only handles the upload. Reading is your storage's job: instance.avatar.url returns whatever your Storage returns — a presigned GET if AWS_QUERYSTRING_AUTH is on, a CDN URL if you set AWS_S3_CUSTOM_DOMAIN.

Development

uv sync --all-extras
uv run pytest
uv run pytest --cov
uv run ruff check . && uv run ruff format --check .
uv run mypy
uv run tox

The suite runs against moto. The end-to-end tests go further and drive a real S3-compatible HTTP server, so the presigned-POST body and the multipart ETag round trip are exercised over the wire rather than mocked.

Refresh the vendored FilePond assets with ./scripts/vendor_filepond.sh.

Supported combinations

Blank cells are combinations Django itself does not support. CI runs every filled cell, and each tox environment asserts the Django version it actually imported — so a matrix entry cannot silently test the wrong one.

Licence

MIT. FilePond and its plugins are vendored under src/vinta_s3_direct/static/vinta_s3_direct/vendor/ and are also MIT-licensed (© PQINA).