Metadata-Version: 2.5
Name: vinta-django-s3-direct
Version: 0.1.0
Summary: Direct-to-S3 uploads for Django forms, the admin and DRF, powered by FilePond.
Project-URL: Homepage, https://github.com/vintasoftware/vinta-django-s3-direct
Project-URL: Repository, https://github.com/vintasoftware/vinta-django-s3-direct
Project-URL: Issues, https://github.com/vintasoftware/vinta-django-s3-direct/issues
Project-URL: Changelog, https://github.com/vintasoftware/vinta-django-s3-direct/blob/main/CHANGELOG.md
Author-email: Vinta Software <contact@vinta.com.br>
License-Expression: MIT
License-File: LICENSE
Keywords: direct-upload,django,filepond,presigned,s3,upload
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Web Environment
Classifier: Framework :: Django
Classifier: Framework :: Django :: 5.2
Classifier: Framework :: Django :: 6.0
Classifier: Framework :: Django :: 6.1
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: boto3>=1.34
Requires-Dist: django>=5.2
Provides-Extra: drf
Requires-Dist: djangorestframework>=3.15; extra == 'drf'
Provides-Extra: storages
Requires-Dist: django-storages[s3]>=1.14; extra == 'storages'
Description-Content-Type: text/markdown

# vinta-django-s3-direct

Upload files from Django forms, the Django admin and DRF **straight from the
browser to S3**, with [FilePond](https://pqina.nl/filepond/) as the UI.

The bytes never pass through your Django process. Your web workers stay free,
your request timeouts stop mattering, and a 2 GB video costs you one signature
instead of an hour of streamed I/O.

```python
from django.db import models
from vinta_s3_direct.fields import S3DirectImageField


class Profile(models.Model):
    avatar = S3DirectImageField(destination="avatars", blank=True)
```

That is the whole integration. `profile.avatar.url`, `.name`, `.size`,
`.delete()` and every other `FieldFile` API behave exactly as they would for a
server-side upload, because the field stores an ordinary storage-relative name
and delegates to your Django storage.

---

## Why not django-s3direct?

This package is a rewrite of the idea behind
[django-s3direct](https://github.com/bradleyg/django-s3direct), which has not
kept up with modern Django or modern S3. Three things are different:

**It stores a storage name, not a URL.** `django-s3direct` ships its own
`S3DirectField`, a bare `Field` over a text column that holds the full object
URL. Everything downstream — signed URLs, moving buckets, switching to a CDN,
deleting a file — becomes string surgery, and the storage never learns the file
exists. Here the fields are real `FileField`/`ImageField` subclasses backed by
your configured `Storage`, so none of that is your problem.

**The server decides, and signs, everything.** In `django-s3direct` the widget is
a `TextInput` carrying the object URL, and the form field stores whatever comes
back. A user who can upload to *any* destination can therefore point *any* field
at *any* key in the bucket. Here the browser posts an HMAC-signed token, and the
form field verifies both the signature and the destination before writing
anything.

**It signs uploads the way S3 expects.** `django-s3direct` reimplements AWS
Signature V4 in JavaScript and calls back to Django to sign each chunk. This
package uses presigned POST for small files — which lets S3 itself enforce the
content type and an exact byte count — and presigned multipart part URLs for
large ones.

## Requirements

- Python 3.10+
- Django 5.2+
- An S3-compatible bucket (AWS S3, MinIO, Cloudflare R2, Ceph, …)

## Install

```bash
pip install vinta-django-s3-direct
```

Optional extras: `[drf]` for the serializer field, `[storages]` to pull in
`django-storages`.

## Setup

**1. Add the app and the URLs.**

```python
INSTALLED_APPS = [
    ...,
    "vinta_s3_direct",
]
```

```python
urlpatterns = [
    ...,
    path("s3direct/", include("vinta_s3_direct.urls")),
]
```

**2. Configure your storage** as you normally would with `django-storages`:

```python
STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3boto3.S3Boto3Storage",
        "OPTIONS": {"bucket_name": "my-media-bucket", "region_name": "us-east-1"},
    },
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
}
```

**3. Declare your destinations.**

```python
VINTA_S3_DIRECT = {
    "DESTINATIONS": {
        "avatars": {
            "key_prefix": "uploads/avatars",
            "auth": "vinta_s3_direct.auth.is_authenticated",
            "allowed_content_types": ["image/png", "image/jpeg", "image/webp"],
            "max_size": 5 * 1024 * 1024,
        },
        "recordings": {
            "key_prefix": "uploads/recordings",
            "auth": "myapp.policies.can_upload_recordings",
            "allowed_content_types": ["video/mp4"],
            "max_size": 5 * 1024 * 1024 * 1024,
            "multipart_threshold": 16 * 1024 * 1024,
        },
    },
}
```

**4. Configure CORS on the bucket** so the browser may PUT/POST to it and read
the `ETag` header back (multipart needs that last part):

```json
[
  {
    "AllowedOrigins": ["https://your-site.example"],
    "AllowedMethods": ["POST", "PUT"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]
```

## Destination options

A *destination* is a named upload policy. Every request names one, and the
server takes every security-relevant decision from it — never from the browser.

| Option | Default | What it does |
| --- | --- | --- |
| `key_prefix` | `""` | Folder the generated key lands in. |
| `key_generator` | `vinta_s3_direct.keys.default_key_generator` | `(filename, destination) -> name`. |
| `auth` | `vinta_s3_direct.auth.is_authenticated` | `(request) -> bool`. Checked on every endpoint. |
| `allowed_content_types` | `"*"` | List of accepted MIME types, or the wildcard. |
| `min_size` / `max_size` | `1` / 5 GiB | Accepted byte range. |
| `storage` | project default | A `STORAGES` alias or a dotted path. |
| `bucket` / `region` / `endpoint_url` | from the storage | Per-destination overrides. |
| `public_endpoint_url` | `endpoint_url` | Browser-facing endpoint; see *MinIO* below. |
| `acl` | `None` | Canned ACL. Leave unset on buckets with Object Ownership enforced. |
| `cache_control` | `None` | Header value, or `(filename) -> str`. |
| `content_disposition` | `None` | Header value, or `(filename) -> str`. |
| `server_side_encryption` | `None` | e.g. `"AES256"` or `"aws:kms"`. |
| `allow_multipart` | `True` | Turn chunked uploads off entirely. |
| `multipart_threshold` | 8 MiB | Files at or above this use multipart. |
| `multipart_chunk_size` | 5 MiB | Part size (S3's minimum is 5 MiB). |
| `signature_expires` | `3600` | Lifetime of the presigned URLs, in seconds. |
| `token_max_age` | `3600` | How long an upload token stays valid. |
| `allow_revert` | `True` | Whether FilePond's undo button may delete the object. |
| `verify_on_complete` | `True` | `HeadObject` after upload to confirm size and existence. |

A `"DEFAULTS"` key applies to every destination:

```python
VINTA_S3_DIRECT = {
    "DEFAULTS": {"server_side_encryption": "AES256", "token_max_age": 1800},
    "DESTINATIONS": {...},
}
```

### Writing an `auth` callable

It receives the whole `HttpRequest`, so it can look at the user, the session, or
a tenant resolved by middleware:

```python
def can_upload_recordings(request):
    return request.user.is_authenticated and request.user.has_perm("media.add_recording")
```

Four ready-made ones live in `vinta_s3_direct.auth`: `allow_any`, `deny_all`,
`is_authenticated` (the default) and `is_staff`.

## Django admin

Nothing to do:

```python
@admin.register(Profile)
class ProfileAdmin(admin.ModelAdmin):
    pass
```

That really is the whole integration, and it holds for every `ModelAdmin`, every
custom `AdminSite`, and inline formsets. It is worth knowing why, because this is
the part that most often breaks quietly:
`ModelAdmin.formfield_for_dbfield` walks `db_field.__class__.mro()` and maps
`FileField`/`ImageField` to `AdminFileWidget`. Our fields subclass both, so the
admin hands them a plain file input — which cannot work here, since the value
comes back in POST data rather than in `request.FILES`. The model field simply
discards that widget.

The tidier-looking alternative — registering our fields in the admin's
`FORMFIELD_FOR_DBFIELD_DEFAULTS` from `AppConfig.ready()` — is subtly broken: a
`ModelAdmin` snapshots that table when it is *constructed*, and
`django.contrib.admin`'s own `ready()` may run `autodiscover()` before this
package's app is ready. Handling it in the field sidesteps app-loading order
altogether.

Inline formsets work too — the JavaScript listens for Django's `formset:added`
event and initialises new rows.

## Plain forms

```python
from vinta_s3_direct.forms import S3DirectFormField


class UploadForm(forms.Form):
    document = S3DirectFormField(destination="documents")
```

Render `{{ form.media }}` in your `<head>`, and the form normally in the body.
The form does **not** need `enctype="multipart/form-data"` — only a signed token
is posted.

One nicety over a plain `FileField`: if the form fails validation on some *other*
field, the upload survives the re-render, because the posted value is a token
rather than the file itself.

## Django REST Framework

```python
from rest_framework import serializers
from vinta_s3_direct.drf import S3DirectSerializerMixin


class ProfileSerializer(S3DirectSerializerMixin, serializers.ModelSerializer):
    class Meta:
        model = Profile
        fields = ["id", "avatar"]
```

The field is asymmetric on purpose: clients **write** an upload token and
**read** back a ready-to-use URL. They never see, and never choose, the key.

```jsonc
// PATCH /api/profiles/1/
{"avatar": "eyJkZXN0aW5hdGlvbiI6ImF2YXRhcnMi...:1t8Xk2:9c..."}

// 200 OK
{"id": 1, "avatar": "https://my-media-bucket.s3.amazonaws.com/uploads/avatars/photo_a1b2.png?X-Amz-..."}
```

Use it directly if you prefer:

```python
from vinta_s3_direct.drf import S3DirectSerializerField

avatar = S3DirectSerializerField(destination="avatars", expire=900)
```

## The upload flow

For a non-Django client — a React SPA, a mobile app — talk to the endpoints
directly. All five are POST, JSON in and JSON out, and CSRF-protected.

```
POST /s3direct/begin/
  {"destination": "avatars", "filename": "me.png",
   "content_type": "image/png", "size": 51200}
→ {"transport": "post", "name": "uploads/avatars/me_a1b2.png",
   "session": "...", "url": "https://...", "fields": {...}}
```

Then POST the file to `url` with `fields` as form data, and finish:

```
POST /s3direct/complete/   {"session": "..."}
→ {"token": "...", "name": "uploads/avatars/me_a1b2.png", "size": 51200}
```

Post `token` as the field's value in your form or API request.

For a large file, `begin` answers `{"transport": "multipart", "part_size": ...,
"part_count": ...}` instead. Ask `POST /s3direct/sign-parts/` for presigned part
URLs, `PUT` each slice, collect the `ETag` headers, and pass them to `complete`:

```
POST /s3direct/sign-parts/ {"session": "...", "part_numbers": [1, 2, 3]}
POST /s3direct/complete/   {"session": "...",
                            "parts": [{"part_number": 1, "etag": "\"abc\""}, ...]}
```

`POST /s3direct/abort/` discards an in-flight upload and
`POST /s3direct/revert/` deletes a completed one that was never attached to a
model.

## Security model

- **The client never picks the key.** `begin` generates it, and it is sealed
  into the signed session; the part and complete endpoints read it from there.
- **Limits are enforced by S3, not by the browser.** A presigned POST pins the
  content type and an exact `content-length-range`, so a patched client cannot
  upload something larger or of a different type than was authorised.
- **Multipart is checked afterwards.** S3 will not enforce a size limit on a
  multipart upload, so `complete` issues a `HeadObject`, compares the real size
  against the destination, and deletes the object if it does not fit.
- **Tokens are scoped and short-lived.** A token records its destination, and
  the form field refuses one minted for a different destination — so a token from
  a permissive destination cannot be replayed into a stricter field.
- **Every endpoint re-runs `auth`.** Not just `begin`.

Signing uses Django's `SECRET_KEY` via `django.core.signing`, so
`SECRET_KEY_FALLBACKS` works during a key rotation.

## MinIO and other private endpoints

When your app reaches S3 at an address the browser cannot resolve — a MinIO
container on a private Docker network, say — set both endpoints:

```python
"uploads": {
    "endpoint_url": "http://minio:9000",           # what Django uses
    "public_endpoint_url": "http://localhost:9000",  # what the browser uses
}
```

A presigned SigV4 URL commits to its host, so the browser-facing URL is signed by
a separate client configured with the public endpoint. Rewriting the host after
signing — which is what you may have had to do with `django-s3direct` — would
invalidate the signature.

## Serving files back

This package only handles the *upload*. Reading is your storage's job:
`instance.avatar.url` returns whatever your `Storage` returns — a presigned GET
if `AWS_QUERYSTRING_AUTH` is on, a CDN URL if you set `AWS_S3_CUSTOM_DOMAIN`.

## Development

```bash
uv sync --all-extras
uv run pytest
uv run pytest --cov
uv run ruff check . && uv run ruff format --check .
uv run mypy
uv run tox
```

The suite runs against `moto`. The end-to-end tests go further and drive a real
S3-compatible HTTP server, so the presigned-POST body and the multipart `ETag`
round trip are exercised over the wire rather than mocked.

Refresh the vendored FilePond assets with `./scripts/vendor_filepond.sh`.

### Supported combinations

|  | Django 5.2 | Django 6.0 | Django 6.1 |
| --- | :-: | :-: | :-: |
| Python 3.10 | ✓ | | |
| Python 3.11 | ✓ | | |
| Python 3.12 | ✓ | ✓ | ✓ |
| Python 3.13 | ✓ | ✓ | ✓ |
| Python 3.14 | | ✓ | ✓ |

Blank cells are combinations Django itself does not support. CI runs every
filled cell, and each tox environment asserts the Django version it actually
imported — so a matrix entry cannot silently test the wrong one.

## Licence

MIT. FilePond and its plugins are vendored under
`src/vinta_s3_direct/static/vinta_s3_direct/vendor/` and are also MIT-licensed
(© PQINA).
