Metadata-Version: 2.4
Name: django-async-sftp-server
Version: 0.0.4
Summary: Async SFTP server (built on asyncssh) that exposes a Django-ORM-backed virtual filesystem
Home-page: https://github.com/swat-mobility/django-async-sftp-server
License: LGPL-3.0-or-later
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: No Input/Output (Daemon)
Classifier: Framework :: Django
Classifier: Framework :: Django :: 2.2
Classifier: Framework :: Django :: 3.0
Classifier: Framework :: Django :: 3.1
Classifier: Framework :: Django :: 3.2
Classifier: Framework :: Django :: 4.0
Classifier: Framework :: Django :: 4.1
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 5.2
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: File Transfer Protocol (FTP)
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE.GPL
Requires-Dist: Django>=2.2.5
Requires-Dist: asyncssh>=2.22
Requires-Dist: asgiref>=3.3
Provides-Extra: dev
Requires-Dist: tox; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: flake8-bugbear; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: isort; extra == "dev"
Dynamic: license-file

# django-async-sftp-server

An SFTP server for Django projects. Instead of serving a real filesystem, it presents your
Django models to SFTP clients as a filesystem they can browse, download from, and upload to:
folders and files are your own models, and file content is stored in a standard Django
`FileField`. Because content lives in a `FileField`, any Django `Storage` backend works
underneath — local disk, Amazon S3, Google Cloud Storage, and so on.

You run the server as a Django management command. You decide who may connect and what they may
do by writing a single **authorization class**. The package is built on
[asyncssh](https://asyncssh.readthedocs.io/en/latest/) and runs on Django 2.2.5 and newer.

## Requirements

* Python 3.10 or newer
* Django 2.2.5 or newer

The full tested matrix is listed under [Supported Python and Django versions](#supported-python-and-django-versions).

## Installation

```bash
pip install django-async-sftp-server
```

Add `"sftp_server"` to `INSTALLED_APPS`:

```python
INSTALLED_APPS = [
    ...,
    "sftp_server",
]
```

This registers a system check that validates your configuration when you run `manage.py check`.

## Getting started

### 1. Define your folder and file models

A folder model and a file model back the virtual filesystem. The quickest way to create them is
to subclass the abstract base classes the package provides:

```python
# myapp/models.py
from django.db import models
from sftp_server.models import AbstractSFTPFile, AbstractSFTPFolder


class Folder(AbstractSFTPFolder):
    pass


class File(AbstractSFTPFile):
    folder = models.ForeignKey(Folder, on_delete=models.CASCADE, related_name="files")
```

* `AbstractSFTPFolder` gives you `name`, a self-referential `parent`, and
  `created_at` / `updated_at`.
* `AbstractSFTPFile` gives you `name`, a `content` `FileField`, and `created_at` / `updated_at`.
  You must add the foreign key from your file model to your folder model yourself, since it has
  to point at *your* concrete `Folder`.

You are not required to use these base classes. Any pair of models works, as long as you tell the
package which fields play which roles through the field maps in `SFTP_SERVER` (see
[Configuration reference](#configuration-reference)).

### 2. Configure the package

```python
# settings.py
SFTP_SERVER = {
    "FOLDER_MODEL": "myapp.Folder",
    "FILE_MODEL": "myapp.File",
    "FILE_FIELDS": {"parent": "folder"},          # map the "parent" role to your FK field
    "AUTHORIZATION_CLASS": "myapp.sftp_auth.MyAuthorization",
}
```

`FILE_FIELDS` is needed here only because the foreign key on the file model is named `folder`
rather than the default `parent`. If you use the field names from the abstract base classes
unchanged, you can omit the field maps entirely. See the
[Configuration reference](#configuration-reference) for every available key.

### 3. Write an authorization class

Every connection and every operation is checked by your authorization class. Subclass
`BaseSFTPAuthorization` and implement, at a minimum, host keys, one authentication method, and
the permission checks you want to enforce:

```python
# myapp/sftp_auth.py
from sftp_server.auth import BaseSFTPAuthorization, SFTPPrincipal


class MyAuthorization(BaseSFTPAuthorization):
    def get_host_keys(self, *, root_id):
        """Return this root folder's SSH host key(s) as PEM content (str or bytes)."""
        ...

    def supports_password_auth(self, *, root_id):
        return True

    def authenticate_password(self, *, username, password, root_id):
        """Return an SFTPPrincipal on success, or None to reject."""
        ...

    def can_read(self, *, principal, root_id, path):
        return True

    # ...and the other can_* checks you need
```

Every method is a plain synchronous function, so you write ordinary Django code with no async
awareness required. The full interface is documented under
[The authorization class](#the-authorization-class).

### 4. Create a root folder and start the server

```bash
python manage.py migrate
python manage.py createsftprootfolder --name reports   # prints the new folder's id
python manage.py runsftpserver --root-id 1 --host 0.0.0.0 --port 8022
```

`createsftprootfolder` creates a new top-level folder (a row with no parent) and prints its
primary key. `--name` is optional; omit it to create an unnamed root folder.

`runsftpserver` serves one root folder as `/`. Pass its primary key as `--root-id`, and the
listener address as `--host` / `--port`. To serve several independent trees, see
[Running multiple root folders](#running-multiple-root-folders).

Clients can now connect with any SFTP client, for example:

```bash
sftp -P 8022 someuser@localhost
```

## Configuration reference

All configuration lives in a single `SFTP_SERVER` dictionary in your settings:

```python
SFTP_SERVER = {
    # Required
    "FOLDER_MODEL": "myapp.Folder",              # "app_label.ModelName"
    "FILE_MODEL": "myapp.File",
    "AUTHORIZATION_CLASS": "myapp.sftp_auth.MyAuthorization",  # dotted path

    # Optional — field-name maps.
    # Each role defaults to the name shown here, which matches the fields the
    # AbstractSFTPFolder / AbstractSFTPFile base classes use, so a project built on
    # those bases needs none of this. Set a role to None if your model has no such
    # field (for example, no creation/modification tracking).
    "FOLDER_FIELDS": {
        "name": "name",
        "parent": "parent",          # self-referential foreign key
        "created_at": "created_at",
        "modified_at": "updated_at",
    },
    "FILE_FIELDS": {
        "name": "name",
        "parent": "parent",          # foreign key to FOLDER_MODEL
        "content": "content",        # the FileField holding the bytes
        "size": None,                # optional cached-size field; None derives size from content
        "created_at": "created_at",
        "modified_at": "updated_at",
    },

    # Optional — tuning

    # Each upload is buffered until the client closes the file. Small uploads stay in
    # memory; larger ones spill to a temporary file. These control that buffer.
    "SPOOL_MAX_SIZE": 1048576,       # bytes buffered in memory before spilling to disk
    "SPOOL_BUFFERING": -1,           # buffering mode (-1 uses the platform default)
    "SPOOL_PREFIX": None,            # spill-file name prefix
    "SPOOL_SUFFIX": None,            # spill-file name suffix
    "SPOOL_DIR": None,               # spill directory (None uses the platform default)

    "SHUTDOWN_GRACE_PERIOD": 30,     # seconds to let in-flight sessions finish on shutdown

    "THREAD_SENSITIVE": False,       # serialize all sync/ORM work onto one thread (see below)

    "SYNC_TO_ASYNC": "asgiref.sync.sync_to_async",  # dotted path (see below); rarely overridden
}
```

A few keys are intentionally *not* settings, because they vary per running process rather than
per project:

* **Listener address and served root** — `--host`, `--port`, and `--root-id` are arguments to
  `runsftpserver`.
* **SSH host keys** — returned by `get_host_keys` on your authorization class, so they can be
  stored and looked up however you prefer (database, secrets manager, environment).

`SFTP_SERVER` is validated whenever `manage.py check` runs and whenever `runsftpserver` starts:
the model must exist, every configured field name must exist on it, and the authorization class
must implement the full interface. A misconfiguration is reported as a clear startup error rather
than surfacing mid-connection.

### About `SYNC_TO_ASYNC`

The server runs asynchronously, but the Django ORM is synchronous, so ORM calls are run through
the callable named by `SYNC_TO_ASYNC`, which defaults to `"asgiref.sync.sync_to_async"` and needs
no configuration for virtually all projects. It is exposed as a setting only so that advanced
deployments can supply their own wrapper (for example, to add instrumentation); most users never
need to set it.

### About `THREAD_SENSITIVE`

This is the `thread_sensitive` flag passed to `SYNC_TO_ASYNC`, and it controls a
throughput-vs-SQLite-safety trade-off. It defaults to `False`, which is the right choice for any
genuinely concurrent database (PostgreSQL, MySQL):

* **`False` (default)** — each synchronous/ORM call may run on any thread in the event loop's
  default executor pool (at most `min(32, os.cpu_count() + 4)` threads). Django connections are
  thread-local, so this uses up to one connection per pool thread rather than one for the whole
  process, and independent sessions' ORM work runs in parallel. On *file-based SQLite*, concurrent
  writes across those threads can produce sporadic `database is locked` errors.
* **`True`** — serializes *all* synchronous work in the process onto a single dedicated worker
  thread. This removes the SQLite locking concern, but it also serializes work that is not a mere
  ORM lookup — most importantly the storage backend's `save()` during an upload, which for a large
  file on a remote backend (S3/GCS) can take an effectively unbounded time and will block every
  other session's ORM work for its full duration.

In short: leave it `False` on a concurrent database; set it to `True` only if you run on SQLite and
actually observe lock contention.

## The authorization class

Your authorization class is the single place that decides who may connect and what they may do.
Subclass `sftp_server.auth.BaseSFTPAuthorization`. Every method is a plain synchronous function —
never a coroutine — so you write ordinary Django code, and methods that query the database may do
so directly.

```python
from sftp_server.auth import BaseSFTPAuthorization, SFTPPrincipal


class MyAuthorization(BaseSFTPAuthorization):
    def get_host_keys(self, *, root_id):
        ...

    def supports_password_auth(self, *, root_id):
        return True

    def authenticate_password(self, *, username, password, root_id):
        ...

    # can_read, can_list, can_stat, can_write, can_create_dir, can_delete,
    # can_rename, can_setstat, and the customization hooks described below
```

### Authentication

* `supports_password_auth` / `supports_public_key_auth` decide which authentication methods a
  client may attempt. They are called on the connection path, so keep them cheap and do not query
  the database.
* When a method is supported, `authenticate_password` / `authenticate_public_key` are called to
  verify the credentials. These may safely query the database. Return an `SFTPPrincipal` on
  success, or `None` to reject.

An `SFTPPrincipal` is a plain, fully-resolved value object representing the authenticated user.
It is passed back to your `can_*` checks on every subsequent operation.

### Authorization

Permission checks are expressed as eight primitives, not one per SFTP operation. Several wire
operations map onto the same check — for example, both `remove` (file) and `rmdir` (folder)
reduce to `can_delete`:

| Check | Governs |
|-------|---------|
| `can_list` | listing a folder's contents |
| `can_stat` | reading a path's attributes |
| `can_read` | opening a file for reading |
| `can_write` | opening a file for writing |
| `can_create_dir` | creating a folder |
| `can_delete` | deleting a file or folder |
| `can_rename` | renaming or moving a file or folder |
| `can_setstat` | changing a path's attributes |

Operations that have no meaning for a model-backed filesystem (symlinks, hard links, byte-range
locks, `statvfs`) are always rejected and never reach your class.

See [`dev/filesystem/authorization.py`](dev/filesystem/authorization.py) for a minimal, fully
working example built on Django's stock `auth.User`.

## Owners, groups, and permissions

By default the virtual filesystem reports fixed permission bits and no owner or group, and client
`chmod` / `chown` / `chgrp` requests are accepted but change nothing. Override these hooks on your
authorization class to change that behavior:

* **`get_attrs`** — return an `SFTPAttrsOverride` (uid, gid, and POSIX mode bits) for a path, to
  report per-path ownership and permissions to `stat` and directory listings. The package adds
  the folder/file type bit itself.
* **`set_attrs`** — persist a client's requested owner, group, or permission change however you
  like (for example, to a field on your own model). It is called with `creating=True` right after
  a new file or folder is created — even if the client requested no attributes, so you can assign
  defaults — and with `creating=False` on an explicit `chmod` / `chown` / `chgrp`. Each of `uid`,
  `gid`, and `permissions` is `None` when it was not part of the request. The already-fetched
  model instance is passed in, so no extra database lookup is needed.
* **`get_user_name`** / **`get_group_name`** — customize the owner and group *names* shown in
  `ls -l`-style listings. When they are not overridden or return `None`, the listing falls back to
  the textual representation of the `uid` / `gid` itself. Like `supports_password_auth`, these are
  called while formatting output, so keep them cheap and free of database queries.

All four default to a safe no-op, so an authorization class that doesn't override them keeps
working unchanged.

## Signals

For side effects such as indexing, notifications, auditing, or virus scanning, the package emits
Django signals from `sftp_server.signals`. (Use these for side effects, not for authorization —
authorization belongs in your authorization class.) Every signal fires *after* the event has
already happened, and none can veto an operation.

```python
from django.dispatch import receiver

from sftp_server.signals import sftp_file_write_close
from sftp_server.sftp import AsyncSFTPServer


@receiver(sftp_file_write_close, sender=AsyncSFTPServer)
def on_written(sender, *, principal, root_id, path, instance, created, size, connection_id, **kwargs):
    ...
```

| Signal | Fired when | Extra kwargs |
|--------|-----------|--------------|
| `sftp_file_read` | a file is opened for reading | `principal, root_id, path, instance` |
| `sftp_file_write` | a file is opened for writing | `principal, root_id, path, instance, created` |
| `sftp_file_read_close` | a read handle is closed | `principal, root_id, path, instance` |
| `sftp_file_write_close` | a write handle is closed and its content committed | `principal, root_id, path, instance, created, size` |
| `sftp_file_removed` | a file is deleted | `principal, root_id, path` |
| `sftp_dir_created` | a folder is created | `principal, root_id, path, instance` |
| `sftp_dir_removed` | a folder is removed | `principal, root_id, path` |
| `sftp_renamed` | a file or folder is renamed or moved | `principal, root_id, src_path, dst_path, overwrite` |
| `sftp_attrs_changed` | a path's attributes are set | `principal, root_id, path, is_dir, instance, mtime, uid, gid, permissions` |
| `sftp_connection_made` | a connection is accepted | `root_id, conn, peer_addr` |
| `sftp_connection_lost` | a connection is closed | `root_id, conn, peer_addr, exc` |
| `sftp_auth_succeeded` | authentication succeeds | `root_id, username, method, principal` |
| `sftp_auth_failed` | authentication is rejected | `root_id, username, method` |

The `sender` is `AsyncSFTPServer` for the file and folder signals, and `AsyncSSHServer` for the
connection and authentication signals (import them from `sftp_server.sftp` and `sftp_server.ssh`).
`method` is `"password"` or `"public_key"`.

### Correlating events with a connection

Every signal also carries a `connection_id` — an opaque, unique, hashable `uuid.UUID` identifying
the connection the event happened on. Use it to correlate file events with their connection. For
example, to find files that a dropped connection left open: record `(connection_id, path)` on
`sftp_file_write` / `sftp_file_read`, clear it on the matching `*_close`, and on
`sftp_connection_lost` whatever is still recorded for that `connection_id` was not closed cleanly.
(A `principal` cannot be used as this key, because one principal may hold several concurrent
connections.)

### File open/close lifecycle

Files have a symmetric open/close lifecycle. `sftp_file_read` / `sftp_file_write` fire once when a
handle is opened (not per chunk), and `sftp_file_read_close` / `sftp_file_write_close` when it is
closed. A write handle commits its buffered content to storage on close, so
`sftp_file_write_close` carries the committed `size`. If a client disconnects without closing a
write handle, `sftp_file_write` fires but is not followed by `sftp_file_write_close`. The
`created` flag distinguishes a newly created file from an existing one being reopened.

### Writing receivers

Two behaviors matter when writing receivers:

* **Dispatch is robust.** Each signal is sent with `send_robust`, so an exception in one receiver
  does not abort the operation or stop the other receivers; it is logged via the `sftp_server`
  logger instead.
* **Where receivers run differs by signal:**
  * File, folder, and authentication signal receivers may freely use the ORM. File and folder
    signals fire only after the operation's database transaction commits, so they never fire on
    rollback.
  * Connection signal receivers (`sftp_connection_made` / `sftp_connection_lost`) run on the
    asyncio event loop. Keep them lightweight and do not make blocking ORM calls in them.

## Running multiple root folders

A single `runsftpserver` process always serves exactly one root folder. To serve several
independent trees, create one root folder per tree and run one process per tree, typically on
different ports:

```bash
python manage.py createsftprootfolder --name reports    # -> id 1
python manage.py createsftprootfolder --name invoices   # -> id 2

python manage.py runsftpserver --root-id 1 --host 0.0.0.0 --port 8022
python manage.py runsftpserver --root-id 2 --host 0.0.0.0 --port 8023
```

The `root_id` is passed explicitly to every authorization method and signal, so a single
authorization class can serve every root folder unchanged.

## Supported Python and Django versions

- Python versions:
    - 3.10
    - 3.11
    - 3.12
    - 3.13
- Django versions:
    - 2.2 (2.2.5+)
    - 3.0
    - 3.1
    - 3.2
    - 4.0
    - 4.1
    - 4.2
    - 5.0
    - 5.1
    - 5.2

See [`tox.ini`](tox.ini) for the exact matrix tested in CI.

## Limitations

* Symlinks, hard links, byte-range locks, and `statvfs` are not supported.
* Reading at arbitrary offsets requires a seekable `Storage` backend. If the backend cannot seek,
  opening a file for reading fails immediately with a clear error rather than returning corrupt
  data.
* A file's content is committed to storage once, when the client closes the upload — not
  incrementally.
* When the setting `THREAD_SENSITIVE=True`, all database access within a single server process is serialized, favoring correctness over  raw throughput.

## Example projects

* [`dev/`](dev/) — a complete, runnable example project (models, authorization class, and the
  test suite) backed by local disk storage.
* [`sample/`](sample/) — the same project with file content backed by real cloud object storage
  via [django-storages](https://django-storages.readthedocs.io/), with interchangeable settings
  modules for Google Cloud Storage and Amazon S3.

## License

GNU Lesser General Public License v3.0 (LGPLv3) — see [`LICENSE`](LICENSE) (and
[`LICENSE.GPL`](LICENSE.GPL), which it incorporates by reference).
