Metadata-Version: 2.4
Name: django-file-finder
Version: 4.0.dev0
Summary: A media asset management application for Django.
Author-email: Jacob Rief <jacob.rief@gmail.com>
Maintainer-email: Django CMS Association and contributors <info@django-cms.org>
License: MIT
Project-URL: Homepage, https://github.com/django-cms/django-filer
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Framework :: Django
Classifier: Framework :: Django :: 5.2
Classifier: Framework :: Django :: 6.0
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE-filer.txt
License-File: LICENSE-finder.txt
Requires-Dist: django>=5.2
Provides-Extra: cte
Requires-Dist: django-cte; extra == "cte"
Provides-Extra: heif
Requires-Dist: pillow-heif; extra == "heif"
Provides-Extra: audio
Requires-Dist: ffmpeg-python; extra == "audio"
Requires-Dist: django-entangled; extra == "audio"
Provides-Extra: image
Requires-Dist: Pillow; extra == "image"
Requires-Dist: django-entangled; extra == "image"
Provides-Extra: svg
Requires-Dist: django-entangled; extra == "svg"
Requires-Dist: py-svg-hush; extra == "svg"
Provides-Extra: video
Requires-Dist: ffmpeg-python; extra == "video"
Requires-Dist: django-entangled; extra == "video"
Provides-Extra: s3
Requires-Dist: django-storages[s3]; extra == "s3"
Provides-Extra: e2e
Requires-Dist: playwright; extra == "e2e"
Requires-Dist: pytest; extra == "e2e"
Requires-Dist: pytest-django; extra == "e2e"
Requires-Dist: Pillow; extra == "e2e"
Dynamic: license-file

# Django-Filer ("Finder" branch)

The "Finder" branch of django-filer is a complete rewrite of the original **django-filer** project.

A rewrite was necessary because the original codebase was not maintainable anymore. Please read this
[discussion](https://github.com/django-cms/django-filer/discussions/1348) for details. Apart from the maintainability issues, using the File and Folder
models was not possible in a multi-tenant environment. Extending the file model with a more
specialized implementation was possible in theory, but so complicated that apart from the
`ImageModel` no other implementations ever have been created.


## Less third-party dependencies

The "Finder" branch of **django-filer** has less third-party dependencies. It does not depend on
[django-polymorphic](https://django-polymorphic.readthedocs.io/en/stable/),
[django-mptt](https://django-mptt.readthedocs.io/en/latest/) and
[easy-thumbnails](https://easy-thumbnails.readthedocs.io/en/latest/) anymore.

For large datasets [django-cte](https://github.com/dimagi/django-cte) is reccomended, in order to improve the speed
of tree travesals, which is important while searching.

Since each `FileModel` contains a `JSONField` to store arbitrary data, [django-entangled](https://github.com/jrief/django-entangled)
is reccomended, in order to give users the opportunity to edit the contents of that field.

The client part of the new admin user interface has no runtime dependencies. It is compiled into two
JavaScript files, which are included by the corresponding admin views. One of them is used for the
django-admin interface, the other one for the frontend. The frontend part is also used for the new
file selection widget.


## Extensibility

The "Finder" branch of django-filer is designed to be extensible for every MIME-type one desires.
Therefore, those extensions might need their own dependencies in order to extract or visualize a
certain type of file. A minimal installation can be configured to only distinguish between files
and folders.


## Multiple Ambits to hold Folder Roots

The "Finder" branch of **django-filer** allows multiple ambits, each able to hold multiple root
folders. Such an ambit must be created using the management command `manage.py finder add-ambit …`.
Each ambit has a slug and a name. The slug is used in the URL to access that ambit, while the name
is used for display purposes. All ambits are displayed in the left sidebar of the Django admin. For
each ambit, a different Django storage can be configured.


## New Model Features

The "Finder" branch introduces a new model named `InodeModel`. This is the base class for all other
models, such as the `FileModel`, the `FolderModel` and all other specialized implementations. By
using a UUID instead of a numerical primary key, it is possible to use different database tables
while preserving the possibility to perform queries across all tables. This allows us to register
models inheriting from the `AbstractFileModel` without the need of **django-polymorphic**, and thus
a join between two or more tables for each query.

Multiple ambits are allowed, each containg one root folder, one trash folder per user and a list of
allowed tags. Each ambit must explictly be created using a management command. All ambits are
displayed in the left sidebar of the Django admin. For each ambit, a different Django storage can be
configured. It also is possible to restrict an ambit to a Site and/or to an AdminSite.

The Admin interface has also been completely rewritten. For instance, there is no more list view for
the `FolderModel` and the `FileModel` (or any specialized implementation). Instead, there are only
details views for each of those models. This is much nearer to what a user would expect from a file
system. Therefore, if a user wants to access the list view of a folder, he is redirected immediately
to the detail view of the root folder of the given ambit. From there, he can traverse the folder
tree in the same manner he's used to from his operating system.


## New User Interface

The user interface has also been completely rewritten. It offers drag and drop functionality for
moving files and folders. It now is possible to select multiple files and folders at once by a
rectangular selection with the mouse.
Double-clicking on a folder opens it, by loading the detail view for that `FolderModel`. Selected
files or folders can be dragged into a target folder. They can also be cutted or copied to a
clipboard and pasted into another folder.


### Trash Folder

Each user now has its *own private trash* folder. Whenever he deletes a file or folder, it is moved
to that trash folder. From there, he can restore the file or folder, or delete it permanently. The
trash folder can be emptied automatically after a given time period.


## Permission System

The permission system of **django-filer** is using the concept of Access Control Lists (ACLs)
similar to Posix or NTFS ACLs. This allows granting fine-grained permissions to everybody,
individual users and/or groups for each file and folder.

Permissions are controlled through the model named `AccessControlEntry`. This model has a foreign
key onto `InodeModel` and a so-called "principal". A principal is either a user, a group or for
everyone. They are mutually exclusive and implemented as a nullable foreign key onto `User`, or a
foreign key onto `Group`. Either of them can be set, but not both. If neither is set, the permission
is considered for everyone.

By using a separate model `AccessControlEntry`, **django-filer** can now compute the permissions
using just one database (sub-)query while filtering all inodes. Until version 3, the permissions had
to be computed traversing all ancestors starting from the current folder up to the root of the
folder tree. This was a time-consuming operation and made **django-filer** slow for large datasets.

Each `AccessControlEntry` has a bitmap field used to represent:
* `write` set for a folder: Allows the user to upload a file, to change the name of that folder,
  to reorder files in that folder, to move files out of that filer.
* `write` set for a file: Allows the user to replace a file, to edit the file's meta-data, and to
  move the file to another folder, if the source and target folders have write permissions.
* `read` set for a folder: Allows the user to open that folder and view its content.
* `read` set for a file: Allows the user to view a thumbnail of that file inside its containing
  folder and use that file, including copying.
* `admin`: Allows the user to change the permissions of that folder or file.
* A generic foreign key pointing onto the `InodeModel`. This creates a one-to-many relation between
  different file types and folders on one side and the access control list on the other.

In addition to `AccessControlEntry` the permission system also provides a model named
`DefaultAccessControlEntry`. This acts as a template for newely files or folders created inside, or
move to another folder.
The `DefaultAccessControlEntry` is implemented as a separate model with a foreign key onto the
`FolderModel`, because it is not possible to use the `AccessControlEntry` for both purposes, since
the default permissions must be inherited by all children of a folder, while the regular
permissions only apply to the file or folder they are attached to.

A foreign key onto the folder model to set a permission template. Read below for details.

If a folder has `write` but no `read` permission, the user can upload files into that folder, but
doesn't see files from other users. This is named the "Dropbox" functionality.

Each file and folder has a foreign key named `owner`, pointing onto the `User` model. The owner of a
file or folder can change its permissions if he has the global permission to do so. When creating a
new file or folder, the currently loggedin user is set as the owner of that file or folder.

Only a superuser and the owner of a file or folder can change its permissions. The superuser can
change the permissions of any file or folder. Staff users can change the permissions only of files
they own themselves.

Only a superuser can change the owner of a file or folder.

In addition to the file and folder permissions, each folder requires a template of permissions on how to
inherit them to files and folders created as children of that specific folder. This can be achieved with
a separate foreign key in model `AccessControlEntry` pointing onto the `FolderModel`.

Microsoft gives a good explanation on the implementation of
[ACLs in their Data-Lake implementation](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-access-control).


### Alternative Views

The user interface offers four alternative views for the folder tree.

**Tiles** is a "flat" view, which shows all files and folders as a bunch of tiles.

**Mosaic** is very similar to the "Tiles" view, but shows a much smaller version of those file,
folder and image icons.

**List** is a view with detailed information about every file or folder. Here every item requires a
separate row.

**Column** is a view which shows columns for the currently selected folder and each of its
ancestors. This allows to easily move files between those folders.


### Multiple Favrourite Folders

Each user can have multiple favourite folders. This allows him to quickly access those folders from
the navigation bar. It is also possible to drag a file from the current view into one of the tabs
for of the favorite folders.


### Implementation Details

The new user interface is based on the [React](https://reactjs.org/) framework with the
[DnD-Kit](https://docs.dndkit.com/) as its main additional dependency.


## Installation

The new version of **django-filer** is not yet available on PyPI. Therefore, you have to install it
from GitHub:

```shell
git clone https://github.com/django-cms/django-filer.git
cd django-filer
git switch finder
pip install --no-deps -e .
```

The new version of **django-filer** requires Django-5.2 or later.

The new version of **django-filer** currently has only been tested with SQLite and Postgres, but
should also work on MariaDB and MySQL.

In `settings.py` of your project, add these extra dependencies or those you really need:

```python
    INSTALLED_APPS = [
        ...
        'finder',
        'finder.contrib.archive',  # supports zip, tar, tar.gz, tar.bz2, tar.xz
        'finder.contrib.audio',  # supports mp3, ogg, flac, wav
        'finder.contrib.common',  # supports PDF, spreadsheets
        'finder.contrib.image.pil',  # supports bmp, gif, jpeg, png, tiff
        'finder.contrib.image.svg',  # supports svg
        'finder.contrib.video',  # supports mp4, webm, ogv
        ...
    ]
```

If you use:
* `finder.contrib.audio` or `finder.contrib.video`, assure that `ffmpeg-python` is installed.
* `finder.contrib.image.pil`, assure that `Pillow` is installed.
* `finder.contrib.image.svg` needs no extra dependency; it reads an SVG's size from the
  document and thumbnails it by rewriting the root element. Documents that state their
  size only in relative units (`width="100%"` without a `viewBox`) cannot be measured
  that way and are shown with a generic icon.
* Postgres as a database, install `psycopg2` or `psycopg2-binary` if available for your platform.

Each root folder requires two storage backends. One for the public files and one for their
thumbnails and/or samples. Add these storage backends to the `STORAGES` setting in your
`settings.py`:

```python
STORAGES = {
    'default': {
        'BACKEND': 'django.core.files.storage.FileSystemStorage',
    },
    'staticfiles': {
        'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage',
    },
    'finder_public': {
        'BACKEND': 'finder.storage.FinderSystemStorage',
        'OPTIONS': {
            'location': '/path/to/your/media/filer_public',
            'base_url': '/media/filer_public/',
            'allow_overwrite': True,
        },
    },
    'finder_public_samples': {
        'BACKEND': 'finder.storage.FinderSystemStorage',
        'OPTIONS': {
            'location': '/path/to/your/media/filer_public_thumbnails',
            'base_url': '/media/filer_public_thumbnails/',
            'allow_overwrite': True,
        },
    },
}
```

If instead of using the local file system you want to use another storage backend, such as MinIO,
Amazon S3 or Google Cloud Storage, configure the corresponding storage backend in the `STORAGES`
setting as:

```python
STORAGES = {
    ...
    'finder_public': {
        'BACKEND': 'storages.backends.s3.S3Storage',
        'OPTIONS': {...},
    },
    'finder_public_samples': {
        'BACKEND': 'storages.backends.s3.S3Storage',
        'OPTIONS': {...},
    },
}
```

Note that multiple root folders can share the same storage location or bucket.

If you declare neither `finder_public` nor `finder_public_samples`, finder derives them
from your `default` storage on startup: same filesystem root and URL prefix, but a
subdirectory per alias, the `FinderSystemStorage` backend and `allow_overwrite` turned on.
With Django's defaults that gives you

```
MEDIA_ROOT/finder_public/          served from  MEDIA_URL + finder_public/
MEDIA_ROOT/finder_public_samples/  served from  MEDIA_URL + finder_public_samples/
```

which is enough to get started. Declare them yourself once you want the files somewhere
else — on S3, on another volume, or shared between projects.

> [!NOTE]
> The derivation needs a `FileSystemStorage` as your `default`. If that is a remote
> backend there is no location to put a subdirectory under, and `manage.py check` reports
> `finder.W001`. Declare the two storages explicitly in that case.

Run the migrations for app `finder`:

```shell
python manage.py migrate finder
```

This creates a first ambit for you, so that the admin and the form widgets work right away.
It is named by `FINDER_DEFAULT_AMBIT`, which is `"public"` — the slug the
`FinderFileField`s and `FinderFolderField`s of your models refer to when they declare no
ambit of their own. Override it if you want a different name:

```python
FINDER_DEFAULT_AMBIT = 'assets'
```

Its root folder grants read/write to everybody, which is the same thing
`manage.py finder add-ambit` does. Restrict it through the permission editor in the admin,
or set up the ambits yourself instead:

```python
FINDER_CREATE_DEFAULT_AMBIT = False
```

The setting is only read while `finder.0001_initial` is applied, so it has to be in
place before you migrate for the first time.

Create further ambits with the configuration above:

```shell
python manage.py finder add-ambit private --values name="Private Folder" storage=finder_private sample_storage=finder_private_samples
```

You can create multiple root folders, each with their own unique slug and name. You may configure a
storage backend for each root folder or share them between multiple root folders.

If you already have **django-filer** installed and that database is filled, you can migrate the
meta-data of those files and folders into the new database tables. The physical files on disk are
not affected by this migration. Remember to leave the `filer` app in `INSTALLED_APPS`.

```shell
python manage.py filer_to_finder
```

This does not affect the original database tables. You can still use the original **django-filer**
codebase in parallel to the new "Finder" branch. They both share the same underlying file system,
usually `media/filer_public` but store their meta information in completely independent database
tables. The only caveat is that you should not erase files in the trash folder, because then they
are also gone on disk.

The client part of the new admin user must be compiled before it can be used. This requires a modern
version (18 or later) of [Node.js](https://nodejs.org/en/) and [npm](https://www.npmjs.com/), which usually are installed anyway.
The following commands installs the requirements and compiles the code:

```shell
cd django-filer
npm install --include=dev
npm run buildall
```


## Usage

After restarting the development server, you can access the new admin user interface at the URL
http://localhost:8000/admin/finder/foldermodel/ . Clicking on that link will redirect you to the
root folder of the current site.


## Admin Backend

The admin backend is implemented as a web interface mimicking the features of the Finder app of your
operating system's graphical user interface. It is possible to …

* Download, upload files and to view them as thumbnailed images.
* Navigate through the folder tree by double-clicking on a folder.
* Move files and folders by drag and drop.
* Select multiple files and folders by a rectangular selection with the mouse.
* Cut or copy files and folders to a clipboard and paste them into another folder.
* Delete files and folders, which are moved to a trash folder.
* Restore files and folders from the trash folder or to delete them permanently.
* Create new folders and to upload files.
* Rename files and folders by slowly double-clicking on their name.
* Create multiple tabs for favourite folders quick access. They also can be used to drag files from
  the current view into one of the tabs.
* Search for files and folders.
* Tag files and filter them by their tags.


## Interface to other Django projects

**django-filer** (Finder branch) ships with a new file selection field. This field can be used in
any Django model. It allows to select a file from the files stored inside **django-filer**. It also
allows to upload a local file and use it immediatly.

Example:

```python
from django.db import models
from finder.models.fields import FinderFileField

class MyModel(models.Model):
    my_file = FinderFileField(
        null=True,
        blank=True,
        ambit='my-ambit',  # this must be the slug of an existing ambit
    )
```

Forms generated from this model will have a file selection widget for the `my_file` field. When
rendered as HTML, this widget becomes the webcomponent `<finder-file-select …></finder-file-select>`
with a few additional attributes. The JavaScript part of the widget must be included using the
script tag `<script src="{% static 'finder/js/finder-select.js' %}"></script>`.

The demonstration app provides an example how this field can be used in a form. Just vist the URL
http://localhost:8000/demoapp/ and click on the blank area to either select an existing file or
upload a new one.


## Thumbnailing

In addition to the x- and y-coordinates of the focal point, the `ImageModel` now also stores the
main area of interest as a square. This information then is used to create thumbnails using
[Art Direction](https://www.smashingmagazine.com/2016/01/responsive-image-breakpoints-generation/).

The method `finder.contrib.image.pil.models.ImageModel.crop()` creates a thumbnail of the given
image taking the focal point and the resolution into account. The following rules apply when
cropping and thumbnailing an image:

* The thumbnailed image always contains at least the main area of interrest, except for this
  situation: If the thumbnail is narrower than the main area of interest, the thumbnail will be
  cropped to the center of that area.
* The resolution of the area to be thumbnailed is always at least the same as the resolution of the
  original image.


will take the resolution of the corresponsding image into
consideration. This will allow to create different versions of the same canonical image, depending
on the width of the device the image is displayed.

By extending the focal point to three degrees of freedom, it is possible to create a thumbnail
taking the resolution into account. This allows us to create thumbnails taking art direction into
consideration. For instance, if a user uploads a portrait image, the focal point can be set to the
head of the person in that image.


## Validating Uploads

Every uploaded payload is validated before it is written to storage. A validator is a callable

```python
def my_validator(file_name, file, owner, mime_type):
    ...
```

that raises `finder.exceptions.FileValidationError` to reject the upload. It is handed the uploaded
file rewound and opened for writing, so it may also **rewrite the content in place** rather than
reject it — that is how the bundled SVG sanitizer works. Rejecting a payload leaves nothing behind:
validation runs before the file reaches the storage backend.

This is deliberately the same contract django-filer uses for `FILER_ADD_FILE_VALIDATORS`, so
validators written for filer — including third-party ones — work here unchanged. A validator
signalling rejection with a plain `ValueError` is accepted too.

By default finder rejects the formats a browser executes in the media origin, and sanitizes SVG
uploads (which needs `py-svg-hush`, part of the `svg` extra):

| MIME-type | Validator |
|---|---|
| `text/html` | `finder.validators.deny_html` |
| `application/xhtml+xml`, `application/xml`, `text/xml`, `application/xslt+xml` | `finder.validators.deny` |
| `image/svg+xml` | `finder.contrib.image.svg.validators.sanitize_svg` |

Unlike django-filer, finder does **not** deny `application/octet-stream`: `FileModel` is the
documented fallback for everything that matches no other model, and browsers report that MIME-type
for plenty of harmless files. Add `{'application/octet-stream': ['finder.validators.deny']}` for
filer's behaviour.

Add your own validators with `FINDER_PAYLOAD_VALIDATORS`, a dict mapping a MIME-type to a list of
validators. The MIME-type may be exact, a subtype wildcard (`image/*`) or `*/*`; more general
matches run first. Values are dotted paths, callables or classes.

```python
FINDER_PAYLOAD_VALIDATORS = {
    'image/*': ['myapp.validators.strip_exif'],
}
```

Drop a default with `FINDER_REMOVE_PAYLOAD_VALIDATORS`, a list of MIME-types:

```python
FINDER_REMOVE_PAYLOAD_VALIDATORS = ['image/svg+xml']
```


## Further Steps

* A quota system, which allows to limit the amount of disk space a user can use.


## Further Readings

https://developer.mozilla.org/en-US/docs/Web/HTML/Guides/Responsive_images
https://www.smashingmagazine.com/2014/05/responsive-images-done-right-guide-picture-srcset/
https://www.smashingmagazine.com/2016/01/responsive-image-breakpoints-generation/
https://www.smashingmagazine.com/2016/09/automating-art-direction-with-the-responsive-image-breakpoints-generator/
https://cloudfour.com/thinks/responsive-images-101-part-9-image-breakpoints/
https://cloudfour.com/thinks/sensible-jumps-in-responsive-image-file-sizes/

https://cloudfour.com/thinks/the-real-conflict-behind-picture-and-srcset/

## License

This part of the code will be licensed under the terms of the [MIT License](https://opensource.org/licenses/MIT).
