Metadata-Version: 2.4
Name: ichec_django_core
Version: 5.1.0
Summary: Library of base Django app building blocks and utilities.
Author-email: Irish Centre for High End Computing <platformengineering@ichec.ie>
License: MIT
Project-URL: Repository, https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core
Project-URL: Homepage, https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core
Keywords: Web Application,Django
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Requires-Python: >=3.14
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: django>=6.0
Requires-Dist: markdown
Requires-Dist: pillow
Requires-Dist: pyyaml
Requires-Dist: pydantic
Requires-Dist: djangorestframework>=3.18
Requires-Dist: django-filter
Requires-Dist: django-countries
Requires-Dist: django-cors-headers
Requires-Dist: django-downloadview
Requires-Dist: django-prometheus>=2.5
Requires-Dist: mozilla-django-oidc
Requires-Dist: drf-spectacular>=0.30.0
Requires-Dist: django-fsm-2>=4.0
Requires-Dist: django-axes>=8
Requires-Dist: django-health-check>=4
Provides-Extra: async
Requires-Dist: celery>=5.3; extra == "async"
Requires-Dist: redis>=5.0; extra == "async"
Provides-Extra: container
Requires-Dist: psycopg2-binary; extra == "container"
Requires-Dist: gunicorn; extra == "container"
Provides-Extra: s3
Requires-Dist: django-storages[s3]; extra == "s3"
Provides-Extra: keycloak
Requires-Dist: python-keycloak; extra == "keycloak"
Provides-Extra: workflows
Requires-Dist: django-simple-history>=3.13.0; extra == "workflows"
Provides-Extra: types
Requires-Dist: types-requests; extra == "types"
Provides-Extra: sentry
Requires-Dist: sentry-sdk[django]>=2; extra == "sentry"
Dynamic: license-file

# ICHEC Django Core

`ichec-django-core` is the base for ICHEC's Django web apps. It gives your app
secure default settings, member and organisation models, sign-in through
Keycloak, and a REST API for all of these. You write the parts that are specific
to your app.

The [reference app](https://git.ichec.ie/platform-engineering/infrastructure/reference-app)
is a complete app built on it, deployed with the `ichec.platform` Ansible
collection.

## What you get

- **Members and organisations.** Every Django user is a `Member`. Members
  belong to organisations and groups, get notifications, and can export or
  erase their personal data.
- **Sign-in.** Keycloak through OpenID Connect, Django's own login for local
  work, and personal API tokens for scripts.
- **A REST API.** Each model has an endpoint under `/api/`, with an OpenAPI 3.1
  schema, ETags, and errors as RFC 9457 problem details.
- **Access rules.** Each model declares once who may do what with it. Every
  list, check and `available_actions` field comes from that declaration.
- **Status changes.** A model's status changes only through named transitions.
  Each one is its own endpoint, such as `POST /api/applications/{id}/submit/`.
- **Files.** Uploads, image thumbnails, and delivery through Django, nginx or
  S3, with resumable uploads through tusd.
- **Forms.** An opt-in app for forms defined by JSON Schema, with drafts and
  file answers.
- **Workflows.** An opt-in app for review flows held as data, so each call
  sets its own steps: reviews, decisions, waits and checklists, with ranking.

## Try the example app

The [app](./app) directory is a minimal portal built on the library. Run it
locally to see what you get before you write any code. You need
[uv](https://docs.astral.sh/uv/) (`brew install uv`).

```shell
git clone https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core.git
cd ichec-django-core
uv run tox exec -e dev -- python manage.py migrate
uv run tox exec -e dev -- python manage.py createsuperuser
uv run tox -e dev
```

Sign in at <http://localhost:8000/accounts/login/> as the admin you created,
then open <http://localhost:8000/api/> to browse the API. The `dev` env in
[pyproject.toml](./pyproject.toml) holds the development settings.

## Use it in your app

This guide assumes you can build a basic Django app. If not, work through the
[Django tutorial](https://www.djangoproject.com/start/) first.

Add the library to your dependencies:

```shell
uv add ichec-django-core
```

### Settings

Import the library's settings, then set the few values that depend on your
project layout. Your settings file then looks like [app/settings.py](./app/settings.py):

```python
from pathlib import Path

from ichec_django_core import settings
from ichec_django_core.settings import *

BASE_DIR = Path(__file__).resolve().parent.parent

ROOT_URLCONF = "app.urls"
WSGI_APPLICATION = "app.wsgi.application"
ASGI_APPLICATION = "app.asgi.application"

TEMPLATES = settings.get_templates(BASE_DIR)
DATABASES = settings.get_databases(BASE_DIR)
STATIC_ROOT = settings.get_static_root(BASE_DIR)
MEDIA_ROOT = settings.get_media_root(BASE_DIR)
```

Override any other setting below the import.

The library reads its settings from environment variables. Django won't start
without `DJANGO_SECRET_KEY` and `DJANGO_ALLOWED_HOSTS`, or without
`DJANGO_SQL_PASSWORD` when it uses a database server rather than SQLite.
For local work, set them in a tox env, as the `dev` env in
[pyproject.toml](./pyproject.toml) does. In production, the deployment passes
them to the container.
[settings.py](./src/ichec_django_core/settings.py) lists every variable and its
default.

Logs go to the console, which a container sends to journald. To also write a
rotating log file, set `DJANGO_LOG_ROOT` to a directory the app can write. If it
can't, the app warns and logs to the console only.

Django's own pages send a strict Content Security Policy. Set
`DJANGO_CSP_REPORT_ONLY=1` to try it first. `settings.py` shows how to extend it.

### URLs

Register the library's API views on your own router, and include its other views,
as in [app/urls.py](./app/urls.py):

```python
from django.urls import include, path
from rest_framework import routers

from ichec_django_core.urls import register_drf_views

router = routers.DefaultRouter()
register_drf_views(router)

urlpatterns = [
    path("api/", include(router.urls)),
    path("", include("ichec_django_core.urls")),
]
```

Register your app's own viewsets on the same router, so the whole API sits under
one `/api/` root.

Errors come back as RFC 9457 problem details, `application/problem+json`, with
a JSON Pointer to each invalid field. Set the error handlers too, so errors on
`/api/` paths outside DRF's views do the same:

```python
handler404 = "ichec_django_core.views.errors.not_found"
handler500 = "ichec_django_core.views.errors.server_error"
```

`/health/live/` and `/health/` serve liveness and readiness checks. Extend
`HEALTH_CHECKS` to add your own.

## Say who may do what

Each model declares an `Access` policy: the roles someone can have towards a
row, then the roles that may take each action. An action the policy leaves out
is refused to everyone.

```python
from django.db import models

from ichec_django_core.access import HasPermission, Owner, Policy


class Application(models.Model):
    class Access(Policy):
        applicant = Owner("applicant")
        manager = HasPermission("change_application")

        view = applicant | manager
        change = applicant.when(status="draft")
        submit = applicant
        extra_actions = ("submit",)
```

A viewset built on `BaseModelViewSet` lists only the rows the caller may view,
and refuses the actions the policy doesn't allow them. `policy_for(Application).table()`
prints the rules in words, for review and for tests. `register` sets the policy
of a model you don't own, such as the forms app's `PopulatedForm`. To hide a
field from callers who may see the row, such as reviewer notes from the
applicant, list it in the policy's `restricted_fields` with the action that
shows it, and add `RestrictedFieldsMixin` to the serializer.
[access.py](./src/ichec_django_core/access.py) lists the role types.

A status changes through a django-fsm-2 `@transition` method. The policy says
who may run it, and the transition says from which states. Add
`TransitionsMixin` to the viewset and list the transitions in `transitions`,
and each becomes an endpoint. `run_transition` is the one path for every
trigger, whether it comes from the API, a scheduled task or an event. See
[transitions.py](./src/ichec_django_core/transitions.py).

## Sign in with Keycloak

The example app uses Django's own login, which lets anyone register. Real
deployments sign users in through Keycloak with OpenID Connect (OIDC).

Register your app as a client in the Keycloak realm, then set:

```shell
WITH_OIDC=1
OIDC_RP_CLIENT_ID=my-app
OIDC_RP_CLIENT_SECRET=...
OIDC_OP_AUTHORIZATION_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/userinfo
OIDC_OP_JWKS_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/certs
OIDC_OP_LOGOUT_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/logout
```

Leave `DJANGO_WITH_USER_LOGIN` unset in production. The built-in login locks out
repeated failed sign-ins, and Django must only be reachable through nginx.
To sign in through a Keycloak on your machine, with a `Test` realm on port 8080,
run `uv run tox -e dev-oidc`.
[mozilla-django-oidc](https://mozilla-django-oidc.readthedocs.io/en/stable/installation.html)
does the OIDC work, and its docs explain each setting.

## Optional features

Each of these is off until you turn it on.

- **Background tasks.** Emails and Keycloak syncs run inline by default. To run them
  on a Celery worker, install the `async` extra and set
  `DJANGO_TASK_BACKEND=celery`, `CELERY_BROKER_URL` and `CELERY_RESULT_BACKEND`.
  Then call `create_celery_app()` in your app's `celery.py`, as in
  [app/celery.py](./app/celery.py), and run `celery -A your_app worker`. To
  run your own functions the same way, decorate them with
  `@background_task("your_app.task_name")` from `ichec_django_core.tasks`.
- **Error tracking.** Install the `sentry` extra and set `SENTRY_DSN`. Reports
  carry no personal data.
- **File storage.** Uploaded files are served by Django by default. Set
  `DJANGO_DOWNLOAD_DELIVERY` to `xaccel` to have nginx serve them, or to `s3` to
  keep them in object storage. The comments in
  [settings.py](./src/ichec_django_core/settings.py) explain each mode.
- **Resumable uploads.** Large files can go to a [tusd](https://github.com/tus/tusd)
  sidecar, which resumes them after a dropped connection. Set
  `DJANGO_TUS_UPLOADS=1`, route `/api/files/tus/` to tusd, and run it with
  `-base-path /api/files/tus/ -behind-proxy -hooks-http <app>/api/tus_hook/
  -hooks-enabled-events pre-create,pre-finish`. tusd must write where Django
  reads: `-upload-dir` is `$MEDIA_ROOT/uploads/tus`, or on S3 set
  `-s3-object-prefix uploads/tus/`. The final PATCH returns the upload, as
  `POST /api/files/` does. Browsers must send their `X-CSRFToken` header to tusd.
  The manifest's `tus_uploads` tells clients whether tusd is there.
- **Forms.** Forms defined by JSON Schema, with drafts and file answers. Add
  `"ichec_django_core.forms"` to `INSTALLED_APPS`, and `register_drf_views` and
  the library's URLs include the forms API. The model that owns a form, such as
  an application, submits it. Register a policy for `PopulatedForm` to say who
  sees and edits answers, such as
  `change = Owner("application.applicant").when(application__status="draft")`,
  and one for `Form` to say who downloads its templates. See
  [forms/services.py](./src/ichec_django_core/forms/services.py).

  Forms used to be part of the core app. If one of your models links to
  a form, point the link at `ichec_forms` in a migration that depends on
  `("ichec_forms", "0001_initial")` and sets
  `run_before = [("ichec_django_core", "0036_move_forms_to_ichec_forms")]`.
- **Workflows.** Each call runs its applications through a flow held as
  data, so a portal changes a call's review without a release. Install the
  `workflows` extra, and add `"simple_history"`, `"ichec_django_core.forms"`
  and `"ichec_django_core.workflows"` to `INSTALLED_APPS`.

  Add `WorkflowOwnerMixin` to the model that owns a flow, such as a call, and
  `WorkflowSubjectMixin` to the model that goes through it, such as an
  application. Set `owner_field` on the subject to its foreign key to the
  owner. Serve their transitions, such as `submit`, `close`, `reopen` and
  `move`, with `TransitionsMixin`. `workflows/serializers.py` has the request
  bodies for the ones that take a date or a reason.

  A flow is a list of steps of four types: `collect` a form from each person
  in a role, `decide` an outcome, `wait` for an event or a date, and work
  through a `checklist`. Each step's `role` is a path from the subject to
  members, such as `call.board_members`. Coordinators edit flows at
  `/api/flows/` and `/api/steps/`, and assignees work their tasks at
  `/api/tasks/`. Run `python manage.py run_workflow_schedule` every few
  minutes from cron, to close calls at their closing time. To run your own
  code on a decision, such as recording the facility the chair chose, connect
  to `engine.decided`.

  Reviews follow the application, not the reviewer. Make the subject an
  `AnonymisableMixin` with `"tasks"` in `pii_forward_fields`, and erasing it
  erases the reviews of it. A system check fails until you do. Erasing a
  reviewer keeps their reviews and rankings, as the record of a decision, and
  only their member row stops naming them. Both find the reviews in their
  export. See
  [workflows/models.py](./src/ichec_django_core/workflows/models.py) and
  [step_types.py](./src/ichec_django_core/workflows/step_types.py).
- **Keycloak attributes.** The app can copy values such as a member's role or
  avatar onto their Keycloak user, so other apps in the realm can read them.
  Set `KEYCLOAK_ATTRIBUTE_SYNC_ENABLED=1`, install the `keycloak` extra, and
  list the attributes in `KEYCLOAK_ATTRIBUTE_SYNC`. See
  [sso/\_\_init\_\_.py](./src/ichec_django_core/sso/__init__.py).

## Test your app

`ichec_django_core.test` has helpers for your app's tests:

- `AuthAPITestCase` calls your API as each test user.
- `AuthorizationSweepTestCase` calls every endpoint in your schema as each role.
  It checks each view with an access policy against the policy, and the rest
  against a file listing who may call them. For a sweep of a deployed stack,
  `python manage.py sweep_declaration` writes both as one file. The reference
  app shows it in use.
- The checks in `test/privacy.py` prove that erasing a member removes all their
  personal data, including from your app's models.

## Work on this library

Install the development tools and run the checks that CI runs:

```shell
uv sync --group dev --extra async
uv run pytest
uv run ruff format --check src tests
uv run ruff check src tests
uv run mypy src
```

If you use [direnv](https://direnv.net/), run `direnv allow` once and the
environment activates when you enter the directory.

If you change a model or serializer, regenerate the API schema and commit it:

```shell
uv run python manage.py spectacular --file schema.yaml --validate
```

## Licence

Copyright of the Irish Centre for High End Computing (ICHEC), released under the
MIT License. See [LICENSE](./LICENSE).
