Metadata-Version: 2.4
Name: ichec_django_core
Version: 8.4.0
Summary: Library of base Django app building blocks and utilities.
Author-email: Irish Centre for High End Computing <platformengineering@ichec.ie>
License: MIT
Project-URL: Repository, https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core
Project-URL: Homepage, https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core
Keywords: Web Application,Django
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Requires-Python: >=3.14
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: django>=6.0.9
Requires-Dist: markdown
Requires-Dist: pillow
Requires-Dist: pyyaml
Requires-Dist: pydantic
Requires-Dist: djangorestframework>=3.18
Requires-Dist: django-filter
Requires-Dist: django-countries
Requires-Dist: django-cors-headers
Requires-Dist: django-downloadview
Requires-Dist: django-prometheus>=2.5
Requires-Dist: prometheus-client>=0.20
Requires-Dist: mozilla-django-oidc
Requires-Dist: drf-spectacular>=0.30.0
Requires-Dist: django-fsm-2>=4.0
Requires-Dist: django-axes>=8
Requires-Dist: django-health-check>=4
Requires-Dist: nh3>=0.2.18
Requires-Dist: pypdf[crypto]>=5
Provides-Extra: async
Requires-Dist: celery>=5.3; extra == "async"
Requires-Dist: redis>=5.0; extra == "async"
Provides-Extra: container
Requires-Dist: psycopg2-binary; extra == "container"
Requires-Dist: gunicorn; extra == "container"
Provides-Extra: s3
Requires-Dist: django-storages[s3]; extra == "s3"
Provides-Extra: keycloak
Requires-Dist: python-keycloak; extra == "keycloak"
Provides-Extra: codegen
Requires-Dist: datamodel-code-generator[ruff]==0.83.0; extra == "codegen"
Provides-Extra: workflows
Requires-Dist: django-simple-history>=3.13.0; extra == "workflows"
Provides-Extra: types
Requires-Dist: types-requests; extra == "types"
Provides-Extra: sentry
Requires-Dist: sentry-sdk[django]>=2; extra == "sentry"
Dynamic: license-file

# ICHEC Django Core

`ichec-django-core` is the base for ICHEC's Django web apps. It gives your app
secure default settings, member and organisation models, sign-in through
Keycloak, and a REST API for all of these. You write the parts that are specific
to your app.

The [reference app](https://git.ichec.ie/platform-engineering/infrastructure/reference-app)
is a complete app built on it, deployed with the `ichec.platform` Ansible
collection.

![ichec-django-core's building blocks: workflows, forms, files, people, sign-in and background tasks over a secure foundation, all through a standards-based REST API](docs/diagrams/ichec-django-core.svg)

## Try the example app

The [app](./app) directory is a minimal portal built on the library. Run it
locally to see what you get before you write any code. You need
[uv](https://docs.astral.sh/uv/) (`brew install uv`).

```shell
git clone https://git.ichec.ie/platform-engineering/modules/web/ichec-django-core.git
cd ichec-django-core
uv run tox exec -e dev -- python manage.py migrate
uv run tox exec -e dev -- python manage.py createsuperuser
uv run tox -e dev
```

Sign in at <http://localhost:8000/accounts/login/> as the admin you created,
then open <http://localhost:8000/api/> to browse the API. The `dev` env in
[pyproject.toml](./pyproject.toml) holds the development settings.

## Use it in your app

This guide assumes you can build a basic Django app. If not, work through the
[Django tutorial](https://www.djangoproject.com/start/) first.

Add the library to your dependencies:

```shell
uv add ichec-django-core
```

### Settings

Import the library's settings, then set the few values that depend on your
project layout. Your settings file then looks like [app/settings.py](./app/settings.py):

```python
from pathlib import Path

from ichec_django_core import settings
from ichec_django_core.settings import *

BASE_DIR = Path(__file__).resolve().parent.parent

ROOT_URLCONF = "app.urls"
WSGI_APPLICATION = "app.wsgi.application"
ASGI_APPLICATION = "app.asgi.application"

TEMPLATES = settings.get_templates(BASE_DIR)
DATABASES = settings.get_databases(BASE_DIR)
STATIC_ROOT = settings.get_static_root(BASE_DIR)
MEDIA_ROOT = settings.get_media_root(BASE_DIR)
```

Override any other setting below the import.

The library reads its settings from environment variables, which
[settings.py](./src/ichec_django_core/settings.py) lists with their defaults.
Django won't start without `DJANGO_SECRET_KEY` and `DJANGO_ALLOWED_HOSTS`. For
local work, set them in a tox env, as the `dev` env in
[pyproject.toml](./pyproject.toml) does. In production, also set
`DJANGO_PORTAL_URL` and `DJANGO_DEFAULT_FROM_EMAIL`: a deploy check fails
without them.

### Scheduled commands

Run these on a timer. The `webapp_django` role of the ICHEC Ansible collection
does it for you.

| Command | When |
|---|---|
| `retry_emails` | every 5 minutes |
| `run_workflow_schedule` | every 5 minutes, with workflows |
| `send_notification_digests` | daily |
| `purge_uploads` | daily |

### Logs

The app logs JSON lines, each with the ID of the request that wrote it. It logs
sign-ins, refused requests, API token use, refused uploads and admin changes
as security events, each with a stable name to alert on.
[docs/logging.md](./docs/logging.md) lists the fields and the event names.

### Languages

Every string the library shows people is marked for translation, and its
catalogue is in `src/ichec_django_core/locale`. The API answers in the
language a request asks for with `Accept-Language`, of those in `LANGUAGES`.
Each member can choose a `language`, and their emails and notifications are
in it. An email to several people is in the portal's default language.

To add a language, add it to `LANGUAGES`, and translate both the library's
strings and your app's. Add the library's translation to the library, in
`locale/<code>/LC_MESSAGES/django.po`, so every portal gets it: the `en`
catalogue lists the strings. For your app's own strings, mark them with
`gettext` and `{% translate %}`, and run `python manage.py makemessages -l
<code>` in your app. `compilemessages` builds both catalogues.

### URLs

Register the library's API views on your own router, and include its other
views, as in [app/urls.py](./app/urls.py):

```python
from django.urls import include, path
from rest_framework import routers

from ichec_django_core.urls import register_drf_views

router = routers.DefaultRouter()
register_drf_views(router)

urlpatterns = [
    path("api/", include(router.urls)),
    path("", include("ichec_django_core.urls")),
]

handler404 = "ichec_django_core.views.errors.not_found"
handler500 = "ichec_django_core.views.errors.server_error"
```

Register your own viewsets on the same router, so the whole API sits under one
`/api/` root. If Django serves your Angular app, add `spa_route()` from
[views/spa.py](./src/ichec_django_core/views/spa.py) last.

### Say who may do what

Each model declares who may do what with it, once:

```python
from django.db import models

from ichec_django_core.access import HasPermission, Owner, Policy


class Application(models.Model):
    class Access(Policy):
        applicant = Owner("applicant")
        manager = HasPermission("change_application")

        view = applicant | manager
        change = applicant.when(status="draft")
```

A viewset built on `SearchableModelViewSet` then lists only the rows the
caller may see, and refuses what the policy doesn't allow.
[docs/access.md](./docs/access.md) covers status changes, roles and hidden
fields, and [docs/members.md](./docs/members.md) who sees whom.

### Generate your app's client

Your API's schema, `schema.yaml`, and a typed Python client come from your
code. Install the `codegen` extra, give your app a `client` package, and run:

```shell
python manage.py generate_client myapp.client
```

Commit what it writes. In CI, add `--check` to fail when it's out of date.

## Sign in with Keycloak

The example app uses Django's own login, which lets anyone register. Real
deployments sign users in through Keycloak with OpenID Connect (OIDC).

Register your app as a client in the Keycloak realm, then set:

```shell
WITH_OIDC=1
OIDC_RP_CLIENT_ID=my-app
OIDC_RP_CLIENT_SECRET=...
OIDC_OP_AUTHORIZATION_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/userinfo
OIDC_OP_JWKS_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/certs
OIDC_OP_LOGOUT_ENDPOINT=https://keycloak.example.org/realms/my-realm/protocol/openid-connect/logout
```

Leave `DJANGO_WITH_USER_LOGIN` unset in production. The built-in login locks out
repeated failed sign-ins, and Django must only be reachable through nginx.
To sign in through a Keycloak on your machine, with a `Test` realm on port 8080,
run `uv run tox -e dev-oidc`.
[mozilla-django-oidc](https://mozilla-django-oidc.readthedocs.io/en/stable/installation.html)
does the OIDC work, and its docs explain each setting.

## Optional features

Each of these is off until you turn it on. Each link says how.

- **Forms** defined by JSON Schema, with drafts and file answers:
  [docs/forms.md](./docs/forms.md).
- **Workflows**, review flows held as data, so each call configures its own:
  [docs/workflows.md](./docs/workflows.md).
- **Background tasks** on Celery: [app/celery.py](./app/celery.py) and
  [tasks](./src/ichec_django_core/tasks/__init__.py).
- **Facet counts** beside a list's results:
  [views/facets.py](./src/ichec_django_core/views/facets.py).
- **File storage** through nginx or S3, **resumable uploads** through tusd,
  and **malware scanning** with ClamAV: [docs/files.md](./docs/files.md).
- **Email delivery records**, retries and metrics:
  [docs/email.md](./docs/email.md).
- **Error reporting** from the server and the browser app: set `SENTRY_DSN`
  and `DJANGO_ERROR_REPORTING_DSN`.
- **Branding** from an [ichec-brand](https://git.ichec.ie/platform-engineering/modules/ichec-brand)
  pack: set `DJANGO_BRAND_PACK` to the pack's directory. The manifest, the
  templates and the emails get its name, logos, tokens and header and footer
  hints, and its images are static files under `brand/`. An app that sets its own `STATICFILES_FINDERS`
  adds `ichec_django_core.brand.BrandFinder`.
  [brand.py](./src/ichec_django_core/brand.py) says how a pack is read.
- **Keycloak attributes** copied onto each member's Keycloak user:
  [sso/\_\_init\_\_.py](./src/ichec_django_core/sso/__init__.py).

## Test your app

`ichec_django_core.test` has helpers for your app's tests:

- `AuthAPITestCase` calls your API as each test user.
- `PolicyEnforcementTestCase` checks your API against each model's policy.
- `AuthorizationSweepTestCase` calls every endpoint as each role.
- `ListQueryBudgetTestCase` fails any list that asks the database once per row.
- `test/privacy.py` checks that erasing a member removes their personal data.
- `assert_emails_render` fails on an email template that names a variable its
  context lacks. [docs/email.md](./docs/email.md) says how to use it.
- A pytest plugin, which loads by itself. It hashes test passwords with MD5,
  gives each run its own temporary `MEDIA_ROOT`, and fails a test that leaves a
  file, socket, database connection or thread open. Under xdist, it migrates
  one test database and copies it for each worker, and it can spread a long
  parametrized test, such as an API conformance test, across the workers. The
  options are in `src/ichec_django_core/pytest_plugin.py`.

[docs/e2e.md](./docs/e2e.md) runs a user journey both in-process and against a
deployed stack.

## Work on this library

Install the development tools and run the checks that CI runs:

```shell
uv sync --group dev --extra async
uv run pytest
uv run ruff format --check .
uv run ruff check .
uv run mypy src
```

The tests run on SQLite locally, and on Postgres 18 in CI, as production
runs it. Postgres finds faults that SQLite hides, such as a foreign key to a
table that doesn't exist yet. To run them on Postgres, start a server and
run the `postgres` env:


```shell
podman run -d --rm --name ichec-test-postgres -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:18
uv run tox -e postgres
```

`DJANGO_SQL_HOST`, `DJANGO_SQL_PORT`, `DJANGO_SQL_USER` and
`DJANGO_SQL_PASSWORD` point it at another server.

If you use [direnv](https://direnv.net/), run `direnv allow` once and the
environment activates when you enter the directory.

The `messages` check needs GNU gettext (`brew install gettext`). If you add or
change a string people read, mark it for translation, and update the
catalogue with `uv run tox -e messages`, and commit it. An app can check its
own catalogue the same way, with `python manage.py check_messages -l <code>`
run from its package.

If you change a model or serializer, regenerate the API schema and client, and
commit them:

```shell
uv run python manage.py generate_client ichec_django_core.client
```

## Licence

Copyright of the Irish Centre for High End Computing (ICHEC), released under the
MIT License. See [LICENSE](./LICENSE).
