Metadata-Version: 2.4
Name: dtaas
Version: 2.2.0
Summary: DTaaS CLI
License: INTO-CPS-Association
Author: Astitva Sehgal
Requires-Python: >=3.10,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Requires-Dist: PyYAML (>=6.0.3,<7.0.0)
Requires-Dist: click (>=8.5.0,<9.0.0)
Requires-Dist: cryptography (>=50.0.2,<51.0.0)
Requires-Dist: email-validator (>=2.3.0,<3.0.0)
Requires-Dist: fqdn (>=1.6.0,<2.0.0)
Requires-Dist: python-gitlab (>=8.6.0,<9.0.0)
Requires-Dist: python-on-whales (>=0.81.0,<0.82.0)
Requires-Dist: requests (>=2.34.2,<3.0.0)
Requires-Dist: tomlkit (>=0.15.1,<0.16.0)
Description-Content-Type: text/markdown

# DTaaS Command Line Interface

Command-line tool for the
[INTO-CPS-Association](https://github.com/INTO-CPS-Association/DTaaS)
Digital Twin as a Service platform. Use it to generate deployment projects,
manage users, and operate a running DTaaS instance.  All from a single `dtaas`
entry point.

---

## Quick Start

From a clean machine to a running DTaaS deployment in seven steps.

```bash
# 1. Create and activate a virtual environment (recommended)
python -m venv .venv && source .venv/bin/activate   # Linux / macOS
# python -m venv .venv && .venv\Scripts\activate    # Windows

# 2. Install the package
pip install dtaas

# 3. Generate dtaas.toml + a sample users.csv to fill in
dtaas config generate

# 4. Open dtaas.toml and fill in your server DNS, paths, and credentials
#    (see Configuration Reference below for all fields)

# 5. Validate the configuration fix any reported errors before continuing
dtaas config validate

# 6. Generate deployment files for your chosen scenario
dtaas deployment generate --type secure-server

# 7. Bring the deployment up
dtaas platform install

# Tear it down when done
dtaas platform uninstall
```

> **Deployment type cheat-sheet**
>
> | `--type` | When to use |
> |---|---|
> | `localhost` | Local dev / demo only |
> | `insecure-server` | Multi-user HTTP demo: **not internet-facing** |
> | `secure-server` | Multi-user HTTPS: production-ready |
> | `secure-server-gitlab` | HTTPS + bundled GitLab: production-ready |
> | `workspace-localhost` | Workspace + Dex on localhost |
> | `workspace-secure-server` | Workspace + Keycloak: production-ready |

---

## Table of Contents

- [DTaaS Command Line Interface](#dtaas-command-line-interface)
  - [Quick Start](#quick-start)
  - [Table of Contents](#table-of-contents)
  - [Installation](#installation)
  - [Commands](#commands)
    - [config](#config)
    - [deployment generate](#deployment-generate)
      - [Configuration substitution](#configuration-substitution)
      - [TLS certificate placement](#tls-certificate-placement)
    - [platform install](#platform-install)
    - [platform uninstall](#platform-uninstall)
    - [Lifecycle operations: status / stop / start / pause / resume](#lifecycle-operations)
    - [platform update --certs](#platform-update---certs)
    - [platform update --config](#platform-update---config)
    - [user add](#user-add)
    - [user delete](#user-delete)
    - [user pause / stop / resume](#user-pause--stop--resume)
    - [config reconcile](#config-reconcile)
  - [User files](#user-files)
  - [Configuration Reference dtaas.toml](#configuration-reference-dtaastoml)
    - [Which sections does my deployment need?](#which-sections-does-my-deployment-need)
    - [Annotated dtaas.toml](#annotated-dtaastoml)

---

## Installation

Installation inside a virtual environment is strongly recommended to avoid
conflicts with system-wide packages.

```bash
python -m venv .venv
source .venv/bin/activate     # Linux / macOS
# .venv\Scripts\activate      # Windows

pip install dtaas
```

Verify the install:

```bash
dtaas --help
dtaas --version   # which build is installed, the first thing to pin down
                  # when a command behaves unexpectedly
```

---

## Commands

### config

Manage `dtaas.toml` independently of the rest of the project. This is the
first step in the setup workflow generate a template, fill it in, then
validate before running any other command.

#### Generate a fresh template

```bash
dtaas config generate
```

This writes `dtaas.toml` and a sample `users.csv` (bulk input for
`dtaas user add --file`) into the target directory.

#### Validate an existing file

```bash
dtaas config validate
```

`validate` reads `dtaas.toml` (from `--output-dir` first, then the current
directory) and reports all problems at once:

| Field | Rule |
|---|---|
| `git-repo` | Must be an `http(s)` URL |
| `[common].server-dns` | Must be `localhost`, an IP, or a fully qualified hostname |
| `[common].path` | Must be an absolute path to an existing directory |
| `[common.security].certs-src` | When present, must be an absolute path to an existing directory |
| `[common.resources].set_limits` | `false` turns the limits off; any other value, or no key, keeps them on |
| `[common.resources].cpus` | Positive number (e.g. `4` or `0.5`) |
| `[common.resources].pids_limit` | Integer |
| `[common.resources].mem_limit`, `shm_size` | Byte size with required unit (e.g. `4G`, `512m`) |
| `[[users]]` | When present, must be an array of tables; usernames must be unique |
| `[[users]].username` | Required, valid username |
| `[[users]].email` | Required, valid RFC 5321/5322 address (no DNS lookup) |
| `[[users]].groups` | When present, must be a list of strings |
| `[[users]].load_balance` | When present, must be `true` or `false` |
| `[[users]].password` | When present, must be a string (the starting user's initial GitLab password, used when `user add` names them) |
| `[gitlab].provision` | When present, `true` or `false` (default `false`) |
| `[gitlab].api_url` | Must be an `http(s)` URL; **required** when `provision` is `true` |
| `[gitlab].pat` | When present, must be non-empty (remove the key to use `DTAAS_GITLAB_PAT`) |
| `[gitlab].ssl_verify` | When present, `true`, `false`, or a CA-bundle path string |
| Deployment-section URLs | When present, must be `http(s)` URLs |
| Deployment-section `default-user` | When present, must be a valid username |

Deployment-section URLs are `react-app-oauth-url`, `oauth-url`,
`auth-authority`, and `keycloak-issuer-url` across `[frontend]`,
`[localhost]`, `[insecure-server]`, `[secure-server]`,
`[secure-server-gitlab]`, `[workspace-localhost]`, and
`[workspace-secure-server]`; each is checked only when its section is
present. `config validate` checks every section in the file, while
`platform update --config` checks only the installed type's section, plus
`[frontend]`.

`path` and `certs-src` are checked against the local filesystem, run
`validate` on the deployment host. If a check fails with a permission error
(e.g. `certs-src` points under `/etc/letsencrypt`, which is root-owned), the
CLI says so and prints the elevated form to re-run:
`sudo -E env PATH="$PATH" dtaas <command>`.

The `[common.resources]` limit fields (`cpus`, `pids_limit`, `mem_limit`,
`shm_size`) are required only when `set_limits` is `true` (the default). With
`set_limits = false` they are optional and ignored; any value still present is
validated.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | For `generate`: target directory (created if missing). For `validate`: search location |
| `--force` | off | (`generate` only) Overwrite an existing `dtaas.toml` |

---

### deployment generate

Copies the full project structure for a specific deployment scenario
(`docker-compose.yml`, config examples, and supporting files) into a target
directory ready to be customised. It also writes the user workspace
templates that `user add` builds new workspaces from: `users.server.yml`,
`users.server.secure.yml`, `users.resources.yml`, and the `files/template/`
skeleton.

```bash
dtaas deployment generate --type <name>
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--type NAME` | *(required)* | Deployment scenario (see table below) |
| `--output-dir PATH` | `.` | Target directory (created if missing) |
| `--force` | off | Overwrite files that already exist |

#### Available types

| `--type` | Deployment scenario | Support level |
|---|---|---|
| `localhost` | Single-machine Docker deployment | dev/demo only |
| `insecure-server` | Multi-user HTTP server | insecure/demo only |
| `secure-server` | Multi-user HTTPS/TLS server | **production-supported** |
| `secure-server-gitlab` | HTTPS/TLS + integrated GitLab | **production-supported** |
| `workspace-localhost` | Workspace service with Dex on localhost | dev/demo only |
| `workspace-secure-server` | Workspace service with Keycloak | **production-supported** |

> **Warning** Types marked *dev/demo only* or *insecure/demo only* run
> over plain HTTP with default or static credentials. **Do not expose them to
> the internet or shared networks.** Production-supported types still require
> manual hardening steps documented in the `README.md` and configuration
> notes (`CONFIG.md`, or `CONFIGURATION.md` for `workspace-secure-server`)
> shipped with each generated project.

**Examples:**

```bash
# Localhost demo in the current directory
dtaas deployment generate --type localhost

# Production HTTPS server in a subdirectory
dtaas deployment generate --type secure-server --output-dir ./my-server

# Regenerate, overwriting existing files
dtaas deployment generate --type insecure-server --output-dir ./demo --force
```

#### Configuration substitution

When `dtaas.toml` is present, `deployment generate` reads
deployment-specific values from it and substitutes them into the generated
files automatically.

The CLI searches `--output-dir` first, then falls back to the current working
directory. Each `--type` reads from its matching top-level section in
`dtaas.toml`. Values are written into dotenv files (`config/.env`,
`config/conf.server`) and the React client config (`config/client.js`).

The `[frontend]` section supplies `REACT_APP_CLIENT_ID` and
`REACT_APP_AUTH_AUTHORITY` for the DTaaS web client, these are a separate
OAuth application from the traefik-forward-auth credentials configured in
`[insecure-server]` / `[secure-server]`. The `[common]` and `[[users]]`
sections are substituted across all types.

If `dtaas.toml` is not found, a note is printed and generated files keep
their default placeholder values.

#### TLS certificate placement

For the TLS types (`secure-server`, `secure-server-gitlab`,
`workspace-secure-server`), `deployment generate` also populates the
`certs/` directory in the output. It reads `[common.security].certs-src` from
`dtaas.toml` and copies the latest `fullchain.pem` and `privkey.pem` there.

> **Tip: verify the Docker image tag**
> `users.server.yml` and `users.server.secure.yml` contain a pinned workspace
> image tag (e.g. `intocps/workspace:main-56c6f68`). Check
> [Docker Hub](https://hub.docker.com/r/intocps/workspace/tags) and update the
> tag to a current, stable version before deploying.

---

### platform install

Brings a generated deployment up with a single command.

```bash
dtaas platform install
```

Internally runs `docker compose up -d` against the `docker-compose.yml` in
the installation directory. Before starting, it ensures per-user workspace
directories for every `[[users]]` record exist, recreating each from
`files/template/` if missing and sets ownership to `1000:100`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory containing the generated deployment |

The CLI looks for `dtaas.toml` in `--output-dir` first, then the current
working directory, so a single top-level `dtaas.toml` can serve a deployment
generated into a subdirectory:

```bash
dtaas platform install --output-dir ./insecure
```

The command fails with a clear error if `docker-compose.yml` is missing,
`dtaas.toml` cannot be found, or the Docker daemon is unreachable.

---

### platform uninstall

Tears the deployment down, stopping and removing containers and networks.

```bash
dtaas platform uninstall
```

User containers added with `user add` run as a separate Compose project;
they are torn down first so they do not hold the shared network open.
**Per-user workspace files are preserved by default.**

To also delete the generated per-user workspace directories **and** the
CLI-owned `dtaas.users.registry.json` / `.dtaas.state.json`:

```bash
dtaas platform uninstall --remove-user-files
```

This is destructive, so the command prompts for confirmation. Skip the prompt
in non-interactive scripts with `--yes`:

```bash
dtaas platform uninstall --remove-user-files --yes
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |
| `--remove-user-files` | off | Also delete per-user workspace dirs + registry/state files |
| `--yes` / `-y` | off | Skip the confirmation prompt for `--remove-user-files` |

> `--remove-user-files` removes only the per-user directories inside
> `<output-dir>/files/`, preserving `files/common/` and `files/template/` so
> a later `platform install` can recreate user directories. It refuses to follow
> a symlinked `files/`. Double-check `--output-dir` before using this flag.

---

### Lifecycle operations

Operational controls for an **already-installed** deployment: observe it with
`status`, and suspend or resume it with `stop`/`start` and `pause`/`resume`.
None of these remove containers or networks, that is `uninstall`'s job. They
exit `0` on success (including the idempotent "nothing installed" case) and
non-zero on failure, so they are safe to call from CI/ops scripts.

`platform stop`/`start`/`pause`/`resume` act on the **core services only** —
they never touch per-user containers. Suspend or resume individual additional
users with [`dtaas user stop`/`pause`/`resume`](#user-pause--stop--resume)
instead. `platform status`, being read-only, still reports the whole
installation (core services **and** user containers).

**Lifecycle command matrix:**

| Command | `docker compose` verb | Effect | Containers kept? | Reverse with |
|---|---|---|:---:|---|
| `platform install` | `up -d` | Create and start every service | n/a | `platform stop` / `platform uninstall` |
| `platform status` | `ps` (read-only) | Report per-service state; no change | n/a | n/a |
| `platform stop` | `stop` | Terminate the processes, keep the containers | yes | `platform start` |
| `platform start` | `start` | Start previously stopped containers | yes | `platform stop` |
| `platform pause` | `pause` | Freeze the processes (memory preserved) | yes | `platform resume` |
| `platform resume` | `unpause` | Thaw previously paused processes | yes | n/a |
| `platform uninstall` | `down` | Stop **and remove** containers and networks | no | `platform install` |

> **`stop` vs `pause`.** `stop` sends `SIGTERM`/`SIGKILL`: processes end, and a
> restart re-runs them from scratch (reverse with `platform start`). `pause` uses
> the kernel cgroup freezer: processes are suspended in place with their memory
> intact and resume instantly (reverse with `platform resume`), but a paused
> container still holds its resources. Use `stop` to free CPU; use `pause` for
> a brief, instantly reversible suspension. `pause` expects running containers
> and will error if the deployment is already stopped.

#### platform status

Reports the state of every service, for both the deployment and user
workloads.

```bash
dtaas platform status
```

```text
PROJECT     SERVICE            STATE
deployment  traefik            running
deployment  client             running
deployment  gitlab             not created
users       alice              paused
```

State values are `running`, `paused`, `stopped` (a terminated container, what
Docker calls `exited`), `restarting`, or `not created` (a service defined in
`docker-compose.yml` that has no container yet).

For automation, `--json` emits the same records as machine-readable JSON:

```bash
dtaas platform status --json
```

```json
[
  {"project": "deployment", "service": "traefik", "state": "running"}
]
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |
| `--json` | off | Emit machine-readable JSON instead of the table |

#### platform stop / platform start

`stop` stops the core services with `docker compose stop`; `start` brings the
stopped containers back with `docker compose start`. Containers and networks
are **kept**, so `stop` is not an uninstall.

```bash
dtaas platform stop
dtaas platform start
```

After `stop`, the command notes how many per-user containers are still
running and suggests `dtaas user stop --all`. Both commands print
`There is no existing DTaaS / Workspace installation` and exit `0` when
nothing is installed (no containers in any state), so they are safe to call
repeatedly.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |

#### platform pause / platform resume

`pause` freezes the running core service containers in place with
`docker compose pause`; `resume` thaws them with `docker compose unpause`.
Memory is preserved and resume is near-instant.

```bash
dtaas platform pause
dtaas platform resume
```

Like `stop`, `pause` notes any per-user containers still running. Both
report the absent-installation case and exit `0` when nothing is installed
(no containers in any state).

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory |

---

### platform update --certs

Rotates TLS certificates of a running deployment in place: no project
regeneration or manual file copying required.

```bash
dtaas platform update --certs
```

The command reads `[common.security].certs-src` from `dtaas.toml`, then:

1. **Validates** the new certificate pair (parseable, private key matches cert,
   no expired intermediates).
2. **Stops** the `traefik` service to release open file handles.
3. **Swaps** the validated files into `<output-dir>/certs/`, backing up the
   live pair and restoring it on any failure.
4. **Restricts** the private key to `0600` (POSIX; prints a warning on Windows).
5. **Restarts** `traefik` and waits for it to come back up.

If validation fails, live certificates are left untouched. The command is safe
to run repeatedly.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--certs` | *(required)* | Refresh the deployment's TLS certificates |
| `--output-dir PATH` | `.` | Installation directory |

---

### platform update --config

Re-applies the values in `dtaas.toml` to an already-installed deployment's
service config files without regenerating the project.

```bash
dtaas platform update --config
```

Treats `dtaas.toml` as the single source of truth, re-runs the same
substitution as `deployment generate`, and if anything changed, recreates all
deployment services with `docker compose up -d --force-recreate`. The
deployment type is auto-detected from `docker-compose.yml`.

```bash
# Preview changes without writing or restarting
dtaas platform update --config --dry-run

# Apply changes and restart
dtaas platform update --config

# Update a deployment in a subdirectory
dtaas platform update --config --output-dir ./my-server
```

`--config` validates `dtaas.toml` before making any changes and refuses to
apply if problems are found. It is **idempotent** a second run with no
`dtaas.toml` changes reports `No configuration changes` and restarts nothing.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--config` | *(required)* | Re-apply `dtaas.toml` to installed service config files |
| `--dry-run` | off | Report what would change without writing or restarting |
| `--output-dir PATH` | `.` | Installation directory |

> `--certs` and `--config` may be combined in a single invocation; at least
> one of them is required.

---

### user add

Provisions users on a running DTaaS instance. Additional users are recorded in
the CLI-owned `dtaas.users.registry.json`
(see [User files](#user-files)), not in `dtaas.toml`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAME` | — | Add one user (requires `--email`, except for a dtaas.toml starting user) |
| `--file PATH` / `-f` | — | Bulk-add users from a CSV |
| `--email TEXT` | — | Email for `USERNAME` (enables forward-auth routing); read from `[[users]]` for a starting user, and another address given for one is reported and not applied |
| `--group TEXT` | `additional` | Group tag for `USERNAME`; repeat the flag for multiple groups, e.g. `--group dtaas --group testers` |
| `--load-balance / --no-load-balance` | on | Mark `USERNAME` for load balancing |
| `--password TEXT` | — | GitLab password for `USERNAME`; only used when GitLab provisioning is enabled (see below). It wins over `[[users]].password` in dtaas.toml. Visible in shell history and the process list, prefer the `users.csv` `password` column (`chmod 600` it) or the interactive prompt for non-interactive/scripted use |

Add a single user:

```bash
dtaas user add --email alice@intocps.org --group dtaas --load-balance alice
```

> Click accepts `--email`, `--group`, and `--load-balance` in any position
> relative to `USERNAME` the form above is the recommended convention,
> matching `useradd [options] LOGIN`.

`--group` is repeatable, not comma-separated pass it once per group to add a
user to multiple groups:

```bash
dtaas user add --email alice@intocps.org --group dtaas --group testers alice
```

Or bulk-add from a CSV:

```bash
dtaas user add --file users.csv
```

`dtaas config generate` writes a sample `users.csv` next to `dtaas.toml`:

```csv
username,email,groups,load_balance,password
alice,alice@intocps.org,additional,true,
bob,bob@intocps.org,additional;beta-testers;load-balanced-tier1,false,
```

`groups` is a `;`-separated list and `load_balance` is `true`/`false`. The
`password` column is used only with GitLab provisioning (see below). Both
forms merge into the registry (never hand-edited) with `desired_status` set to
`running`, then **only the newly-added users are started**: already-running
users are left untouched, so adding one user never recreates the rest. A
username already in the registry is reported (`'alice' already exists,
skipping`) and never added twice or overwritten. A username declared in
`dtaas.toml`'s `[[users]]` is reported as a starting user, whose workspace
`dtaas.toml` manages; with GitLab provisioning on, naming one creates its
GitLab account (see below).

A `USERNAME` or `--file` is required (not both): a bare `dtaas user add`
with neither is rejected rather than silently reprovisioning the whole
registry. A run that added nobody (every name was already registered, or was
a starting user) closes with `No new users added.` and still exits 0, so
re-running the same CSV stays idempotent. What the deployment supports is
checked before anything is registered, so a run that cannot provision (a
localhost installation, or a missing per-user template) leaves no users
registered behind it. To (re)provision **every** registry user at once (for
example after `compose.users.yml` was lost), use `dtaas config reconcile
--fix` instead.

#### GitLab provisioning (optional)

With `[gitlab].provision = true`, `user add` also creates a GitLab account
for each user it adds, and issues that account a Personal Access Token
(PAT). The containers are provisioned first, and a GitLab failure never
undoes them.

**Setup, once per installation.** Turn provisioning on in `dtaas.toml`:

```toml
[gitlab]
provision = true
api_url   = "https://gitlab.example.com"
```

Then export an admin token. It must belong to a GitLab administrator and
carry the `api` scope, plus `admin_mode` when GitLab Admin Mode is on:

```bash
export DTAAS_GITLAB_PAT="glpat-xxxxxxxxxxxxxxxxxxxx"
```

`[gitlab].pat` in `dtaas.toml` is the fallback when the variable is unset.

**Example 1: add one user.** Without `--password`, the CLI asks for the
initial GitLab password with hidden input:

```console
$ dtaas user add --email alice@intocps.org alice
GitLab password for 'alice':
Repeat for confirmation:
GitLab account and token created for 'alice'.
  The token is in gitlab_user_tokens.json.
Users added successfully
```

**Example 2: add users from a CSV.** Fill in the `password` column of the
`users.csv` that `dtaas config generate` writes, and keep the file private:

```csv
username,email,groups,load_balance,password
alice,alice@intocps.org,additional,true,Kq7v-Rm3x-Zt9w
bob,bob@intocps.org,additional;beta-testers,false,Wd4p-Hs8n-Qc2y
```

```console
$ chmod 600 users.csv
$ dtaas user add --file users.csv
GitLab account and token created for 'alice'.
  The token is in gitlab_user_tokens.json.
GitLab account and token created for 'bob'.
  The token is in gitlab_user_tokens.json.
Users added successfully
```

A row with an empty `password` cell refuses the whole run before anything
is changed, naming the users concerned.

**Example 3: a starting user.** Naming a `[[users]]` user of `dtaas.toml`
creates only their GitLab account and token. Their workspace and email come
from `dtaas.toml`, and the password from `[[users]].password` (or the
prompt):

```console
$ dtaas user add username1
'username1' is a dtaas.toml starting user.
  Its workspace is managed there, not by 'user add'.
GitLab account and token created for 'username1'.
  The token is in gitlab_user_tokens.json.
No new users added.
```

**Example 4: retry after a failure.** Suppose GitLab had refused bob's
password in example 2: alice got her account, bob got only his container,
and the command exited non-zero. Fix the password in `users.csv` and run the
same command again. Only the missing work is done, and alice is not issued
a second token:

```console
$ dtaas user add --file users.csv
'alice' already exists, skipping
'bob' already exists, skipping
GitLab account step skipped for 'alice'.
  A Personal Access Token was already issued on an earlier run (see gitlab_user_tokens.json). A re-run does not reissue one.
GitLab account and token created for 'bob'.
  The token is in gitlab_user_tokens.json.
No new users added.
```

**Files written.** Each user's token is saved to `gitlab_user_tokens.json`
in the current directory (mode `0600`) the moment it is issued:

```json
{
  "alice": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "bob": "glpat-yyyyyyyyyyyyyyyyyyyy"
}
```

Hand each user their token, and treat the file as a credential store. The
registry records the account id and the issued token, which is how a re-run
knows what is done; starting users get the same markers in a section of
their own:

```json
{
  "users": {
    "alice": {
      "email": "alice@intocps.org",
      "groups": [
        "additional"
      ],
      "load_balance": true,
      "desired_status": "running",
      "gitlab_user_id": 42,
      "gitlab_pat_issued": true
    }
  },
  "starting_users": {
    "username1": {
      "gitlab_user_id": 7,
      "gitlab_pat_issued": true
    }
  }
}
```

**Good to know:**

- The issued token is named `dtaas`, has the `read_repository` and
  `write_repository` scopes only, and expires after 365 days.
- Password order: `--password` or the CSV `password` column, then
  `[[users]].password` for a starting user, then the hidden prompt (single
  user only). The CLI uses a password once, to create the account, and
  never saves or logs it. `[[users]].password` and the CSV column are
  plaintext files you keep, though: a run that reads one warns when the
  file is readable by other users (`chmod 600` it), and once the account
  exists it reminds you to remove `[[users]].password`, which stays a valid
  GitLab login until the user changes it. `--password` is visible in shell
  history and `ps`, so prefer the prompt or the CSV.
- GitLab refuses a password that is too short or built from common words
  (for example one containing `pass` or the username).
- A username or email already taken on GitLab fails the run for that user:
  the account is left alone and gets no token, since this run did not
  create it and does not know its credentials. Delete it in GitLab, or use
  another username or email, and run the same command again.
- Any other GitLab failure makes `user add` exit non-zero, naming the users
  concerned; their containers stay up.
- An account created on an earlier run whose token was not issued or
  saved needs no password on a retry: name the user again and only the
  token is issued.
- A token is never left live without being saved: when
  `gitlab_user_tokens.json` cannot be read or written no account is
  created, a token that cannot be saved is revoked at once, and a retry
  revokes any token an earlier run issued but did not save.
- When `DTAAS_GITLAB_PAT` and `[gitlab].pat` hold different tokens, the
  exported one is used and a `Note:` line says so.
- Run one `dtaas` command at a time per installation: the registry and
  `gitlab_user_tokens.json` are rewritten without a lock, so two runs at
  once can lose each other's changes.
- For a GitLab behind an internal CA, set `[gitlab].ssl_verify` to the CA
  bundle's path. `false` turns verification off and prints a warning.
- `user delete` never removes the GitLab account; see
  [user delete](#user-delete).

Every scenario with its full output, including each failure and how to fix
it, is in the DTaaS documentation under *Install with CLI*, section *GitLab
provisioning*.

For each registry user the CLI copies `files/template/` into
`files/<username>/`, creating the directory when it is missing, and sets its
ownership to `1000:100` where it has the permission to; without that
permission it leaves the ownership as it is and carries on.

When a user has an email, the CLI automatically adds a traefik-forward-auth
routing rule for them to `config/conf.server`. A user added from a
`users.csv` row with an empty `email` cell gets no rule. Restart the
container for the change to take effect:

```bash
docker compose --env-file config/.env up -d --force-recreate traefik-forward-auth
```

#### Resource limits (optional)

By default each user container is created with the CPU, memory, process, and
shared-memory caps from `[common.resources]`, merged in from the
`users.resources.yml` overlay. To onboard users without any caps, set
`set_limits = false` in that section:

```toml
# Constrained users (default): limits enforced
[common.resources]
set_limits = true
cpus       = 4
mem_limit  = "4G"
pids_limit = 4960
shm_size   = "512m"
```

```toml
# Unconstrained users: no caps written, limit fields optional
[common.resources]
set_limits = false
```

The flag is read on every `user add`, so a deployment can host both constrained
and unconstrained users by toggling `set_limits` between runs.

> **Notes**
>
> - `user add` writes every registry user into `compose.users.yml` but
>   starts only the users added in that run. Users whose `desired_status` is
>   not `running` are never started.
> - `config reconcile --fix` with an empty registry does nothing.
> - Usernames may include '.', '_' and '-' (must start with a letter or digit).
>   (Whitespace, path separators, and shell metacharacters are rejected.)
> - This command does not enable AuthMS authentication.

---

### user delete

Removes one or more users from a running DTaaS instance, like `userdel`.

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAMES` | — | One or more usernames to remove |
| `--file PATH` / `-f` | — | Bulk-delete users listed in a CSV (only the `username` column is used) |
| `--dry-run` | off | Preview the removal without making any changes |

Pass the usernames as arguments:

```bash
dtaas user delete username1 username2
```

Or bulk-delete from a CSV (the same `users.csv` format used by
`user add --file` other columns are ignored):

```bash
dtaas user delete --file users.csv
```

`USERNAMES` and `--file` are mutually exclusive, and one of them is required.

Each user is deprovisioned (its container stopped, its compose service and
forward-auth rule removed) and dropped from `dtaas.users.registry.json`. Users
that are not currently provisioned are reported and skipped, but are still
removed from the registry.

> **Note: the GitLab account stays.** `user delete` removes the user from the
> DTaaS installation and revokes the Personal Access Tokens named `dtaas` that
> the CLI issued to them, so they lose repository access. Their GitLab account
> stays, so the user can still sign in there. A revoked token is dropped from
> `gitlab_user_tokens.json`. Revoking uses the same admin token as `user add`
> (`DTAAS_GITLAB_PAT` or `[gitlab].pat`, and `[gitlab].api_url`), even when
> `[gitlab].provision` is now off. When a token cannot be revoked (GitLab
> unreachable, admin token missing or refused), the delete still completes,
> the token stays in `gitlab_user_tokens.json`, and the command exits with an
> error naming the user: revoke that token in GitLab. To remove an account, a
> GitLab administrator deletes it in GitLab. **Do that before re-adding the
> same username**: `user add` finds the account already there, cannot know its
> credentials, and so issues no token for it. A `dtaas.toml` starting user is
> handled the same way on GitLab (token revoked, markers cleared), but its
> workspace is one of the installation's own, defined in `docker-compose.yml`,
> so `user delete` keeps its container and forward-auth rule.

Preview a removal without making any changes with `--dry-run`:

```bash
dtaas user delete username1 username2 --dry-run
```

It lists which users would be deprovisioned and removed from the registry, then
exits without stopping containers or editing any file.

The CLI automatically removes the traefik-forward-auth routing rules for
deleted users from `config/conf.server`. Restart the container for the change
to take effect:

```bash
docker compose --env-file config/.env up -d --force-recreate traefik-forward-auth
```

---

### user pause / stop / resume

Suspend or resume **specific additional (registry) users** without touching
the rest of the installation, targeting one or more `USERNAMES`, a
`--file`/`-f users.csv` (only the `username` column is read, the same way
`user delete` does), or `--all` for every additional user at once.

```bash
dtaas user pause alice bob
dtaas user stop alice
dtaas user resume alice bob
dtaas user pause --file users.csv
dtaas user stop --all
```

To suspend or resume the whole installation, pair `--all` here with the
core-only [`dtaas platform stop`/`pause`/`resume`](#lifecycle-operations).

| Command | `docker compose` verb | Effect | Reverse with |
|---|---|---|---|
| `user pause` | `pause` | Freeze the named users' containers (memory preserved) | `user resume` |
| `user stop` | `stop` | Terminate the named users' containers, keep them | `user resume` |
| `user resume` | `unpause` or `start`, as needed | Thaw a paused user, or restart a stopped one | — |

Each command also writes a `desired_status` (`"paused"`/`"stopped"`/`"running"`)
into `dtaas.users.registry.json` for the users it acted on. This is what makes
the suspension durable: a later `dtaas user add` or `dtaas config
reconcile --fix` checks `desired_status` and will **not** silently restart a
user you paused or stopped. Only `user resume` (or hand-editing the
registry) clears it back to `"running"`.

Only additional users can be targeted here. Naming a `dtaas.toml` starting
user is rejected with an error: suspend or resume the whole installation
(starting users included) with [`dtaas platform pause`/`stop`/`resume`](#lifecycle-operations)
instead.

A username not found in the registry, or found but not currently provisioned
(e.g. `user add` was never run for them), is reported and skipped rather than
aborting the whole batch:

```text
'carol' is not a registered user, skipping
alice, bob paused successfully
```

**Options:**

| Option | Default | Description |
|---|---|---|
| `USERNAMES` | — | One or more usernames to target |
| `--file PATH` / `-f` | — | Bulk-target users listed in a CSV (only the `username` column is used) |
| `--all` | off | Target every additional (registry) user; mutually exclusive with `USERNAMES`/`--file` |

---

### config reconcile

Reports drift between `dtaas.users.registry.json` (which **should** be
provisioned) and the live `compose.users.yml` services (which **are**
provisioned).

```bash
dtaas config reconcile
```

It lists:

- **missing** registered but not currently provisioned;
- **unexpected** provisioned but not in the registry (investigate: may be a
  manual edit or a partial delete);
- **drifted** provisioned, but the live config no longer matches what
  `.dtaas.state.json` recorded when it was last provisioned;
- **absent** provisioned (the user has a compose service), but the container
  is gone, for example after an interrupted `user add`;
- **desired-status drift** provisioned, but the live container state does not
  match the user's registry `desired_status` (e.g. `desired 'paused' but
  container is 'running'`).

When everything matches, it prints `In sync: no drift detected.` If Docker is
unreachable, it warns that container state could not be verified, reports
only the drift it can read from the compose file, and never reports
`In sync`.

Without `--fix`, this is read-only. Pass `--fix` to reprovision **missing** and
**drifted** users, restart the containers of **absent** users, and
pause/stop/start every provisioned user to match its `desired_status`:

```bash
dtaas config reconcile --fix
```

`--fix` runs the same code as `dtaas user add`, so it acts on the current
directory: combining it with an `--output-dir` other than the current
directory is rejected. It also refuses to run when Docker is unreachable.

**unexpected** services are never touched by `--fix` removing something
that's actually running is a deliberate action use
`dtaas user delete` for those.

**Options:**

| Option | Default | Description |
|---|---|---|
| `--output-dir PATH` | `.` | Installation directory to inspect |
| `--fix` | off | Reprovision missing/drifted registry users after reporting |

---

## User files

User management spans three files, each with a single owner, modelled on the
config/state split Terraform uses for `.tf` vs `terraform.tfstate`:

| File | Owner | Contents | Git |
|---|---|---|---|
| `dtaas.toml` `[[users]]` | Human, at install time | **Starting** users: one self-contained record per user (`username`, `email`, `groups`, `load_balance`, optional `password`) | Hand-edited; keep out of Git, as it can hold credentials |
| `dtaas.users.registry.json` | CLI (`user add` / `delete` / `pause` / `stop` / `resume`) | **Additional** users under `users`: the same fields, plus `desired_status` (`running`/`paused`/`stopped`) and any GitLab markers. Its `starting_users` section holds the GitLab markers alone, for dtaas.toml starting users that `user add` has provisioned on GitLab | Tracked CLI-written, never hand-edited |
| `.dtaas.state.json` | CLI, at provisioning time | Observed runtime facts: container id, status, provisioned-at, config hash | Ignored runtime cache |

- **`dtaas.toml`** is written once by a human and never rewritten by the CLI,
  so a comment-bearing, reviewed config is never silently mutated.
- **`dtaas.users.registry.json`** is a database the CLI owns and mutates
  atomically (the way `useradd` owns `/etc/passwd`). Edit its users through
  `dtaas user add --file users.csv` / `delete` / `pause` / `stop` /
  `resume`, not by hand. `users.csv` copied by `dtaas config generate`
  is the human-editable bulk input that feeds `add`/`delete`. `desired_status`
  defaults to `running` for a user who has never been paused or stopped, and
  `user add`/`config reconcile --fix` skip starting any user whose
  `desired_status` is not `running`.
- **`.dtaas.state.json`** is a disposable cache of what is actually running,
  refreshed on every add/delete. It is git-ignored and safe to delete.

---

## Configuration Reference dtaas.toml

`dtaas.toml` is the single source of truth for all CLI commands. Generate a
blank template with `dtaas config generate`, fill it in, then confirm
it is valid with `dtaas config validate` before running any other command.

### Which sections does my deployment need?

#### Server deployments

| Section | `localhost` | `insecure-server` | `secure-server` | `secure-server-gitlab` |
|---|:---:|:---:|:---:|:---:|
| `[common]` | Required | Required | Required | Required |
| `[common.security]` | Not used | Not used | Required | Required |
| `[common.resources]` | Optional | Optional | Optional | Optional |
| `[gitlab]` | Optional | Optional | Optional | Optional |
| `[[users]]` | Optional | Required | Required | Required |
| `[frontend]` | Not used | Required | Required | Required |
| `[localhost]` | Required | Not used | Not used | Not used |
| `[insecure-server]` | Not used | Required | Not used | Not used |
| `[secure-server]` | Not used | Not used | Required | Not used |
| `[secure-server-gitlab]` | Not used | Not used | Not used | Required |

#### Workspace deployments

| Section | `workspace-localhost` | `workspace-secure-server` |
|---|:---:|:---:|
| `[common]` | Required | Required |
| `[common.security]` | Not used | Required |
| `[common.resources]` | Optional | Optional |
| `[gitlab]` | Optional | Optional |
| `[[users]]` | Optional | Required |
| `[workspace-localhost]` | Required | Not used |
| `[workspace-secure-server]` | Not used | Required |

### Annotated dtaas.toml

The full file below shows every possible key with inline comments. Copy it
as a starting point and delete sections that do not apply to your deployment
type (see matrix above).

```toml
# ── Metadata (top level) ──────────────────────────────────────────────────────
# Describes the instance. Only git-repo is checked by validate (an http(s) URL).
name     = "Digital Twin as a Service (DTaaS)"
version  = "2.2.0"
owner    = "The INTO-CPS-Association"
git-repo = "https://github.com/into-cps-association/DTaaS.git"

# ── Common settings (all deployment types) ────────────────────────────────────
[common]
# Public hostname of the server. Use "localhost" for local deployments,
# a fully-qualified domain name (e.g. "dtaas.example.com") for servers,
# or a bare IP address.
server-dns = "dtaas.example.com"

# Absolute path to the DTaaS installation directory on the deployment host.
# Must exist before running validate.
path = "/opt/dtaas"

# ── TLS settings (required for: secure-server, secure-server-gitlab,
#                               workspace-secure-server) ──────────────────────
[common.security]
tls = true

# Absolute path to the directory containing fullchain.pem and privkey.pem.
# Used by deployment generate (seeds certs/) and platform update --certs.
certs-src = "/etc/letsencrypt/live/dtaas.example.com"

# ── Per-user container resource limits (optional, all types) ──────────────────
[common.resources]
# Enforce the limits below. Set to false to add users without any caps.
# The 4 fields are then optional and ignored. Defaults to true when omitted.
set_limits = true
cpus       = 4        # CPU cores; may be fractional, e.g. 0.5
mem_limit  = "4G"     # memory limit unit required: G, m, k …
pids_limit = 4960     # maximum number of processes per container (integer)
shm_size   = "512m"   # shared memory unit required

# ── Starting users (all deployment types) ─────────────────────────────────────
# One self-contained [[users]] block per user, hand-edited once at install
# time. Presence in this file is the desired state there are no add/delete
# lists. Additional users added later with `dtaas user add` live in the
# CLI-owned dtaas.users.registry.json instead.
# Usernames must match GitLab accounts and be unique across the array.
#
# email enables traefik-forward-auth routing rules automatically;
# groups/load_balance carry per-user tags. password is optional: it is the
# initial GitLab password when `dtaas user add <username>` provisions this
# starting user's account. Avoid committing a real secret here; prefer
# supplying it at runtime instead.
[[users]]
username     = "alice"
email        = "alice@example.com"
groups       = ["default", "dtaas"]
load_balance = true

[[users]]
username     = "bob"
email        = "bob@example.com"
groups       = ["default", "dtaas"]
load_balance = false
# password   = "Kq7v-Rm3x-Zt9w"   # initial GitLab password, see above

# ── React web client OAuth app (insecure-server, secure-server,
#                                secure-server-gitlab) ────────────────────────
# This is a SEPARATE OAuth application from the traefik-forward-auth app
# configured in [insecure-server] / [secure-server] below.
# Redirect URI: https://<server-dns>/Library
# Scopes: api read_user read_repository openid profile   Type: not Confidential
[frontend]
react-app-client-id = "dtaas-client"
react-app-oauth-url = "https://gitlab.example.com"

# ── GitLab user provisioning (optional, all types) ───────────────────────────
# Off by default. When provision = true, `dtaas user add` also creates
# each new user's GitLab account and a Personal Access Token (see
# "Add Users" in cli.md). The provisioning token must belong to an
# administrator and carry the api scope (plus admin_mode on an instance with
# Admin Mode enabled); supply it via the DTAAS_GITLAB_PAT environment variable
# rather than committing it here. A [gitlab].pat key is the fallback when the
# variable is unset or empty. ssl_verify may be a CA-bundle path for a GitLab
# behind an internal CA; false disables verification (a warning is printed)
# and true (default) uses the system trust store.
[gitlab]
provision  = false
api_url    = "https://gitlab.example.com"
# pat      = "glpat-xxxxxxxxxxxxxxxxxxxx"   # prefer DTAAS_GITLAB_PAT
# ssl_verify = "/etc/ssl/certs/corp-ca.pem" # or true (default) / false

# ── localhost deployment (dev / demo only) ────────────────────────────────────
# GitLab OAuth app for the web client; auth-authority is the GitLab URL.
[localhost]
default-user   = "alice"
client-id      = "dtaas-local"
auth-authority = "https://gitlab.example.com"

# ── insecure-server deployment (HTTP, demo only not internet-facing) ─────────
# GitLab OAuth app for traefik-forward-auth.
# Redirect URI: http://<server-dns>/_oauth
# Scopes: openid profile read_user   Type: Confidential
[insecure-server]
oauth-url           = "https://gitlab.example.com"
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"   # random; used to sign session cookies

# ── secure-server deployment (HTTPS/TLS production-ready) ───────────────────
# Same GitLab OAuth app as insecure-server, with Redirect URI using https.
[secure-server]
oauth-url           = "https://gitlab.example.com"
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"

# ── secure-server-gitlab deployment (bundled GitLab production-ready) ───────
# oauth-url points at the bundled GitLab, served under /gitlab.
[secure-server-gitlab]
oauth-url           = "https://dtaas.example.com/gitlab"
oauth-client-id     = "abc123"
oauth-client-secret = "s3cr3t"
oauth-secret        = "random-signing-string"

# ── workspace-localhost deployment (Dex on localhost dev / demo only) ───────
[workspace-localhost]
default-user   = "alice"
client-id      = "workspace-local"
auth-authority = "http://localhost:5556/dex"

# ── workspace-secure-server deployment (Keycloak production-ready) ──────────
[workspace-secure-server]
keycloak-admin          = "admin"
keycloak-admin-password = "change-me"
keycloak-realm          = "dtaas"
keycloak-issuer-url     = "https://keycloak.example.com/realms/dtaas"
keycloak-client-id      = "workspace"
keycloak-client-secret  = "s3cr3t"
oauth-secret            = "random-signing-string"
client-id               = "dtaas-frontend"
auth-authority          = "https://keycloak.example.com/realms/dtaas"
```

