Metadata-Version: 2.5
Name: trident-undertow
Version: 0.0.1
Summary: Python client and command line interface for the Trident dataspace server.
Project-URL: Homepage, https://git.ift.tuwien.ac.at/pub/pypi/trident-undertow
Project-URL: Source, https://git.ift.tuwien.ac.at/pub/pypi/trident-undertow
Project-URL: Issues, https://git.ift.tuwien.ac.at/pub/pypi/trident-undertow/-/issues
Author: TU Wien, IFT
License: MIT
License-File: LICENSE
Keywords: compute-to-data,dataspace,pontus-x,trident
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: coloredlogs>=15.0.1
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: requests>=2.32
Requires-Dist: rich>=13.0
Requires-Dist: tabulate>=0.9
Requires-Dist: typer>=0.12.5
Provides-Extra: dev
Requires-Dist: pytest-mock>=3.14; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: hdf5
Requires-Dist: h5py>=3.0; extra == 'hdf5'
Description-Content-Type: text/markdown

# undertow

Python client and command line tool for a Trident dataspace server: files, services, offerings, compute jobs
and their results, master data and provisioning.

## Install

```sh
pip install trident-undertow    # the command and the Python package are `undertow`
pip install "trident-undertow[hdf5]"   # adds h5py for reading HDF5 metadata (files inspect, upload, sync)
```

Python 3.10 or newer.

## Configure

Settings are read from the command line options, then the environment (also from a `.env` file in the working
directory, see `.env.example`), then `~/.undertow/credentials`.

| Variable | Meaning |
|---|---|
| `UNDERTOW_URL` | Trident server URL including its path prefix, e.g. `https://trident.example.org/tti` |
| `UNDERTOW_USERNAME`, `UNDERTOW_PASSWORD` | password login |
| `UNDERTOW_AUTH` | `password` (default), `device` or `client_credentials` |
| `UNDERTOW_OIDC_ISSUER` | issuer URL of the OIDC provider, e.g. `https://id.example.org` (the same one Trident trusts) |
| `UNDERTOW_OIDC_CLIENT_ID` | OIDC client, default `undertow` |
| `UNDERTOW_OIDC_CLIENT_SECRET` | secret of a confidential client (`client_credentials`) |
| `UNDERTOW_OIDC_SCOPE` | optional scope |
| `UNDERTOW_TOKEN_CACHE` | token cache of the device flow, default `~/.undertow/tokens.json`; `none` turns it off |
| `UNDERTOW_SECRETS_FILE` | secrets file for provisioning (see below) |
| `UNDERTOW_LOG_LEVEL` | `DEBUG`, `INFO`, `WARNING` (default), ... |

The options `--url`, `--username/-u`, `--password/-p`, `--auth`, `--output/-o table|json|yaml` and `--log/-l` come
before the command, e.g. `undertow -o json files list`.

`~/.undertow/` holds `credentials` (written by `system login`), `tokens.json` (device flow) and optionally
`secrets.yaml`. All files are created with mode 0600.

## Log in

There are three ways to get a token, chosen with `UNDERTOW_AUTH` or `--auth`:

- `password` (default): username and password, sent to Trident's `POST /auth/login`. Trident gets the token.
- `device`: interactive login in the browser. The tool prints a URL and a code, you log in there (MFA and SSO
  work, the password never reaches the tool). Uses the public client `undertow`.
- `client_credentials`: client id and secret get tokens directly from the OIDC provider. No user, no password, no
  browser; meant for CI, provisioning and other automations.

`device` and `client_credentials` work with any OIDC provider, e.g. Keycloak. The tool finds the provider's
endpoints through `<issuer>/.well-known/openid-configuration`; it stops with an error when this document cannot be
read or, for `device`, has no `device_authorization_endpoint`.

Password:

```sh
undertow --url https://trident.example.org/tti -u alice -p '...' system login
undertow system user
```

`system login` stores URL, username and password in `~/.undertow/credentials`; `system logout` removes them and
the cached tokens.

Device:

```sh
export UNDERTOW_OIDC_ISSUER=https://id.example.org
undertow --url https://trident.example.org/tti --auth device system login
```

Later calls reuse and refresh the cached tokens. The provider needs a public client `undertow` with the OAuth 2.0
Device Authorization Grant enabled whose tokens carry Trident's audience (`trident`); `eks-cli addon trident
prepare` creates it.

Client credentials:

```sh
export UNDERTOW_AUTH=client_credentials
export UNDERTOW_OIDC_ISSUER=https://id.example.org
export UNDERTOW_OIDC_CLIENT_ID=ci-client
export UNDERTOW_OIDC_CLIENT_SECRET=...
```

This needs a confidential client with service accounts enabled whose tokens carry Trident's audience (`trident`).

Use the same issuer as Trident, otherwise the token's issuer does not match. The user or service account needs
Trident's roles for the routes it calls.

## Commands

`undertow --help` and `undertow <group> --help` list everything. The main ones:

```sh
# files
undertow files list --search report --all
undertow files upload ./measurement.h5 --name "Measurement A"
undertow files sync --folder ./data
undertow files download measurement.h5 --to ./downloads/
undertow files inspect ./measurement.h5          # HDF5 metadata, needs undertow[hdf5]

# services and offerings
undertow services create --name "Measurement A" --type dataset --file-id 12
undertow offerings create --name "Measurement A" --service-id 5 --type compute --trust-list-id 2
undertow offerings publish 7
undertow offerings revoke 7

# dataspace
undertow dataspace list
undertow dataspace access did:op:...               # the server orders and downloads it
undertow dataspace compute start -d did:op:dataset -a did:op:algorithm --wait
undertow dataspace compute status <job id>
undertow dataspace results list --type compute
undertow dataspace results outputs 42
undertow dataspace results publish 42 --path outputs/model.json            # shows the draft
undertow dataspace results publish 42 --path outputs/model.json --yes      # publishes

# master data
undertow masterdata participants list
undertow masterdata trust-lists ensure --name "Partners" --participant-ids tenant-a,tenant-b
undertow masterdata docker-images ensure --name example/algorithm-base --tag 1.0 --checksum sha256:...
```

## Provisioning

```sh
undertow system provision --config provisioning.yaml --dry-run
undertow system provision --config provisioning.yaml
undertow system sync --config provisioning.yaml      # only the folder sync
```

The file format is described in Trident's `docs/PROVISIONING.md`. undertow sends the master data of the file to
`POST /management/provisioning` of every participant with a `server` entry (URL plus username/password or
`server.auth`), then syncs its `sync_folder`. Values like `${tenant-a.server.password}` are taken from a secrets
file: `--secrets-file`, `UNDERTOW_SECRETS_FILE`, `~/.undertow/secrets.yaml`, or `.secrets.yaml` / `secrets.yaml`
next to the provisioning file or in the working directory. Environment variables work too, and `${NAME:-default}`
sets a default.

```yaml
# secrets.yaml
tenant-a:
  server:
    password: "<password>"
```

## Python

```python
from undertow import TridentClient

client = TridentClient("https://trident.example.org/tti", "alice", "...")
res = client.management.upload_file("measurement.h5", name="Measurement A")
print(res.json()["id"])

for participant in client.management.list_all("participants"):
    print(participant["tag"])
```

Tokens from the OIDC provider: `TridentClient(url, auth=undertow.auth.auth_from_env())`.

## Development

```sh
python3 -m venv .venv && . .venv/bin/activate
make install     # pip install -e ".[dev]"
make test
```

The tests use mocks only and need no server.

## Release

The version is set in `src/undertow/__init__.py` (`__version__`).

```sh
make init        # build tools
make release     # checks PyPI and the git tree, runs the tests, builds, uploads, tags v<version>
```

## License

MIT, see `LICENSE`.
