Metadata-Version: 2.5
Name: nml-tools
Version: 0.6.0
Summary: Toolbox for Fortran Namelists
Project-URL: Homepage, https://github.com/MuellerSeb/nml-tools
Project-URL: Repository, https://github.com/MuellerSeb/nml-tools
Project-URL: Issues, https://github.com/MuellerSeb/nml-tools/issues
Author: Sebastian Mueller
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.8
Requires-Dist: click
Requires-Dist: jinja2
Requires-Dist: jsonschema
Requires-Dist: packaging
Requires-Dist: pyyaml
Requires-Dist: tomli; python_version < '3.11'
Provides-Extra: dev
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: numpy<3,>=1.24; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Provides-Extra: gui
Requires-Dist: guidata; (python_version >= '3.9') and extra == 'gui'
Requires-Dist: guidata<3.7,>=3.6; (python_version < '3.9') and extra == 'gui'
Requires-Dist: numpy; (python_version >= '3.9') and extra == 'gui'
Requires-Dist: numpy<1.25,>=1.21; (python_version < '3.9') and extra == 'gui'
Requires-Dist: qtpy; extra == 'gui'
Description-Content-Type: text/markdown

# nml-tools

<p align="center">
  <img src="https://raw.githubusercontent.com/MuellerSeb/nml-tools/6213cc56244cca589563d0baabf93f2b46610b27/docs/logo.png" alt="nml-tools logo" width="240">
</p>

Generate Fortran namelist modules, Markdown docs, and template namelists from a small JSON Schema-like specification with Fortran-focused extensions.

## Features

- Schema input in YAML or JSON.
- Core keywords from JSON schema: `type`, `properties`, `required`, `default`,
  `examples`, `title`, `description`, `format`.
- Fortran extensions: `x-fortran-namelist`, `x-fortran-kind`,
  `x-fortran-len`, `x-fortran-shape`,
  `x-fortran-default-*`, `x-fortran-type`, `x-fortran-module`.
- Outputs: Fortran module, helper module, Markdown docs, template namelist.
- Config-driven CLI for batching multiple schemas.

### Missing Fortran features

- Derived types currently support one level of intrinsic scalar components
  only; nested derived values and array components are not supported.
- No support for complex types:
  - JSON doesn't have a native complex type.
  - Could be emulated with:
    - Lists containing the real and imaginary parts plus explicit metadata.
    - Objects with `real` and `imag` properties.
    - Strings with a project-defined `format` convention.

## Required values and operational defaults

`nml-tools` treats schema defaults as operational initialization values, not
only as annotations. A property may therefore appear in `required` and also
define a default. If the default completely initializes the property, the
namelist and generated setters do not require an argument for it. A declared
required property without complete default coverage still requires input.

This intentionally differs from standard JSON Schema validation, where
`default` is annotation-only and does not satisfy `required`. Explicit
namelist or setter values override defaults. Validation accounts for absent
defaults without inserting values into the parsed namelist representation.

For intrinsic arrays, a property-level list without repeat or pad must exactly
match the configured total extent. Repeat and pad defaults may target larger
runtime extents, and a scalar `items.default` broadcasts to every positive
extent.

## Fortran extensions (x-fortran-*)

These keywords extend JSON Schema with Fortran-specific requirements.

### Identifier Names

Names that nml-tools turns into Fortran identifiers must be valid Fortran
identifiers and must not contain `__`. Double underscores are reserved for
generated support names such as `seed__default`, `method__enum_values`, and
`period__start_year__min`.

Schema property names and runtime dimension names must also avoid generated
namelist type member names: `is_configured`, `init`, `init_type`, `set_dims`,
`from_file`, `set`, `is_set`, `is_valid`, `set_mask`, and `filled_shape`.

### x-fortran-namelist

- Location: schema root.
- Type: string.
- Meaning: name of the Fortran namelist block.

Example:

```yaml
x-fortran-namelist: optimization
```

### x-fortran-kind

- Location: integer/number properties or array `items`.
- Type: string.
- Meaning: Fortran kind identifier (mapped via `[kinds]` in the config).
- If omitted, plain `integer`/`real` is used.

Example:

```yaml
count:
  type: integer
  x-fortran-kind: i4
```

### x-fortran-len

- Location: string properties or array `items`.
- Type: integer literal or identifier.
- Meaning: Fortran character length (`character(len=...)`).
- Identifiers must be defined in `[constants]` in the config and be integers.

Example:

```yaml
name:
  type: string
  x-fortran-len: buf
```

### x-fortran-shape

- Location: array properties.
- Type: integer, identifier, or list of integers/identifiers.
- Meaning: Fortran array dimensions; identifiers are resolved via `[constants]`
  or `[dimensions]`.
- Required; deferred-size dimensions are not supported.
- Nested arrays are not supported. Use a single array with a shape list for
  multi-dimensional arrays.
- Shape identifiers from `[constants]` produce fixed-size arrays.
- Shape identifiers from `[dimensions]` produce allocatable arrays whose
  dimensions can be changed at runtime with the generated `set_dims()` method.

Example:

```yaml
values:
  type: array
  x-fortran-shape: [3, 2, max_iter]
  items:
    type: number
    x-fortran-kind: dp
```

### x-fortran-type and x-fortran-module

- Location: a `type: object` defined inline as a property or array `items`,
  or a referenced reusable object definition.
- `x-fortran-type` is required and gives the Fortran derived type name.
- `x-fortran-module` is optional. If absent, the type is emitted once in the
  generated helper module. If present, the namelist module imports the type
  from that application-owned module.

Use an inline object for a single-use derived field:

```yaml
properties:
  station:
    title: Selected station
    description: Application-owned station descriptor.
    type: object
    x-fortran-type: station_t
    x-fortran-module: application_types
    properties:
      code:
        type: integer
      label:
        type: string
        x-fortran-len: 16
```

Use `$defs` / `$ref` when a type is reused or when type documentation should
be separate from field documentation:

```yaml
$defs:
  period:
    title: Time period
    description: Bounds for one interval.
    type: object
    x-fortran-type: period_t
    properties:
      start_year:
        type: integer
      label:
        type: string
        x-fortran-len: 16
        default: default
properties:
  period:
    $ref: "#/$defs/period"
    properties:
      label:
        default: main
  periods:
    type: array
    x-fortran-shape: n_periods
    items:
      $ref: "#/$defs/period"
```

The `properties` block on a derived-type `$ref` use site may refine existing
scalar members, including `default`, `title`, `description`, bounds, and enums.
It may not add new members, because the referenced type definition owns the
Fortran layout. In the example above, `period%label` defaults to `main`, while
`periods%label` keeps the reusable definition default `default`.

Only intrinsic scalar members are supported in the first implementation.
Nested derived members, array members, and array-level lists of derived defaults are rejected. Scalar derived objects may
define a mapping-valued `default`; derived array `items.default` may define the
same mapping as a broadcast fallback for every element:

```yaml
properties:
  period:
    $ref: "#/$defs/period"
    default:
      start_year: 2000
  periods:
    type: array
    x-fortran-shape: n_periods
    items:
      $ref: "#/$defs/period"
      default:
        start_year: 1980
        end_year: 1999
```

Initialization and override precedence is:

```text
component defaults and missing sentinels -> object/items default -> explicit input
```

An inner component listed in the derived object's `required` may have a leaf
default or be covered by the selected object/item default. Only uncovered
required components create an input requirement. A derived field with such an
uncovered component must itself appear in the containing object's `required`;
otherwise the schema is rejected. A declared-required derived field with no
uncovered required components is legal and needs no input. Generated logical
storage must have an effective component or object/item default because it has
no missing sentinel.

Aggregate `is_set("period")` and `is_valid()` consider only uncovered required
components. Component queries such as `is_set("period%label")` still report
that component's own effective presence. For fixed-shape derived arrays, every
element must cover every uncovered required component. For imported fields
with string members, including arrays of imported values, generated code
verifies that application storage length exactly matches `x-fortran-len`.

Generated native APIs accept typed values and add `init_type`, for example:

```fortran
type(period_t) :: period
type(period_t), allocatable :: periods(:)
status = config%init_type(period=period, periods=periods)
period%start_year = 2001
status = config%set(period=period)
```

Namelist and template entries use normal component notation:

```fortran
period%start_year = 2001
periods(2)%start_year = 2001
```

Fortran also accepts buffer-style derived values, for example
`period = 2001, "main"`. For helper-owned generated types, that positional
order follows the reusable or inline object definition's schema `properties`
order. For imported application-owned types, the schema order must match the
actual Fortran type declaration order if buffer-style input is used. Component
notation is preferred because it is explicit and robust to layout changes.

### Array presence queries

Generated namelist types with arrays expose `set_mask(name, mask, errmsg)` and
`filled_shape(name, filled, errmsg)` as integer status-returning functions.
Both use the same configuration state and per-item presence rules as indexed
`is_set(name, idx=...)`. Names are case-insensitive and derived component paths
use `%`, for example `entries%start_year`.

Generated array storage and returned masks use one-based indexing in every
dimension. Setter arguments may use other lower bounds; their elements are
copied into a prefix starting at index 1.
The generated helper `idx_check(idx, shp, field, errmsg)` checks indices against
these one-based bounds, with `shp` containing the extent in each dimension.

```fortran
logical, allocatable :: mask(:,:)
integer :: status, filled(2)
character(len=256) :: errmsg

status = cfg%set_mask("values", mask, errmsg=errmsg)
status = cfg%filled_shape("values", filled, errmsg=errmsg)
```

`set_mask` allocates a logical mask to the array's current shape. Its generic
has one specific binding per distinct array rank in the namelist, selected by
the rank of `mask`. True means the indexed `is_set` query returns `NML_OK`.
Defaulted elements (including logical elements, which have no missing sentinel)
are always present; the mask describes effective presence, not assignment history.
A derived root uses its required, non-defaulted components to determine presence.
An incomplete derived element returns `NML_ERR_PARTLY_SET` and leaves the mask
unallocated; component paths remain independently queryable.

`filled_shape` requires an integer vector of size equal to the array rank. It
returns the smallest rectangular prefix containing all set elements, anchored
at index 1 in every dimension. An all-missing array returns zero extents,
and a fully present array returns its full shape. Holes inside the prefix return
`NML_ERR_PARTLY_SET` while retaining the computed extents. Incomplete derived
elements also return `NML_ERR_PARTLY_SET`, but with zero extents.

Unconfigured instances or unavailable storage return `NML_ERR_NOT_SET`.
Unknown names return `NML_ERR_INVALID_NAME`; scalar paths, wrong mask ranks,
and wrong `filled` sizes return `NML_ERR_INVALID_INDEX` with targeted messages.
A mask rank without a generated overload is rejected by the Fortran compiler.
Failed mask queries leave the output unallocated; other filled-shape errors
leave zero extents. Call `set` or `from_file` before querying; `init` and
`set_dims` reset configuration state.

Use runtime dimensions when the exact extent is known. For capacity-sized
optional arrays, use `set_mask` for sparse data, `filled_shape` to require a
complete prefix, or indexed `is_set` for individual elements. Ordinary required
arrays still require complete population during generated `is_valid()` checks.
Optional arrays do not acquire automatic prefix validation.

#### Breaking migration: removed flex-tail dimensions

`x-fortran-flex-tail-dims` has been removed and is rejected, including a value
of zero. Delete it from schemas. Existing `filled_shape` calls retain their
signature, but now inspect every dimension and require a configured instance.
To retain a variable filled-prefix workflow, make the capacity-sized array
schema-optional and explicitly check `filled_shape` status, nonempty extents,
and any dimensions your application requires to be full. Leaving it required
applies ordinary full-array completeness rules. `examples/01_simple` demonstrates
this migration. No deprecation stage is provided.

### x-fortran-default-order

- Location: array properties with `default`.
- Type: string (`F` or `C`, default `F`).
- Meaning: memory order used when reshaping defaults.

Example:

```yaml
x-fortran-default-order: C
```

### x-fortran-default-repeat

- Location: array properties with `default`.
- Type: boolean.
- Meaning: repeat the provided default values to fill the full shape.
- Array defaults must be lists; to broadcast a scalar, set `default` on
  `items` instead.
- Cannot be used together with `x-fortran-default-pad`.

Example:

```yaml
x-fortran-default-repeat: true
```

### x-fortran-default-pad

- Location: array properties with `default`.
- Type: scalar or list of scalars.
- Meaning: pad values used to fill the array if the default list is shorter
  than the shape.
- Cannot be used together with `x-fortran-default-repeat`.

Example:

```yaml
x-fortran-default-pad: 0
```

Combined example:

```yaml
values_repeat:
  type: array
  x-fortran-shape: [2, 3]
  items:
    type: integer
    x-fortran-kind: i4
  default: [1, 2]
  x-fortran-default-repeat: true

values_pad_c:
  type: array
  x-fortran-shape: [2, 3]
  items:
    type: integer
    x-fortran-kind: i4
  default: [1, 2, 3]
  x-fortran-default-order: C
  x-fortran-default-pad: 0
```

## Array set semantics

Generated `set(...)` methods follow Fortran namelist-buffer semantics for
arrays. When an array argument is provided, the setter writes the supplied values
into the leading subsection of the target array. The provided extents must not
exceed the configured target extents, but they do not have to fill the full
array.

Completeness is checked by `is_valid()`, not by `set(...)`. This means a
required array can be partly set without an immediate setter error, but
`is_valid()` will fail until the required entries are fully provided.

Generated Python wrappers use the same semantics: scalar and lower-rank array
inputs are normalized to singleton trailing dimensions before calling the
Fortran setter.

## String format annotations

String schemas may use JSON Schema's `format` keyword as lightweight metadata
for documentation and downstream editor UIs. nml-tools preserves the annotation
and shows it in generated Markdown, but does not validate the runtime value,
check file existence, or parse date/time strings.

Common useful formats include standard JSON Schema values such as `date`,
`time`, and `date-time`, plus nml-tools path-oriented custom values:

```yaml
forcing_file:
  type: string
  format: file-path

output_dir:
  type: string
  format: directory-path

start_time:
  type: string
  format: date-time
```

Omitting `format` means the string is ordinary free text. Downstream tools may
interpret `path` as a generic filesystem path, `file-path` as a file picker, and
`directory-path` as a directory picker.

## Validation keywords

Only a subset of JSON Schema validation keywords is implemented.
Validation is opt-in: call `is_valid()` on the generated type to check required values, enum and bound constraints.

### Enum support
Enums are supported for strings and integers only.
For arrays, enums are defined on `items` (not on the array itself).

- Keywords: `enum`
- The generated Fortran module exposes public `*__enum_values` arrays and
  elemental `*__in_enum` helpers.
- String enums compare against `trim(value)`; enum literals are stored with the
  field length.

### Numeric bounds

You can add minimum/maximum constraints for integer or real values.
For arrays, the bounds apply to each item and must be defined on `items`.

- Keywords: `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`
- Applies to: `integer` and `number` (`real` in Fortran)
- For arrays: bounds belong on `items`, not the array property

Example (scalars):

```yaml
tolerance:
  type: number
  x-fortran-kind: dp
  minimum: 0.0
  exclusiveMaximum: 1.0
```

Example (array items):

```yaml
counts:
  type: array
  x-fortran-shape: 3
  items:
    type: integer
    minimum: 1
```

## Configuration (`nml-config.toml` or `pyproject.toml`)

The CLI reads a TOML config file. If `--config` is omitted, nml-tools first
looks for `nml-config.toml` and then for `[tool.nml-tools]` in
`pyproject.toml`. Paths are resolved relative to the config file location.

Standalone config files use the tables shown below at the TOML root. In
`pyproject.toml`, put the same content under `[tool.nml-tools]`, for example
`[tool.nml-tools.helper]` and `[[tool.nml-tools.namelists]]`.

- `required-version` (string, optional): PEP 440 version specifier set for the
  nml-tools versions supported by this config, for example `>=0.2.2,<0.3`.
- `minimum-version` (string, optional): legacy shorthand for
  `required-version = ">=<minimum-version>"`. Do not set both keys.

Example:

```toml
[tool.nml-tools]
required-version = ">=0.2.2"

[tool.nml-tools.helper]
path = "out/nml_helper.f90"

[[tool.nml-tools.namelists]]
name = "optimization"
schema = "optimization.yml"
mod_path = "out/nml_optimization.f90"
```

### [helper]

Controls the generated helper module.

- `path` (string, required to generate helper): output file for the helper module.
- `module` (string, optional): Fortran module name (default: `nml_helper`).
- `buffer` (int, optional): line buffer length for the helper module.
- `header` (string, optional): text inserted at the top of the helper file.

### [constants]

Named static constants used for fixed dimensions, string lengths, and generated
helper parameters.

- Each entry is a table with integer `value` and optional `doc`.
- Names must not contain `__`; nml-tools reserves double underscores for
  generated helper identifiers.
- Values must be plain integers (no kind suffixes).
- String lengths from `x-fortran-len` may use constants. Runtime dimensions are
  intentionally not supported for string lengths.

Example:

```toml
[constants.buf]
value = 128
doc = "String buffer length."
```

### [dimensions]

Named runtime array dimension defaults.

- Each entry is a table with positive integer `default` and optional `doc`.
- Optional `source = { namelist = "...", property = "..." }` identifies the
  intrinsic integer scalar supplying the dimension during Python/CLI loading.
  `property` defaults to the dimension name. References are resolved against
  configured schemas case-insensitively; component paths and array elements
  are unsupported.
- Names must be unique across `[constants]` and `[dimensions]`.
- Names must not contain `__`; nml-tools reserves double underscores for
  generated helper identifiers.
- Names must not collide with namelist property names, because generated
  Fortran stores the current runtime extent as a field with the dimension name.
- Names must not collide with generated namelist type members such as `init`,
  `set_dims`, or `is_valid`.
- Entries may be used in `x-fortran-shape`, but not in `x-fortran-len`.
- Arrays whose shape contains a `[dimensions]` name are generated as
  allocatable runtime-sized arrays.
- Generated Fortran and Python wrappers expose `set_dims(...)`. Omitted or
  `None` Python values reset the dimension to its configured default.
- Calling `set_dims(...)` deallocates affected arrays and clears configured
  values. Call `set(...)` or `from_file(...)` afterwards.

Example:

```toml
[dimensions.max_iter]
default = 4
doc = "Maximum number of iterations."
```

With a source property:

```toml
[dimensions.n_values]
default = 2
source = { namelist = "settings", property = "count" }
```

Loading resolves an explicit caller or `--dimensions` override first, then an
explicit source assignment, then the dimension default. A schema default for
the source property does not supply a dimension. Missing or null-only source
assignments use the dimension default; nulls after an explicit assignment retain
that assignment. Inferred values must be positive and satisfy the source
property's schema constraints. For dimensions without overrides, repeated
source namelist groups are ambiguous even if they supply equal values.

Sources are inferred before full schema evaluation so runtime-sized arrays use
the actual extents. This metadata does not change generated Fortran readers,
Python wrappers, or generated outputs; generation still uses configured
dimension defaults.

### [kinds]

Defines the kind module and allowed kinds.

- `module` (string): Fortran module to `use`.
- `real` (list of strings): allowed real kinds.
- `integer` (list of strings): allowed integer kinds.
- `map` (table, optional): schema kind name → module kind name.

### [f2py]

Optional settings for f2py wrapper generation.

- `f2cmap_path` (string, optional): output path for a generated `.f2py_f2cmap`
  file.
- `c_types.real` (table): explicit f2py C type mapping for schema real kind
  names used by f2py wrappers.
- `c_types.integer` (table): explicit f2py C type mapping for schema integer
  kind names used by f2py wrappers.
  `c_intptr_t` is added automatically as `long_long` unless explicitly
  overridden.

Generated Python f2py shims assume a package-local extension layout. The f2py
extension module named by `f2py_path` must be installed next to the generated
Python wrapper file, and the wrapper imports it with `from . import <module>`.

The f2py wrappers use the same schema kind aliases as the normal Fortran
module. For example, a schema kind `dp` with `[kinds].map = { dp = "real64" }`
is still emitted as `real(dp)` in the wrapper and imported as
`dp=>real64`. The generated f2py map therefore also uses `dp`:

```toml
[f2py]
f2cmap_path = ".f2py_f2cmap"

[f2py.c_types.real]
dp = "double"

[f2py.c_types.integer]
i4 = "int"
```

The f2py-visible wrapper procedures avoid Fortran `optional` dummy arguments
and assumed-shape arrays. Optional Python values are represented by generated
`has__<name>` flags and harmless dummy values. Inside the Fortran wrapper,
allocated local variables are passed to the generated type-bound `set` method
when a value is present; unallocated allocatables are passed otherwise, so the
type-bound `set` still sees `present(arg) == .false.`. This keeps the f2py ABI
simple while preserving the normal generated Fortran setter semantics. This
relies on the Fortran 2008 rule that an unallocated allocatable actual argument
associated with an optional nonallocatable dummy argument is treated as not
present.

For array arguments, the generated Python shim follows the same partial-set
semantics as the Fortran setter. Scalar inputs become shape `(1, ..., 1)`;
lower-rank inputs get singleton trailing dimensions before being passed to f2py.

Derived values stay natural at the Python API boundary:

```python
cfg.set(period={"start_year": 2001})
cfg.set(periods=[{"start_year": 1980}, {"start_year": 2001}])
```

Only the internal f2py ABI is flattened: `%` paths are encoded with `__`, for
example `period__start_year` and `has__period__start_year`. Generated Fortran
support identifiers also use `__` as an internal separator, for example
`seed__default`, `method__enum_values`, and `n_periods__default`. Do not use
`__` in schema property, component, derived type/module, constant, or runtime
dimension names; it is reserved for generated identifiers. Python
`is_set("period.start_year")`
is translated to the native `is_set("period%start_year")` lookup. Nested
sequences of mappings are accepted for multi-rank derived arrays. Flattened
generated f2py names are made unique case-insensitively and deterministically
shortened when needed to remain valid Fortran identifiers.

The f2py wrappers use opaque integer handles for Fortran-owned namelist
instances. nml-tools assumes that the owning Fortran library creates those
handles from live `target` objects and keeps the objects alive while Python uses
the handle. The raw-address helper generated for f2py support uses `c_loc`/
`c_f_pointer` semantics; taking `c_loc` of the nonpolymorphic generated namelist
target relies on the Fortran 2008 rule that allows scalar nonpolymorphic
variables with no length type parameter. `NML_ERR_INVALID_HANDLE` only detects a
zero handle. Passing a stale, foreign, or otherwise invalid non-zero handle is
undefined behavior and may crash. Projects that need robust invalid-handle
detection should add their own registry/token layer in the owning library.
If the owning Fortran library deallocates, replaces, or otherwise invalidates a
target, users should discard all Python wrappers for that handle or call the
generated wrapper's `invalidate()` method. `invalidate()` only sets that Python
wrapper's stored handle to zero; it does not notify Fortran or deallocate
anything.

### [documentation]

Optional extra module documentation appended after the generated `\brief` and
`\details`. Multiline strings are supported and Doxygen formatting is allowed.

- `module` (string, optional): extra module documentation block.
- `py-style` (`numpy` or `doxygen`, optional): generated Python f2py wrapper
  docstring style (default `numpy`).
- `md_doxygen_id_from_name` (bool, optional): add `{#<namelist>}` to Markdown header (default `false`).
- `md_add_toc_statement` (bool, optional): insert `[TOC]` after the Markdown header (default `false`).

### namelists (array)

Schema entries to generate per-namelist outputs.

- `name` (string, optional): expected namelist name defined by the schema.
  When present, it must match the schema's `x-fortran-namelist`
  case-insensitively; the schema remains canonical for generated namelist names.
  The schema's `x-fortran-namelist` must be a valid user Fortran identifier, and
  `__` is reserved for generated internal names.
- `schema` (string): schema file path.
- `mod_path` (string, optional): Fortran module output path.
- `doc_path` (string, optional): Markdown output path.
- `f2py_path` (string, optional): Fortran source path for f2py wrapper modules.
- `py_path` (string, optional): Python shim path for wrapper classes. Requires
  `f2py_path`.

If a path is omitted, that output is not generated.

Multiple namelists may use the same `f2py_path` or `py_path`; nml-tools
collects the wrappers/classes into the shared file.
`file_profiles[].namelists` and `templates[].namelists` refer to these
expected namelist names.

### file_profiles (array)

Named project-specific namelist file layouts.

- `name` (string): unique profile name.
- `default_file` (string): project file-name hint for tools that open/save real
  namelist files.
- `namelists` (list of strings): configured namelist names in file order.
- `required` (list of strings, optional): profile namelists that must be present
  during `validate --profile` (default `[]`). Entries must be listed in
  `namelists`.
- `title` / `description` (string, optional): display and documentation metadata.

`default_file` is not used for template output paths. Profiles do not define
values or defaults; use schema defaults/examples and `[templates.values]` for
template values. `required` controls validation only; it does not affect
template generation.

```toml
[[file_profiles]]
name = "main"
title = "Main configuration"
description = "Runtime settings for the model."
default_file = "run.nml"
namelists = ["run", "physics"]
required = ["run"]
```

### project_profiles (array)

Optional named sets of file profiles required for a project setup.

- `name` (string): unique, non-empty project profile name.
- `file_profiles` (non-empty list of strings): existing file profile names.
- `title` / `description` (string, optional): display and documentation metadata.

Names and references are matched case-insensitively. Unknown references,
duplicate members, and duplicate project names are rejected. Omission or
`project_profiles = []` defines no project profiles. File profiles can be shared
across project profiles, and their `default_file` values remain filename hints.

```toml
[[project_profiles]]
name = "standard"
title = "Standard simulation"
file_profiles = ["main", "outputs"]

[[project_profiles]]
name = "minimal"
file_profiles = ["main"]
```

All members describe files needed for the selected project setup, each subject
to its existing file-profile validation rules. The GUI uses this metadata to
offer complete projects for editing; generation and validation commands do not
select a project or change their outputs because it exists. When the GUI edits
a project, every dimension-source namelist used by that project must occur in
exactly one of its file profiles.

### templates (array)

Template namelist output configuration.

- `path` (string): template namelist output path.
- `profile` (string, optional): file profile whose namelists are included in the template.
- `namelists` (list of strings, optional): namelist names included in the template.
- `schemas` (list of strings, optional): backward-compatible schema paths for
  namelist inclusion. These paths must match configured `[[namelists]].schema`
  entries and are loaded through the same internal template schema list.
- `title` / `description` (string, optional): documented-template header metadata.
- `doc_mode`: `plain` or `documented`.
- `value_mode`: `empty`, `filled`, `minimal-empty`, or `minimal-filled`.
- `simple_derived_mode`: `components` (the default) or `buffer`.

Each template must define exactly one of `profile`, `namelists`, or legacy
`schemas`. If `schemas` is used, neither `profile` nor `namelists` may be
present. Template entries using `profile` inherit the profile `title` and
`description` unless they override them. In documented mode, title and
description are emitted as leading comments before the first namelist block;
plain templates omit them.

The default `simple_derived_mode = "components"` writes derived values with
explicit component designators and is the most robust representation. The
optional `buffer` mode writes simple derived values as concise positional
records in resolved schema `properties` order:

```fortran
setting = .true., 1
settings(1) = .true., 1
settings(2) = .false., 2
```

Derived-array records are always separate indexed assignments, including
complete subscripts for higher-rank arrays; records are never concatenated into
one whole-array buffer. This preserves record boundaries and prevents a missing
value from shifting later records. Imported Fortran types must declare their
components in exactly the resolved schema order. Buffer templates require the
built-in schema-aware namelist evaluator used by current nml-tools releases;
the former f90nml validation path cannot reliably read indexed positional
derived assignments.

Optional values can override per-namelist fields for filled modes:

```toml
[templates.values.optimization]
niterations = 4
tolerance = 0.1
```

Derived overrides use nested TOML tables and arrays of tables:

```toml
[templates.values.run.period]
start_year = 2001

[[templates.values.run.periods]]
start_year = 1980

[[templates.values.run.periods]]
start_year = 2001
```

Overrides remain name-based mappings in both derived modes. For higher-rank
derived arrays, a flat array of tables supplies leading elements in Fortran
element order (the first subscript varies fastest).

For filled templates, each derived component is chosen in this order: explicit
template override, component example, object/item default, component default,
then enum or type fallback. Minimal modes omit a derived field only when every
component is default-initialized; covering only its required components is not
enough to omit optional non-defaulted components.

## CLI

The command line interface is available as `nml-tools` or `nmlt`.
Common flags: `-h`/`--help`, `-V`/`--version`, `-v`/`-q` (repeatable).

Primary subcommands:

- `generate`: run the full pipeline (helper, Fortran modules, docs, templates).
- `check`: verify configured generated files are up to date.
- `gen-fortran`: generate helper + Fortran modules only.
- `gen-markdown`: generate Markdown docs only.
- `gen-template`: generate template namelists only.
- `validate`: validate a namelist file against schema definitions.

### Generation

Generate all outputs:

```bash
nml-tools generate --config nml-config.toml
```

Generate specific outputs:

```bash
nml-tools gen-fortran --config nml-config.toml
nml-tools gen-markdown --config nml-config.toml
nml-tools gen-template --config nml-config.toml
```

If `f2py_path` is configured, `generate` and `gen-fortran` also emit the
f2py-facing Fortran wrapper file. `generate` additionally emits `py_path`
Python shims. If `[f2py].f2cmap_path` is configured, the generated map can be
passed to f2py with `--f2cmap`.

Check generated files without rewriting them:

```bash
nml-tools check --config nml-config.toml
nml-tools check --config nml-config.toml --diff
```

`check` exits with a non-zero status if any configured generated file is
missing or differs from the current generator output. This is intended for CI
jobs that should ensure checked-in generated files are current.

### Validation

Validation is check-only and implements a focused set of schema constraints
(types, required, enums, bounds, string length, and array shape).
Operational defaults are tracked as the prior value for standard null-input
semantics, but they do not count as explicitly supplied required input.
Unknown keys or namelist groups not covered by the provided schemas are errors.

The input parser follows standard formatted Fortran namelist syntax. It does
not accept processor or legacy-parser extensions such as `$group`, `$end`,
`&end`, `#` comments, kind-suffixed constants, or undelimited character
values. Groups and object names are matched case-insensitively, while errors
report the original spelling and source location.

Config-driven:

```bash
nml-tools validate --config nml-config.toml input.nml
nml-tools validate --config nml-config.toml --profile main run.nml
```

Schema-only:

```bash
nml-tools validate --schema demo.yml --input demo.nml
nml-tools validate --schema a.yml --schema b.yml combined.nml
```

When validation is config-driven, schema constants are loaded from `[constants]`
and runtime array dimensions are loaded from `[dimensions]`. Both can be
overridden for a validation run:

```bash
nml-tools validate --config nml-config.toml \
  --constants buf=128 \
  --dimensions max_iter=10 \
  input.nml
```

In schema-only validation, provide the same values ad hoc:

```bash
nml-tools validate --schema demo.yml \
  --constants buf=128 \
  --dimensions max_iter=10 \
  input.nml
```

`--constants` supplies static schema constants for string lengths and fixed
array shapes. `--dimensions` supplies runtime array dimensions. Names are
matched case-insensitively, normalized to lowercase, and must stay unique across
both sets.

Configured dimension sources are read from the target namelist file. Use one
optional `--dim-file PATH` to supply additional sources from another namelist
file:

```bash
nml-tools validate --config nml-config.toml --profile main \
  --dim-file settings.nml run.nml
```

Only the target file undergoes full validation. The dimension file is parsed
completely, but only configured source entries are evaluated; unrelated groups
and missing required properties are not fully validated. Sources are resolved
against all configured schemas, including namelists outside the selected file
profile. Supplying the target itself as `--dim-file` reads it once.

`--dim-file` requires config, either discovered during config-driven validation
or supplied explicitly with `--config` alongside `--schema`. It accepts one file;
use explicit `--dimensions NAME=INT` values when further sources are needed.
See [the dimension sources example](examples/06_dimension_sources/README.md).

Array assignments use one-based schema bounds and Fortran element order, with
the first subscript varying fastest. Scalar subscripts and sections are
supported, including omitted bounds and positive or negative strides. Values
may use standard null and repetition syntax; nulls leave the prior effective
value unchanged and later overlapping assignments take precedence.

Simple derived values accept component notation and standard positional
effective-item input. For example, `setting = .true., 1` assigns components in
resolved schema-property order, while `settings(2) = .false., 3` assigns one
complete derived-array element without spilling into adjacent elements.

Character substring assignment, complex schema values, nested derived values,
component arrays, nondefault lower bounds, and user-defined formatted I/O are
currently explicit capability boundaries.

## GUI

Install the current checkout with its optional GUI dependencies and a Qt binding
(Python 3.8 or newer):

```bash
python -m pip install '.[gui]' PyQt5
nml-tools gui -c /path/to/nml-config.toml -p simulation -o /path/to/project
```

QtPy also permits other supported Qt bindings; PyQt5 is the tested binding on
Python 3.8. `-c` accepts `nml-config.toml` or a `pyproject.toml` containing
`[tool.nml-tools]`; relative schema paths are resolved from that file. Omit
`-c` to discover either file in the current directory, `-p` to choose a project
inside the GUI, and `-o` to use the current directory. Each project's Settings
page can choose a different output directory.

The editor loads each profile's `default_file` from the output directory and
saves directly to that namelist file. Schema defaults are written when needed;
examples and type-specific suggestions are hints until the user accepts or
edits them. Invalid input is reported. No JSON configuration or
JSON-to-namelist conversion is involved. Selected groups are rewritten on
save; other groups and their comments remain intact.

An interactive launch starts with an empty project tree and a central
`+ : Project` action. Configured projects can be added from the config, while
custom projects can be built from file profiles and namelist schemas. Selecting
a project opens its Settings page; selecting a file shows its namelists and a
syntax-highlighted file preview; selecting a namelist opens its editable form.
Creating a custom project asks for its project name and a new `.toml` filename
in the schemas directory.
The Back and Next controls move through that file's namelists. Standard file
profiles initially select their configured namelists and lock required members;
optional members may be removed and more namelists may be added. Saved `.nml`
files can be loaded through `+ : File` → `Browse…`, associated with a configured
profile or treated as custom. A file outside the project output directory is
imported and later saved under the same filename in that output directory.

One- and two-dimensional arrays use inline tables with copy/paste; a
one-element one-dimensional array presents its element editor directly.
Higher-rank arrays use the guidata array editor. Deferred `:` dimensions retain
a resize action. File-path fields provide browsing and store paths relative to
the project's output directory.

Applications can launch a subset of the configured profiles:

```python
from nml_tools.gui import GuiProjectRequest, launch_gui

launch_gui(
    config_path="/path/to/nml-config.toml",
    projects=[
        GuiProjectRequest(
            project_profile="simulation",
            output_dir="/path/to/project",
            file_profiles={"main": ["mainconfig", "time_periods"], "parameter": []},
            initial_values={"main": {"mainconfig": {"nDomains": 2}}},
            initial_dimensions={"max_domains": 2},
        )
    ],
)
```

Each request configures one project and its output directory. A `None`
`file_profiles` value selects every configured profile; an empty namelist list
selects every namelist in that profile. Names are checked case-insensitively and
pages retain TOML order. `initial_values` is an in-memory dictionary of
profile/namelist/field values applied over existing input. With no project
requests, the tree starts empty for interactive project selection;
`default_output_dir` controls the initial output directory for those projects.

For sourced runtime dimensions, `initial_dimensions` takes precedence over an
explicit saved source property, which takes precedence over the TOML default.
The source link appears only while its namelist belongs to the current project;
removing that namelist returns the dimension to manual input.
The GUI also recovers extents for older unsourced schemas from saved indexed
arrays where possible. Settings controls can then change the dimensions and
rebuild the affected editors; `R` opens a configured source property. Existing
Qt applications reuse their QApplication; importing `nml_tools.gui` alone does
not import Qt.

## Error handling

Generated type-bound procedures return integer status codes and accept an
optional `errmsg` argument. No `error stop` is emitted by generated code.

Status codes (defined in the helper module):

| Code | Meaning |
| --- | --- |
| `NML_OK` (0) | success |
| `NML_ERR_FILE_NOT_FOUND` (1) | file does not exist |
| `NML_ERR_OPEN` (2) | failed to open file |
| `NML_ERR_NOT_OPEN` (3) | file not open |
| `NML_ERR_NML_NOT_FOUND` (4) | namelist not found |
| `NML_ERR_READ` (5) | read/parse error |
| `NML_ERR_CLOSE` (6) | close error |
| `NML_ERR_REQUIRED` (10) | required field missing |
| `NML_ERR_ENUM` (11) | enum constraint failed |
| `NML_ERR_NOT_SET` (12) | field not set (sentinel) |
| `NML_ERR_PARTLY_SET` (13) | array partially set |
| `NML_ERR_BOUNDS` (14) | bounds constraint failed |
| `NML_ERR_INVALID_NAME` (20) | unknown field name |
| `NML_ERR_INVALID_INDEX` (21) | invalid index for array access |
| `NML_ERR_INVALID_HANDLE` (22) | zero opaque f2py handle |

Notes:

- `errmsg`, when present, is filled with a short message (including `iomsg` on read errors).
- When every field has an effective default or is otherwise optional, a readable
  file may omit that namelist group. `from_file` then succeeds with the same
  initialized defaults and optional-value sentinels as an empty group. A missing
  group still fails when any field requires input, and a missing file always fails.
- `is_set` returns `NML_ERR_NOT_SET` if a value is missing, and
  `NML_ERR_INVALID_NAME`/`NML_ERR_INVALID_INDEX` on misuse.
- `NML_ERR_INVALID_HANDLE` only reports zero f2py handles. Non-zero invalid
  handles are outside the generated wrapper contract.

## Documentation format

Generated Fortran doc-strings are currently tailored for Doxygen. Future
versions may make this configurable to support other tools such as FORD or the
Sphinx Fortran domain.

## Installation

```bash
pip install nml-tools
```

## Minimal example

Schema (`demo.yml`):

```yaml
title: Demo namelist
description: Small example namelist for nml-tools.
x-fortran-namelist: demo
type: object
required: [count]
properties:
  count:
    type: integer
    title: Number of steps
    x-fortran-kind: i4
  name:
    type: string
    title: Run label
    x-fortran-len: 64
    default: run1
  weights:
    type: array
    title: Weight vector
    x-fortran-shape: 3
    items:
      type: number
      x-fortran-kind: dp
    default: [0.1, 0.2, 0.3]
```

Config (`nml-config.toml`):

```toml
[helper]
path = "out/nml_helper.f90"
module = "nml_helper"
buffer = 1024

[kinds]
module = "iso_fortran_env"
real = ["real64"]
integer = ["int32"]
map = { dp = "real64", i4 = "int32" }

[[namelists]]
name = "demo"
schema = "demo.yml"
mod_path = "out/nml_demo.f90"
doc_path = "out/nml_demo.md"

[[file_profiles]]
name = "main"
default_file = "demo.nml"
namelists = ["demo"]

[[templates]]
path = "out/demo.nml"
profile = "main"
doc_mode = "documented"
value_mode = "filled"
```

Generate outputs:

```bash
nml-tools generate --config nml-config.toml
```

Expected outputs:

- `out/nml_helper.f90`
- `out/nml_demo.f90`
- `out/nml_demo.md`
- `out/demo.nml`

## Comparison to JSON Schema

This project implements a focused subset of JSON Schema aimed at Fortran
namelist workflows, plus extensions for Fortran-specific types and shapes.

Main missing features compared to JSON Schema:

- No remote schema resolution.
- No composition keywords: `allOf`, `anyOf`, `oneOf`, `not`.
- No conditional keywords: `if`/`then`/`else`.
- No object constraints: `additionalProperties`, `patternProperties`,
  `propertyNames`, `dependencies`.
- No advanced array constraints: tuple typing, `contains`, `minItems`,
  `maxItems`, `uniqueItems`.
- No numeric or string validation keywords like `multipleOf`, `minLength`,
  `maxLength`, `pattern` (bounds are supported via `minimum`,
  `maximum`, `exclusiveMinimum`, and `exclusiveMaximum`).
- Object schemas are supported only as one-level inline or referenced Fortran
  derived types with intrinsic scalar members.

Use the `x-fortran-*` extensions to express kind, length, and shape information
needed for code generation.

### Reusable definitions and references

Schemas may reuse supported scalar or scalar-array field definitions with JSON
Schema Draft 2020-12 `$defs` and `$ref`:

```yaml
x-fortran-namelist: solver
type: object
$defs:
  positive_count:
    type: integer
    minimum: 1
    x-fortran-kind: i4
properties:
  iterations:
    $ref: "#/$defs/positive_count"
    title: Iteration limit
    default: 100
```

References may target the same document or local `.yml`, `.yaml`, or `.json`
files, for example `common-definitions.yml#/$defs/fraction`. Relative file
paths are resolved from the file containing the `$ref`; no project registry or
network retrieval is used. Standard JSON Pointer fragments are supported.

Keywords next to `$ref` compose with the reusable definition. A use site may
narrow numeric bounds or enums, and its `title`, `description`, `examples`, or
`default` is used for generated output when present. Conflicting Fortran
representation keywords such as `x-fortran-kind`, `x-fortran-len`, and
`x-fortran-shape` are rejected. For arrays, a use-site array `default` replaces
the referenced default together with any `x-fortran-default-*` controls.

Object definitions may represent one-level derived-type properties through
`x-fortran-type` and optional `x-fortran-module`. An inline object is a
single-use definition; repeated or independently documented types should use
`$defs` / `$ref`. Referenced type use sites may refine existing scalar
components but cannot add components or change the Fortran type identity.
`$id`/`$anchor` resolution, recursive references, `$dynamicRef`, legacy
`definitions`, and general composition keywords are not supported.
