Metadata-Version: 2.4
Name: EBFM
Version: 0.5.0
Author: EBFM Authors
License-Expression: BSD-3-Clause
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: AUTHORS.txt
Requires-Dist: mypy>=1.0
Requires-Dist: netCDF4>=1.7
Requires-Dist: numpy>=2.0
Requires-Dist: pandas>=2.0
Requires-Dist: pandas-stubs>=2.0
Requires-Dist: pyproj
Requires-Dist: scipy
Requires-Dist: isodate>=0.7.2
Provides-Extra: dev
Requires-Dist: pre-commit>=4.0; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: tox>=4.0; extra == "dev"
Requires-Dist: tox-gh-actions; extra == "dev"
Provides-Extra: mpi
Requires-Dist: mpi4py>=4.1; extra == "mpi"
Provides-Extra: cpl
Requires-Dist: EBFM[mpi]; extra == "cpl"
Requires-Dist: yac>=3.8; extra == "cpl"
Provides-Extra: performance
Requires-Dist: EBFM[mpi]; extra == "performance"
Requires-Dist: numba>=0.59; extra == "performance"
Dynamic: license-file

<!--
SPDX-FileCopyrightText: 2025 EBFM Authors

SPDX-License-Identifier: BSD-3-Clause
-->

# Energy balance and firn model (EBFM)

[![PyPI version](https://img.shields.io/pypi/v/ebfm)](https://pypi.org/p/ebfm/) ![License](https://img.shields.io/pypi/l/ebfm)

This repository provides a Python implementation of an energy balance and firn model (EBFM).

Using YAC, this model can be coupled to other models.

## Basic installation

### Create virtual environment

You can use an arbitrary location to create your virtual environment.
A common choice is `.venv`. In the following, we will use `$VENV` as a placeholder and the path to the virtual environment `.venv` should be set as follows:

```sh
export VENV=/path/to/your/environment/.venv
```

If you do not already have a virtual environment, you can create one using the following commands, which will additionally install the necessary dependencies specified in `pyproject.toml`:

```sh
python3 -m venv $VENV
source $VENV/bin/activate
```

Check if your virtual environment is activated. You should see the name within parentheses just before your command prompt, in this case it should say `(.venv)`.

If you want to know more about virtual environments, please refer to the [Python documentation](https://docs.python.org/3/library/venv.html). \
If you intend to develop EBFM, please take a look at the [developer notes](https://github.com/wardvp/EBFM/edit/matthias-ibach-patch-1/README.md?pr=%2Fwardvp%2FEBFM%2Fpull%2F23#developer-notes) further down in the `README`.

### Install EBFM

You can then install EBFM directly into you virtual environment by running the following command:

```sh
pip3 install EBFM
```

This will install the basic version of EBFM without coupling.

### Checking your installation

Please check your installation by running `ebfm --help` to print the help message and `ebfm --version` to print the installed version.

## Installation with coupling features

Before installing EBFM with coupling features, you must install YAC with the Python bindings. Similar to the instructions above, we will use a virtual environment to install all required dependencies.
During YAC’s configuration step, set the installation prefix to your virtual environment and enable Python bindings (`--prefix=$VENV` `--enable-python-bindings`).
Make sure to do this *after* creating your virtual environment.

The procedure should look similar to below:

```sh
python3 -m venv $VENV
# configure and install YAC with --prefix=$VENV and --enable-python-bindings
source $VENV/bin/activate
pip3 install EBFM[cpl]
```

Adding `[cpl]` will make sure that additional dependencies needed for coupling, such as `yac`, are present in your virtual environment.

If you see any errors during the process, please make sure that your virtual environment is activated. If this is the case, you should see the name within parentheses just before your command prompt, in this case it should say `(.venv)`.

If during the installation of EBFM it appears that `yac` is missing, please double-check with `pip3 freeze` that the package has been installed properly. For detailed instructions on how to install YAC properly and guidance for troubleshooting, see the YAC documentation on [Python bindings](https://yac.gitlab-pages.dkrz.de/YAC-dev/d7/d9e/pythonbindings.html).

## Running EBFM

After installation, a basic, uncoupled simulation can be run with the following command, provided you cloned this repository:

```sh
ebfm --matlab-mesh examples/dem_and_mask.mat
```

### Mesh data

The arguments `--matlab-mesh`, `--elmer-mesh`, and `--netcdf-mesh` allow to provide different kinds of mesh data.
EBFM supports the following formats:

For Elmer-based inputs, the argument `--elmer-mesh-crs-epsg` is needed to define the coordinate reference system (CRS)
of the input x/y coordinates. EBFM uses this CRS to convert coordinates to lon/lat (EPSG:4326) internally.

Important: For some larger files we are using Git LFS. Please make sure to [install Git LFS](https://git-lfs.com/).

* MATLAB Mesh: An example is given in `examples/dem_and_mask.mat`. This mesh
  file provides x-y coordinates and elevation data. Please use the argument
  `--matlab-mesh /path/to/your/mesh.mat`.

  Usage example:

  ```sh
  ebfm --matlab-mesh examples/dem_and_mask.mat
  ```

* Elmer Mesh: An Elmer mesh file with x-y coordinates of mesh points and
  elevation data stored in the z-component. Please use the argument
  `--elmer-mesh /path/to/your/elmer/mesh`.

  Usage example:

  ```sh
  ebfm --elmer-mesh examples/greenland_mesh_v0_with_DEM/MESH --elmer-mesh-crs-epsg 3413
  ```

  Note: You can also use `--elmer-mesh examples/greenland_mesh_v0/MESH` here, but this will lead to a very unphysical
  setup with zero elevation in the whole domain. To generate `examples/greenland_mesh_v0_with_DEM/MESH` please refer
  to the section **Amend elevation data to 2D Elmer mesh** below.

* Elmer Mesh with Elevation data from NetCDF: The Elmer mesh file provides x-y
  coordinate. An additioal NetCDF file is given to provide elevation data for
  these x-y coordinates. Please use the arguments `--elmer-mesh /path/to/your/elmer/mesh`
  and `--netcdf-mesh /path/to/your/elevation.nc`

  Usage example:

  ```sh
  ebfm --elmer-mesh examples/greenland_mesh_v0/MESH --netcdf-mesh examples/BedMachineGreenland-v5_lo.nc --elmer-mesh-crs-epsg 3413
  ```

Note that an Elmer mesh must be provided in a directory following the structure:

```
path/to/your/elmer/MESH/
├── mesh.boundary
├── mesh.elements
├── mesh.header
└── mesh.nodes
```

The option `--is-partitioned-elmer-mesh` will tell EBFM that the provided Elmer
mesh is a partitioned mesh. A partitioned mesh file follows the structure:

```
path/to/your/elmer/mesh
    └── MESH
        ├── mesh.boundary
        ├── mesh.elements
        ├── mesh.header
        ├── mesh.nodes
        └── partitioning.128
            ├── part.1.boundary
            ├── part.1.elements
            ├── part.1.header
            ├── part.1.nodes
            ...
            ├── part.128.boundary
            ├── part.128.elements
            ├── part.128.header
            └── part.128.nodes
```

Usage example for partitioned mesh:

```sh
ebfm --elmer-mesh examples/greenland_mesh_v0/MESH/partitioning.128/ --netcdf-mesh examples/BedMachineGreenland-v5_lo.nc --is-partitioned-elmer-mesh --use-part 42
```

Note: The partitioned mesh is not included in this repository.

### Getting the example data

A reduced-resolution NetCDF file (`examples/BedMachineGreenland-v5_lo.nc`) is
included directly in this repository and can be used as a drop-in for quick
testing. See [`examples/README.md`](examples/README.md) for details on how it
was derived from the original dataset.

For the full-resolution dataset and the example Elmer mesh (`MESH`), refer to
the [TerraDT testcase repository](https://gitlab.dkrz.de/TerraDT/testcase).
Note that access to that repository may need to be requested and requires
access to the Levante supercomputer at DKRZ.

Once available, you can either copy or symlink the required files into the
`examples/` directory of this repository, or point EBFM to their locations
using the CLI arguments shown above.


### Using `reader.py` to amend elevantion data to `examples/greenland_mesh_v0`

The helper script `reader.py` can be used to combine an existing Elmer mesh with a DEM NetCDF file by replacing the z‑coordinates in `mesh.nodes` ahead of time. This can be useful if you want to preprocess a mesh once and reuse it for multiple EBFM runs.

Assuming the example data has been copied into the `examples/` directory as described above, you can run the following command from the repository root:

```sh
python3 src/ebfm/reader.py examples/greenland_mesh_v0/MESH examples/BedMachineGreenland-v5.nc --outpath examples/greenland_mesh_v0_with_DEM/MESH --elmer-mesh-crs-epsg 3413
```

This will write the updated mesh to a new directory. The path `examples/greenland_mesh_v0_with_DEM` should not already exist. The original `examples/greenland_mesh_v0/MESH` directory is copied and left unchanged.

Alternatively, you can modify the mesh directly in place, which overwrites `mesh.nodes`:

```sh
python3 src/ebfm/reader.py examples/greenland_mesh_v0/MESH examples/BedMachineGreenland-v5_lo.nc --in-place --elmer-mesh-crs-epsg 3413
```

The resulting mesh can then be used directly with EBFM similar to the example with the MATLAB file from above:

```sh
ebfm --elmer-mesh examples/greenland_mesh_v0_with_DEM/MESH --elmer-mesh-crs-epsg 3413
```

### Performance and Profiling Runs

#### Installation for Performance Runs

Install this branch with the performance dependencies:

```sh
pip install -e .[performance]
```

This additionally installs `numba` to run with multiple CPU-threads.

#### Running EBFM with performance optimiuations

EBFM now includes several performance improvements and supports several options for performance testing and benchmarking:

- Regular (NumPy) path:

  The default run already includes some performance improvements. Run as usual:

  ```sh
  ebfm --matlab-mesh examples/dem_and_mask.mat
  ```

- Numba kernels:

  To enable Numba-accelerated kernels, use the `--with-numba` flag. You can control the number of threads with `--numba-threads N` (replace `N` with the desired thread count):

  ```sh
  ebfm --matlab-mesh examples/dem_and_mask.mat --with-numba --numba-threads 2
  ```

  Note: If you use more than one thread, you must specify `--numba-threads`. In practice, 2 threads have shown the best performance so far, but optimal settings depend on your hardware and problem size. Feel free to experiment.


#### Timing Your Run

To measure the total runtime, simply prepend your command with `time`:

```sh
time ebfm --matlab-mesh examples/dem_and_mask.mat --with-numba --numba-threads 2
```

#### Comparing Results

A new script is provided to compare model output snapshots:

- `tools/compare_snapshots.py` can be used to compare two output files (e.g., from different runs or configurations).

To create a reference file for comparison, use the `--dump-reference` option at the end of your run:

```sh
ebfm --matlab-mesh examples/dem_and_mask.mat --dump-reference reference_run.npz
```

Then compare with:

```sh
python tools/compare_snapshots.py reference_run.npz new_run.npz
```

Note: If you use the random-forcing within EBFM in your testcases, make sure to additionally set the option `--random-seed`, as explained below.

#### Diagnostics and Reproducibility

- The option `--random-seed` was introduced to allow reproducible runs (especially if the random forcing in the example testcase is used). Set a fixed seed to ensure identical results for repeated runs (important for benchmarking and debugging):

  ```sh
  ebfm --matlab-mesh examples/dem_and_mask.mat --random-seed 42
  ```

- Use `--diagnostics` to print some diagnostics for a quick overview for every timestep:

  ```sh
  ebfm --matlab-mesh examples/dem_and_mask.mat --diagnostics
  ```

### Coupled simulation

The EBFM code allows coupling to other simulation codes. The following arguments
allow to configure the coupling:

```sh
ebfm ...
  --couple-to-elmer-ice
  --couple-to-icon-atmo
  --coupler-config /path/to/coupling/config.yaml
```

Note that the coupling uses the Python bindings of YAC. Additionally, EBFM must
be run in a MPMD (multiple process multiple data) run.
Follow the install instructions from above and run the example command for a coupled simulation with Elmer/Ice and ICON:

```sh
mpirun -np 1 ebfm \
  --elmer-mesh $MESHES/greenland_mesh_v0/MESH/partitioning.128/ \
  --netcdf-mesh $DATA/BedMachineGreenland-v5.nc \
  --is-partitioned-elmer-mesh --use-part 1 \
  --coupler-config $CPL_CONFIG --couple-to-elmer --couple-to-icon \
  : \
  -np 1 $ELMER_ROOT/src/elmer_dummy_f.x $MESHES/greenland_mesh_v0/MESH/partitioning.128 1 $CPL_CONFIG \
  : \
  -np 1 $ICON_ROOT/src/icon_dummy.x $MESHES/icon_grid_0054_R02B08_G.nc $DATA/mbe3064_atm_elmer_monmean_1979.nc $DATA/varlist_elmerfile
```

Be aware that the command above requires setting a few environment variables.
Assuming your project is structured following [this repository](https://gitlab.dkrz.de/TerraDT/ebfm_dummy),
the following settings should help getting started:

```sh
export EBFM_DUMMY_REPO=/path/to/TerraDT/ebfm_dummy
export EBFM_REPO=/path/to/this/repo

export CPL_CONFIG=$EBFM_DUMMY_REPO/config/coupling.yaml
export MESHES=$EBFM_DUMMY_REPO/mesh
export DATA=$EBFM_DUMMY_REPO/data

export EBFM_ROOT=$EBFM_REPO
export ELMER_ROOT=$EBFM_DUMMY_REPO/dummies/ELMER
export ICON_ROOT=$EBFM_DUMMY_REPO/dummies/ICON
```

Depending on the binaries that you want to use `$ELMER_ROOT` and/or `$ICON_ROOT`
may be set to point to the non-dummy versions of the codes.

## Running tests

You can use `tox`to run the test suite in an isolated environment. Select the tox environment matching your Python version, e.g., `py313` for Python 3.13:

```sh
tox -e py313
```

This runs all tests in the `tests/` directory.

To run the examples via tox, use:

```sh
tox -e examples
```

Remove build/test artifacts after your run with:

```sh
tox -e clean
```

Note: Replace `py313` with the environment for your installed Python version (e.g., `py310`, `py311`, `py312`). To test across multiple Python versions, you need to have those versions installed on your system. \
Check your current Python version with:

```sh
python3 --version
```

Alternatively, if you prefer to run your tests directly in your current environment:

```sh
pytest -v tests/
```

## Troubleshooting

### `libnetcdf.so` not found at runtime

*Problem:* `./icon_dummy.x: error while loading shared libraries: libnetcdf.so.15: cannot open shared object file: No such file or directory`
*Solution:* Try rebuilding `icon_dummy.x`
`
### `libyaxt_c.so` not found at runtime

*Problem:* `./icon_dummy.x: error while loading shared libraries: libyaxt_c.so.1: cannot open shared object file: No such file or directory`

*Solution:* `export LD_LIBRARY_PATH='$YAXT_INSTALL_DIR/lib/'

### `#include <proj.h>` not found when building the Elmer dummy

*Problem:*
```sh
...
elmer_grid.c:11:10: fatal error: proj.h: No such file or directory
   11 | #include <proj.h>
      |          ^~~~
compilation terminated.
make: *** [Makefile:48: elmer_grid.o] Error 1
```

*Solution:* `sudo apt-get install libproj-dev`

## Developer notes

### Optional dependencies `[dev]`

Please install the package with the optional dependencies `[dev]` if you are developing (this will automatically
install `pre-commit`).

To concatenate multiple optional dependencies, please run

```sh
pip3 install EBFM[dev,cpl]
```

### pre-commit

This project uses pre-commit hooks for some tasks described in detail below. To setup pre-commit please do the following:

```sh
pipx install pre-commit
pre-commit install
```

**Alternative:** If you are already working in an virtual environment, you can also use `pip3 install pre-commit` instead of `pipx`. Refer to the [documentation of `pipx`](https://pipx.pypa.io/stable/#overview-what-is-pipx) for further information.

As soon as pre-commit is set up, you will not be able to commit if any of the checks fails. With the help of the logging output it should usually be possible to fix the problem.

Note: You can bypass this check with `--no-verify`. Please note that the CI will also run pre-commit and fail if there are problems in any of the checks. Therefore, it is recommended to use the pre-commit hooks locally before pushing code to this repository and only bypass them if there is a good reason.

### Copyright and licensing

This project uses [REUSE](https://reuse.software/) to track information regarding copyright and licensing. Therefore, all files in this repository are required to provide the corresponding information. Please refer to the documentation of REUSE for details.

You can use pre-commit to automatically check if all files in the repository provide the necessary information:

```
pre-commit run reuse --all-files
```

### Code formatting

Automated checks for PEP8 compiance are implemented following [^1] with some modifications. You can use pre-commit hooks to automatically format your code with black:

```sh
pre-commit run black --all-files
```

With flake8 you can check whether your code follows all relevant formatting rules:

```sh
pre-commit run flake8 --all-files
```

Please note that black might not be able to automatically fix all problems and therefore flake8 might fail even if you have run black before. In this case, you will have to manually fix the remaining problems.

### Further hints

* Please consider installing EBFM via `pip3 --editable .` if you are developing the package.

----

[^1]: https://ljvmiranda921.github.io/notebook/2018/06/21/precommits-using-black-and-flake8/
