How to use OceanVal

Validating a simulation involves three steps: register the observational datasets, matchup the simulation output against them, and generate statistics and an HTML report. Always create a new directory before running OceanVal for a new simulation, and run all commands from within it. Once you have reports for multiple simulations, you can also compare them side by side.

Step 1: Register observational datasets

Register the observational datasets you want to validate against by specifying the location of the data files and any necessary metadata, using oceanval.add_point_comparison and oceanval.add_gridded_comparison.

Setting up gridded observational data

Register a gridded (NetCDF) dataset by pointing OceanVal at a directory of observation files and saying whether the data is a climatology:

oceanval.add_gridded_comparison(
    name="oxygen",
    source="CMEMS",
    source_info="Gridded observations from the Copernicus Marine Environment Monitoring Service",
    short_name="oxygen concentration",
    model_variable="oxygen",
    obs_variable="O2_concentration",
    obs_path="/path/to/obs_data/",
    climatology=False,
)

Required parameters:

• namerequired

  A name for the dataset, e.g. "temperature".

• sourcerequired

  The source of the observational data, e.g. "CMEMS".

• model_variablerequired

  The name of the model variable to compare against the observations.

• obs_pathrequired

  Path to the directory containing the observational data files.

• obs_variable

  Name of the variable in the observational files. If omitted, OceanVal assumes a single variable is present.

• climatologyrequired

  Whether the observational data is a climatology.

Using built-in recipes for gridded data

Built-in recipe definitions make it easy to register standard observational datasets — especially useful for a standard climatology such as WOA23 or GLODAP, without manually specifying all the metadata.

oceanval.add_gridded_comparison(
    name="temperature",
    source="WOA23",
    model_variable="temp",
    recipe={"temperature": "woa23"},
    start=2005, end=2014,
    climatology=True,
)

This uses the built-in metadata and file locations for the WOA23 temperature climatology. recipe also works for salinity, oxygen, nitrate, phosphate, silicate, chlorophyll and pH — see all recipes. The underlying helper can also be used directly:

recipe = oceanval.parsers.find_recipe({"temperature": "woa23"}, start=2005, end=2014)
print(recipe["source"], recipe["obs_variable"])

Browse the full recipe catalogue to see every supported observational dataset, with the exact call to use for each.

Setting up point (in-situ) observational data

Register an in-situ dataset by pointing OceanVal at a CSV (or directory of CSVs) and naming the model variable to compare it against:

oceanval.add_point_comparison(
    name="nitrate",
    source="ICES",
    source_info="In-situ observations from the International Council for the Exploration of the Sea",
    short_name="nitrate concentration",
    model_variable="temp",
    obs_path="/path/to/obs_data/",
)

Required parameters:

• namerequired

  A name for the dataset, e.g. "temperature" — used internally to keep track of things. Letters and numbers only.

• sourcerequired

  The source of the observational data, e.g. "NOAA".

• model_variablerequired

  The name of the model variable to compare against the observations.

• obs_pathrequired

  Path to a file or directory containing the observational data. If a directory, ensure it only contains files relevant to this variable — OceanVal recursively uses all NetCDF files inside it.

Validating a variable against more than one dataset

Call oceanval.add_gridded_comparison or oceanval.add_point_comparison once per dataset, with the same name and a different source. Each source is matched up and reported separately:

oceanval.add_gridded_comparison(
    name="temperature",
    model_variable="votemper",
    recipe={"temperature": "cobe2"},
)
oceanval.add_gridded_comparison(
    name="temperature",
    model_variable="votemper",
    recipe={"temperature": "nsbc"},
)

Every call for a variable must use the same model_variable and labels. Registering a source that is already registered for that variable replaces it, so re-running a script or notebook cell is safe. Where a variable has more than one source, the summary page labels it with the source, e.g. "Temperature (COBE2)".

Step 2: Matchup model output with observations

Once your datasets are registered, pair the model output against them with oceanval.matchup:

oceanval.matchup(
    sim_dir="/path/to/simulation/output/",
    start=2000, end=2010,
    cores=4,
    lon_lim=[-80, 0], lat_lim=[20, 60],
    thickness="cell_thickness",
)

Required parameters:

• sim_dirrequired

  Path to the directory containing the model simulation output files.

• start / endrequired

  First / last year of the simulation to use for validation.

• cores

  Number of CPU cores to use for parallel processing.

• lon_lim / lat_lim

  Longitude / latitude limits for the validation region, e.g. [-180, 180].

• thicknessvertical only

  "z_level" or a variable name containing cell thickness — required for vertical validation.

Note: for monthly-resolution simulations, set point_time_res=["year", "month"] when matching up in-situ observations — otherwise day-of-year will almost never match and few matchups will be found.

Summing simulation output: to compare observations against the sum of multiple model variables, set something like "var1+var2+var3" as model_variable.

Step 3: Build the validation report

Once matched, compute statistics and generate plots with oceanval.validate, run from the same directory as the matchup step:

oceanval.validate()

This generates and opens an HTML page you can view in a web browser.

Using your own regions

The report summarises model performance in each region: a map of the regions, the seasonal cycle of the spatial average, and a table of bias, RMSD and correlations. Instead of the built-in "global" or "nwes" regions, you can supply your own netCDF file:

oceanval.validate(subregions="my_regions.nc")

The file should be set up as follows:

• One variable per region, each a 2D map on a longitude/latitude grid. It is regridded to the model grid using nearest neighbour, so it does not need to match the model grid.

• Grid cells in the region are 1, and all other cells are missing values. A 0 is also accepted: it is set to missing and ignored in the regional summaries. Any other value is rejected.

• Each region is named in the report by the variable's long_name attribute, or by the variable name if it has no long_name.

A copy of the file is saved with the results, so oceanval.compare can also compare regions across simulations. When comparing, use the same regions file for every simulation. OceanVal cannot tell two different custom files apart.

Word and PDF versions of the report

The report is HTML by default. If you need something to edit, circulate or drop into a deliverable, ask for a Word version as well:

oceanval.validate(word=True)

This writes oceanval_report.docx — the whole report as a single document — plus one .docx per page, alongside the HTML files. The report itself gains download buttons for them, so you can pick up the Word file straight from the page you are reading.

The Word version is built to be worked on rather than just read:

• Equations are real, editable Word equations rather than pictures, so you can lift a formula straight into a paper or report.

• Figures and tables are centred, the page is A4, and every page carries a page number.

• In the combined document each page becomes a numbered chapter, so figures and tables read as 1.1, 1.2, 2.1 and so on, and cross-references in the text follow the same numbering. The single-page files number their own figures from 1, matching the HTML page they came from.

Word output needs pandoc installed (conda install -c conda-forge pandoc). If it is missing, OceanVal warns and still builds the HTML report.

PDF works the same way, via pdf=True, and the two can be combined. Both are off by default because they add time to the build and most people only need the HTML.

Comparing validation outputs from multiple simulations

Once you have validation reports for different simulations, compare them with oceanval.compare, which summarises the differences between them:

oceanval.compare(
    model_dict={
        "model_a": "/path/to/model_a",
        "model_b": "/path/to/model_b",
    },
    view=True,
    ask=True,
)

model_dict maps a short name for each model to its validation output directory. The HTML comparison report is written to oceanval_comparison/compare/_build/html/notebooks/comparison_seasonal.html.

Comparison reports take word=True and pdf=True as well, producing the same Word and PDF versions as a validation report.