<a id="recipe-benchmarking"></a>

# Model Benchmarking

## Overview

These recipes and diagnostics are based on [recipe_monitor](recipe_monitor.html.md#recipe-monitor) that allow plotting arbitrary preprocessor output, i.e., arbitrary variables from arbitrary datasets. An extension of these diagnostics is used to benchmark a model simulation with other datasets (e.g. CMIP6). The benchmarking features are described in [Lauer et al.](https://doi.org/10.5194/egusphere-2024-1518).

## Available recipes and diagnostics

Recipes are stored in recipes/model_evaluation

* recipe_model_benchmarking_annual_cycle.yml
* recipe_model_benchmarking_boxplots.yml
* recipe_model_benchmarking_diurnal_cycle.yml
* recipe_model_benchmarking_maps.yml
* recipe_model_benchmarking_timeseries.yml
* recipe_model_benchmarking_zonal.yml

Diagnostics are stored in diag_scripts/monitor/

* [multi_datasets.py](../api/esmvaltool.diag_scripts.monitor/multi_datasets.html.md#api-esmvaltool-diag-scripts-monitor-multi-datasets):
  Monitoring diagnostic to show multiple datasets in one plot (incl. biases).

### Recipe settings

See [multi_datasets.py](../api/esmvaltool.diag_scripts.monitor/multi_datasets.html.md#api-esmvaltool-diag-scripts-monitor-multi-datasets) for a list of all possible configuration options that can be specified in the recipe.

#### NOTE
Please note that exactly one dataset (the dataset to be benchmarked) needs to specify the facet `benchmark_dataset: true` in the dataset entry of the recipe. For line plots (i.e. annual cycle, diurnal cycle, time series), it is recommended, to specify a particular line color and line style in the `scripts` section of the recipe for the dataset to be benchmarked (`benchmark_dataset: true`) so that this dataset is easy to identify in the plot. In the example below, MIROC6 is the dataset to be benchmarked and ERA5 is used as a reference dataset.

```yaml
scripts:
  allplots:
    script: monitor/multi_datasets.py
    plot_folder: '{plot_dir}'
    plot_filename: '{plot_type}_{real_name}_{mip}'
    group_variables_by: variable_group
    facet_used_for_labels: alias
    plots:
      diurnal_cycle:
        legend_kwargs:
          loc: upper right
        plot_kwargs:
          'MIROC6':
            color: red
            label: '{alias}'
            linestyle: '-'
            linewidth: 2
            zorder: 4
          ERA5:
            color: black
            label: '{dataset}'
            linestyle: '-'
            linewidth: 2
            zorder: 3
          MultiModelPercentile10:
            color: gray
            label: '{dataset}'
            linestyle: '--'
            linewidth: 1
            zorder: 2
          MultiModelPercentile90:
            color: gray
            label: '{dataset}'
            linestyle: '--'
            linewidth: 1
            zorder: 2
          default:
            color: lightgray
            label: null
            linestyle: '-'
            linewidth: 1
            zorder: 1
```

## Variables

Any, but the variables’ number of dimensions should match the ones expected by each plot.

## References

* Lauer, A., L. Bock, B. Hassler, P. Jöckel, L. Ruhe, and M. Schlund: Monitoring and benchmarking Earth
  System Model simulations with ESMValTool v2.12.0, EGUsphere [preprint], [https://doi.org/10.5194/egusphere-2024-1518](https://doi.org/10.5194/egusphere-2024-1518), 2024.

## Example plots

<a id="fig-benchmarking-annual-cycle"></a>
![image](recipes/figures/benchmarking/annual_cycle.png)

<a id="fig-benchmarking-boxplots"></a>
![image](recipes/figures/benchmarking/boxplots.png)

<a id="fig-benchmarking-diurn-cycle"></a>
![image](recipes/figures/benchmarking/diurnal_cycle.png)

<a id="fig-benchmarking-map"></a>
![image](recipes/figures/benchmarking/map.png)

<a id="fig-benchmarking-timeseries"></a>
![image](recipes/figures/benchmarking/timeseries.png)

<a id="fig-benchmarking-zonal"></a>
![image](recipes/figures/benchmarking/zonal.png)
