Feed `model_id` and `variant_label` to recipe #309

mo-nikosbaltas · 2025-12-30T15:47:18Z

Closes #287

PR creation checklist for the developer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the developer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately,including the [Quick Start] https://github.com/MetOffice/CMEW/blob/main/doc/source/user_guide/quick_start.rst) section? N/A
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

PR creation checklist for the reviewer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the reviewer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately, including the Quick Start section?
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

Better comment Co-authored-by: Alistair Sellar <[email protected]>

Remove comment Co-authored-by: Alistair Sellar <[email protected]>

Better comment Co-authored-by: Alistair Sellar <[email protected]>

…and variant_label

mo-nikosbaltas · 2025-12-30T15:50:41Z

@alistairsellar could you please provide feedback on the errors encountered. These are not related to the implementation but not availability of data (I think, but not an expert on this!)

When running cylc, run_recipe_radiation_budget failed We get this error from ESMValtool.
Looking at job.out:

ERROR No input files found for Dataset: . dataset: 'HadGEM3-GC31-LL',
project: 'CMIP6', exp: 'historical', ensemble: 'r5i1p1f3' .
.
Looked for files matching
/data/users/managecmip/champ/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r5i1
p1f3/Amon/hfls/gn//hfls_Amon_HadGEM3-GC31-LL_historical_r5i1p1f3_gn.nc
/data/users/managecmip/champ/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r5i1
p1f3/Amon/hfls/gn//hfls_Amon_HadGEM3-GC31-LL_historical_r5i1p1f3_gn.nc
Similar "No input files found" errors are printed for rlds, rls, rss for the reference dataset.

At the end of the file:
ERROR Could not create all tasks
ERROR Missing data for preprocessor seasonal_radiation_budget/hfls: .
dataset: HadGEM3-GC31-LL . ensemble: r5i1p1f3 .
ERROR Not all input files required to run the recipe could be found.
So ESMValTool is telling you:

It is looking in the CHAMP CMIP6 archive under /data/users/managecmip/champ/CMIP6/CMIP/...
It cannot find the HadGEM3-GC31-LL historical r5i1p1f3 files for several variables for 1993.
That is why run_recipe_radiation_budget fails.

Looking at the logs further, #287 does the correct thingy.

Reference dataset (dataset index 0) is now:
dataset: HadGEM3-GC31-LL
project: CMIP6
exp: historical
ensemble: r5i1p1f3
activity: CMIP

Evaluation dataset (dataset index 1) is:
dataset: UKESM1-0-LL
project: ESMVal
exp: amip
activity: ESMVal
ensemble: r1i1p1f1

And in the same log I could see:
ESMValTool does find EVAL data:

Found input files for Dataset: hfls, Amon, ESMVal, UKESM1-0-LL, ESMVal, amip, r1i1p1f1, gn, v20251230 Found input files for Dataset: hfss, Amon, ESMVal, UKESM1-0-LL, ESMVal, amip, r1i1p1f1, gn, v20251230 Found input files for Dataset: rlds, Amon, ESMVal, UKESM1-0-LL, ESMVal, amip, r1i1p1f1, gn, v20251230 . etc.

So:
The EVAL path (the CDDS output under ${ROOT_DATA_DIR}) is correct and being picked up.
The REF dataset is correctly wired to CMIP6 HadGEM3-GC31-LL with the variant REF_VARIANT_LABEL we configured in rose-suite.conf.

The failure is specifically: the CHAMP CMIP6 archive does not contain all the expected files for HadGEM3-GC31-LL, historical, r5i1p1f3, year 1993 for all variables.

The current rose-suite.conf has:

MODEL_ID="UKESM1-0-LL"
VARIANT_LABEL="r1i1p1f1"

REF_MODEL_ID="HadGEM3-GC31-LL"
REF_VARIANT_LABEL="r5i1p1f3"

Alistair, can you check whether the files exist for HadGEM3-GC31-LL, r5i1p1f3. If those files (hfls, rlds, rls, rss at 1993) are missing or stored under a different ensemble, then it seems that we get those errors.

To confirm that the #287 implementation is correct we can align REF_ with EVAL_ and check that it works.
In other words, in rose-suite.conf we set:
REF_MODEL_ID="UKESM1-0-LL"
REF_VARIANT_LABEL="r1i1p1f1"

But if we want to keep the REF settings as, REF_MODEL_ID="HadGEM3-GC31-LL"
REF_VARIANT_LABEL="r5i1p1f3"
Then, you need to check the availability of the data. Otherwise we need to choose a different ensemble.

alistairsellar · 2025-12-30T16:26:16Z

It is looking in the CHAMP CMIP6 archive under /data/users/managecmip/champ/CMIP6/CMIP/...

Hrmmm, it is indeed looking there. However it shouldn't be looking there, since that's our mirror of CMIP data and we now want CMEW to be feeding the locally standardised data into ESMValTool, not CMIP data. So ESMValTool should be looking in the cylc-run dir for the standardised data.

alistairsellar

Thanks @mo-nikosbaltas, looks good. My requested changes relate to comments and docstrings only.

CMEW/app/configure_for/bin/test_update_recipe_file.py

CMEW/app/configure_for/bin/update_recipe_file.py

CMEW/app/configure_for/bin/test_update_recipe_file.py

CMEW/app/configure_for/bin/update_recipe_file.py

Co-authored-by: Alistair Sellar <[email protected]>

…b.com:MetOffice/CMEW into 287-feed-model_id-and-variant_label-to-recipe

This reverts commit 704cdb7.

mo-nikosbaltas · 2026-01-05T14:26:03Z

Document workflow.rst changed a040aaf

alistairsellar

Thanks @mo-nikosbaltas. There is some useful detail in what you've added, but the sections would have been quite inconsistent in their level of detail and the space they occupied: the updated sections would have had much more detail (and took up much more space) than the sections not impacted by this PR. For me it made the flow of the page hard to follow. I've therefore condensed your additions, for more consistency across the page.

If you accept the suggestions, then once committed please check that these still compile and render OK, as I haven't checked that.

I can see that some more detail could be useful across all of the sections, using some of the more expansive formatting that you have used here. I propose that we schedule a discussion to get a shared understanding of what we want from the documentation. In the meantime, feel free to open an issue to propose what you feel should be covered in the workflow page.

doc/source/user_guide/workflow.rst

Co-authored-by: Alistair Sellar <[email protected]>

alistairsellar

Apologies, my suggestions included trailing spaces, which has broken the GitHub checks. These suggestions remove some of them - hopefully all...

doc/source/user_guide/workflow.rst

Co-authored-by: Alistair Sellar <[email protected]>

NParsonsMO · 2026-01-06T13:50:09Z

CMEW/app/configure_for/bin/update_recipe_file.py

+            "one for the reference and one for the evaluation run."
+        )
+
+    # Reference dataset: keep existing project/exp/grid but override


I don't think it is keeping the existing project or exp? I think it's overwriting them?

NParsonsMO · 2026-01-06T13:59:08Z

CMEW/app/configure_for/bin/update_recipe_file.py

+        {
+            "dataset": ref_model_id,
+            "project": "ESMVal",
+            "exp": "amip",


Here, I think we're changing this from "historical". Is it deliberate? If so, the comment should be updated.

Ooh, I should have picked this up in first review. Actually I think that experiment might be wrong for both runs, including the original. For this recipe (radiation budget) the choice of experiment doesn't make a difference, but it will for some recipes, so experiment should be something that the user defines as part of the model run definition. I've just opened an issue to add that: #316.

For this PR, I propose that we accept that the second run is no more wrong than the first, and that having them consistently called "amip" is as good as any choice. I.e. I propose that we keep "exp": "amip" for both.

I still think the comment should reflect what's going on, but I will take note to pay more attention to the unchanged code in a review next time.

doc/source/user_guide/workflow.rst

NParsonsMO · 2026-01-06T15:14:29Z

doc/source/user_guide/workflow.rst

  :Executes:
     The ``cdds_convert`` command and the ``restructure_dirs.sh`` script
-     from the |Rose| app
+     from the |Rose| app.


There was no full stop in the equivalent line of "configure_standardise", "configure_for" or "install_env_file". they should probably be consistent.

doc/source/user_guide/workflow.rst

NParsonsMO · 2026-01-06T15:41:28Z

@mo-nikosbaltas some of the review comments are just queries, but if I'm correct that we are overwriting the experiment (from "historical" to "amip", then the comment should reflect this (the recipe does run successfully with the change, but is the data that it assesses the same data?)

mo-nikosbaltas and others added 12 commits December 17, 2025 21:42

Config changes for adding a second model dev run

80f05a4

Reformatted rose-*.conf filesafter CI failed

6cf6121

changes related to #286 and standardise model data

dd6140b

#286 after merging with #285 and main

4ed330c

added CDDS/standardisation support for two models

23f5754

Update CMEW/flow.cylc

3200651

Better comment Co-authored-by: Alistair Sellar <[email protected]>

Update CMEW/app/standardise_model_data/rose-app.conf

4900f99

Remove comment Co-authored-by: Alistair Sellar <[email protected]>

Update CMEW/app/configure_standardise/bin/configure_standardise.sh

52fe013

Better comment Co-authored-by: Alistair Sellar <[email protected]>

changes implemented as suggested in PR 305 review

2988ecb

used the env variable VARIANT_LABEL

1fd6a96

Merge branch 'main' into 287-feed-model_id-and-variant_label-to-recipe

cc0ab8e

changes to support dual model runs with recipes containing models_id …

71beadb

…and variant_label

mo-nikosbaltas added this to the v0.2.0 (multiple model runs) milestone Dec 30, 2025

mo-nikosbaltas requested a review from alistairsellar December 30, 2025 15:47

mo-nikosbaltas self-assigned this Dec 30, 2025

mo-nikosbaltas added enhancement New feature or request recipe Anything related to ESMValTool rose Anything related to Rose labels Dec 30, 2025

ensures two datasets entries are present

7fbc3d9

mo-nikosbaltas changed the title ~~feed model id and variant label to recipe~~ Feed model_id and variant_label to recipe Dec 31, 2025

mo-nikosbaltas marked this pull request as ready for review December 31, 2025 10:55

alistairsellar requested changes Dec 31, 2025

View reviewed changes

mo-nikosbaltas and others added 5 commits December 31, 2025 12:53

Update CMEW/app/configure_for/bin/test_update_recipe_file.py

7d0d99b

Co-authored-by: Alistair Sellar <[email protected]>

Update CMEW/app/configure_for/bin/test_update_recipe_file.py

b492145

Co-authored-by: Alistair Sellar <[email protected]>

Update CMEW/app/configure_for/bin/test_update_recipe_file.py

adc40d1

Co-authored-by: Alistair Sellar <[email protected]>

added/edited comments suggested

96cc418

Merge branch '287-feed-model_id-and-variant_label-to-recipe' of githu…

d83d0cb

…b.com:MetOffice/CMEW into 287-feed-model_id-and-variant_label-to-recipe

mo-nikosbaltas requested a review from alistairsellar December 31, 2025 14:03

mo-nikosbaltas added 2 commits December 31, 2025 16:00

comments refinement

2067f71

documentation for dual runs

704cdb7

alistairsellar mentioned this pull request Jan 2, 2026

Compare two model development runs in a single run of CMEW #259

Closed

34 tasks

Revert "documentation for dual runs"

eec1dee

This reverts commit 704cdb7.

alistairsellar mentioned this pull request Jan 5, 2026

Enhance testing for shell scripts #308

Open

updated for dual-model run

a040aaf

alistairsellar requested changes Jan 5, 2026

View reviewed changes

alistairsellar and others added 6 commits January 5, 2026 19:14

Merge branch 'main' into 287-feed-model_id-and-variant_label-to-recipe

cc0ee6f

Update doc/source/user_guide/workflow.rst

aecaeaf

Co-authored-by: Alistair Sellar <[email protected]>

Update doc/source/user_guide/workflow.rst

26bd40e

Co-authored-by: Alistair Sellar <[email protected]>

Update doc/source/user_guide/workflow.rst

13327eb

Co-authored-by: Alistair Sellar <[email protected]>

Update doc/source/user_guide/workflow.rst

8cf02ca

Co-authored-by: Alistair Sellar <[email protected]>

Update doc/source/user_guide/workflow.rst

1f5076e

Co-authored-by: Alistair Sellar <[email protected]>

alistairsellar requested changes Jan 6, 2026

View reviewed changes

doc/source/user_guide/workflow.rst Outdated Show resolved Hide resolved

doc/source/user_guide/workflow.rst Outdated Show resolved Hide resolved

mo-nikosbaltas and others added 2 commits January 6, 2026 08:53

Update doc/source/user_guide/workflow.rst

6c37de2

Co-authored-by: Alistair Sellar <[email protected]>

Update doc/source/user_guide/workflow.rst

745927e

Co-authored-by: Alistair Sellar <[email protected]>

mo-nikosbaltas requested a review from alistairsellar January 6, 2026 09:01

alistairsellar approved these changes Jan 6, 2026

View reviewed changes

alistairsellar requested a review from NParsonsMO January 6, 2026 10:07

NParsonsMO requested changes Jan 6, 2026

View reviewed changes

Feed model_id and variant_label to recipe #309

Are you sure you want to change the base?

Feed model_id and variant_label to recipe #309

Conversation

mo-nikosbaltas commented Dec 30, 2025 • edited by NParsonsMO Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR creation checklist for the developer

Definition of Done for the developer

PR creation checklist for the reviewer

Definition of Done for the reviewer

Uh oh!

mo-nikosbaltas commented Dec 30, 2025

When running cylc, run_recipe_radiation_budget failed We get this error from ESMValtool. Looking at job.out:

Uh oh!

alistairsellar commented Dec 30, 2025

Uh oh!

alistairsellar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mo-nikosbaltas commented Jan 5, 2026

Uh oh!

alistairsellar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alistairsellar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

NParsonsMO Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

NParsonsMO Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

alistairsellar Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

NParsonsMO Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NParsonsMO Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NParsonsMO commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Feed `model_id` and `variant_label` to recipe #309

Feed `model_id` and `variant_label` to recipe #309

mo-nikosbaltas commented Dec 30, 2025 •

edited by NParsonsMO

Loading

When running cylc, run_recipe_radiation_budget failed We get this error from ESMValtool.
Looking at job.out:

alistairsellar left a comment •

edited

Loading

NParsonsMO commented Jan 6, 2026 •

edited

Loading