Minimise duplication of the CDDS default request items #349

zmaalick · 2026-01-20T09:57:46Z

Closes #209 .

PR creation checklist for the developer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the developer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately, including the Quick Start section?
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

PR creation checklist for the reviewer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the reviewer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately, including the Quick Start section?
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

NParsonsMO

The unit test for configure_standardise/bin/test_create_request_file.py::test_create_request fails.

I also think that the defaults file is not supposed to contain user-configurable variables.

NParsonsMO · 2026-01-21T16:35:08Z

CMEW/app/configure_standardise/etc/request_defaults.cfg

+[metadata]
+branch_method = no parent
+calendar = 360_day
+base_date = 1850-01-01T00:00:00


Any reason this wasn't alphabetical (at the top)? Just slightly easier to find things in future if they are sorted.

521d817

Sort the metadata list. but under [data], start_date and end_date should come together in my opinion. which makes sorting not always a case in such configuration files.

I fully agree on the dates. And they were not alphabetical in that section, presumably for that reason.

But in the main branch's request_ref.cfg and request_eval.cfg, base_date was at the top (line 2), rather than line 4 (first review) or line 3 (re-review). It doesn't make a functional difference, I just wondered why it changed.

It was slightly harder to compare the two outputs (the date change seemed well worth that added few seconds).

CMEW/app/configure_standardise/etc/request_defaults.cfg

NParsonsMO · 2026-01-21T16:37:49Z

CMEW/app/configure_standardise/etc/request_defaults.cfg

+branch_method = no parent
+calendar = 360_day
+base_date = 1850-01-01T00:00:00
+experiment_id = amip


This will also configurable following PR #332.

same as above comment

I am leaving this comment open, as PR #332 is further through the review process, so we will need to accept that change instead of this one when merging main into this branch.

CMEW/app/configure_standardise/etc/request_defaults.cfg

CMEW/app/configure_standardise/rose-app.conf

NParsonsMO · 2026-01-21T17:28:21Z

CMEW/app/configure_standardise/rose-app.conf

 mode=mkdir
+
+[file:$CYLC_WORKFLOW_SHARE_DIR/etc/request_defaults.cfg]
+source=${CYLC_WORKFLOW_RUN_DIR}/app/configure_standardise/etc/request_defaults.cfg


I hoped on seeing this that it might be a way round the unit test failures that I encountered in PR #324, but unfortunately it isn't, and it has a very similar error itself.
When I run pytest myself, all the tests pass fine. But when I do cylc vip -O metoffice -O unittest, I get a file not found error for the file above, ending as below:

cfg_path = base_dir / "etc" / "request_defaults.cfg" if not cfg_path.exists(): > raise FileNotFoundError(f"Defaults file not found: {cfg_path}") E FileNotFoundError: Defaults file not found: [MY_DIR]/CMEW/run3/share/etc/request_defaults.cfg configure_standardise/bin/create_request_file.py:25: FileNotFoundError

However here, the unit test also does not pass with the command cylc vip -O metoffice -O test (as opposed to unittest) either.

Fix the unittests: 7bcab81

NParsonsMO · 2026-01-21T17:34:41Z

CMEW/app/configure_standardise/bin/create_request_file.py

+        # Running inside Cylc
+        base_dir = Path(os.environ["CYLC_WORKFLOW_SHARE_DIR"])
+    else:
+        # Running under pytest / unittest / local execution


This doesn't seem to catch everything, see comment on rose-app.conf

Fixed it: 7bcab81

CMEW/app/configure_standardise/etc/request_defaults.cfg

NParsonsMO

There are merge conflicts that need to be sorted,

and I assume that this needs to be pre-review (although I'm not sure it's come up yet)?

I've left two comments open to check those lines on merging with main.

All unit tests now passing in all methods.

One copyright change requested.

NParsonsMO · 2026-01-22T12:45:33Z

CMEW/app/configure_standardise/etc/request_defaults.cfg

+branch_method = no parent
+calendar = 360_day
+base_date = 1850-01-01T00:00:00
+experiment_id = amip


I am leaving this comment open, as PR #332 is further through the review process, so we will need to accept that change instead of this one when merging main into this branch.

NParsonsMO · 2026-01-22T12:50:35Z

CMEW/app/configure_standardise/bin/create_request_file.py

-        "package": "round-1",
        "root_proc_dir": os.environ["ROOT_PROC_DIR"],
        "root_data_dir": os.environ["ROOT_DATA_DIR"],
        "workflow_basename": "CMEW",


Once main has been merged in I think this will update, but I am commenting here to check.

CMEW/app/configure_standardise/etc/request_defaults.cfg

NParsonsMO · 2026-01-22T12:55:38Z

CMEW/app/configure_standardise/etc/request_defaults.cfg

@@ -0,0 +1,44 @@
+# (C) Crown Copyright 2022-2026, Met Office.


It's only 2026, not 2022-2026.
The year of first publication of the specific file:
https://github.com/MetOffice/CMEW/wiki/Detailed-Working-Practices#coding-requirements

add a cfg file for default values

cd9177c

zmaalick self-assigned this Jan 20, 2026

zmaalick changed the title ~~add a cfg file for default values~~ Minimise duplication of the CDDS default request items Jan 20, 2026

zmaalick added enhancement New feature or request good first issue Good for newcomers technical debt Technical debt in CMEW standardise Anything related to CDDS quality assurance Anything related to Quality Assurance (QA) labels Jan 20, 2026

zmaalick added this to the v0.2.0 (multiple model runs) milestone Jan 20, 2026

update copywrite

87286c5

NParsonsMO self-requested a review January 21, 2026 16:29

NParsonsMO requested changes Jan 21, 2026

View reviewed changes

zmaalick added 4 commits January 22, 2026 08:33

update config list with sorting metadata and adding copywrite

521d817

update copywrite

6d7edff

fix the unittest runs

7bcab81

update with precommit checks

8526711

NParsonsMO requested changes Jan 22, 2026

View reviewed changes

		@@ -0,0 +1,44 @@
		# (C) Crown Copyright 2022-2026, Met Office.

Minimise duplication of the CDDS default request items #349

Are you sure you want to change the base?

Minimise duplication of the CDDS default request items #349

Uh oh!

Conversation

zmaalick commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR creation checklist for the developer

Definition of Done for the developer

PR creation checklist for the reviewer

Definition of Done for the reviewer

Uh oh!

NParsonsMO left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NParsonsMO Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NParsonsMO Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NParsonsMO left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zmaalick commented Jan 20, 2026 •

edited

Loading

NParsonsMO Jan 22, 2026 •

edited

Loading

NParsonsMO Jan 21, 2026 •

edited

Loading

NParsonsMO left a comment •

edited

Loading