Skip to content

Add summary statistic (mean, max, min,...) plots and introduce additional aggregation options#1377

Merged
ukmo-huw-lewis merged 97 commits intomainfrom
1285-add-summary-statistic-plots
Jul 4, 2025
Merged

Add summary statistic (mean, max, min,...) plots and introduce additional aggregation options#1377
ukmo-huw-lewis merged 97 commits intomainfrom
1285-add-summary-statistic-plots

Conversation

@ukmo-huw-lewis
Copy link
Contributor

@ukmo-huw-lewis ukmo-huw-lewis commented May 20, 2025

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

  • Documentation has been updated to reflect change.
  • New code has tests, and affected old tests have been updated.
  • All tests and CI checks pass.
  • Ensured the pull request title is descriptive.
  • Conda lock files have been updated if dependencies have changed.
  • Attributed any Generative AI, such as GitHub Copilot, used in this PR.
  • Marked the PR as ready to review.

@github-actions
Copy link
Contributor

github-actions bot commented May 20, 2025

Coverage

@ukmo-huw-lewis
Copy link
Contributor Author

See sample output: https://wwwspice/~huw.lewis/CSET/CSET_RAL3_LFRic_Means2/

Based on /home/users/huw.lewis/CSET/CSET/cset-workflow/rose-suite.conf

  extract diagnostics from input (else time-averages computed
independently for 25h vs 24h inputs have different sequence timestamp);
b) Use time bounds in time-statistic plot titles to highlight valid data
period used
cset-workflow). Preserves test harness and command-line users can be more
careful testing like-for-like outputs.
@ukmo-huw-lewis
Copy link
Contributor Author

Example cset-workflow output to support review: https://wwwspice/~huw.lewis/CSET/CSET_RAL3LFRic/WestAfrica_4p4_708_1/

@ukmo-huw-lewis ukmo-huw-lewis marked this pull request as ready for review June 30, 2025 08:09
@ukmo-huw-lewis
Copy link
Contributor Author

Latest update to time_series_case_aggregation_all.yaml recipe now exploits SEQ processing to unpack input cubelists across multiple models/cases, and then output a common 1D cube that includes all times. Example output file below (noting this is only 2 cases, so obvious gap between case dates etc). In general use-case here would be if chunking some input data into cycles and wishing to output continuous timeseries, or to visualise multiple case study outputs on same output:

image

Example cset-workflow outputs for sub-set of requested variables:
https://wwwspice/~huw.lewis/CSET/CSET_testing/SWUK_pr1377/

@ukmo-huw-lewis
Copy link
Contributor Author

ukmo-huw-lewis commented Jul 3, 2025

Example workflow outputs with fuller set of variables and output plot types:
https://wwwspice/~huw.lewis/CSET/CSET_RAL3LFRic/WestAfrica_4p4_708_LOCALDATAb/
This captures 'SEQ' and 'MEAN' outputs for each case, and illustrates additional 'all' aggregation output to distil outputs into single summary plot for each plot type.

@Sylviabohnenstengel
Copy link
Member

Rose gui:
A minor change, but I would suggest for clarity purposes to distinguish more clearly between collapsing per datetime forecast and the aggregation section below that aggregates across datetime cases. As I currently see it the "SPATIAL_SURFACE_FIELD_METHOD" allows collapsing or sequence for each forecast individually.

So how about something along those lines:
description=Select analysis method(s) for output mapped plots. Add all options required.
Leave blank or set to "SEQ" (sequence) for plots each diagnostic output time.
For time-collapsed outputs for each datetime, set to "MEAN", "MAX", "MIN", "STD" etc.
image

@ukmo-huw-lewis
Copy link
Contributor Author

Thanks for comment on gui description. It highlights a need to be clearer, so agree could do with updating. Confess I don't fully know what to call the unit of "chunk" as in general been aiming for avoid "case" also. Not sure datetime is quite right wording either given i) python library and ii) may suggest each datetime in output - i.e. hour-by-hour again.

How about.....

description=Select analysis method(s) for output mapped plots. Add all options required.
Leave blank or set to "SEQ" (sequence) for plots each diagnostic output time.
For time-collapsed outputs for each analysis period, set to "MEAN", "MAX", "MIN", "STD" etc.

This at least ties into the ANALYSIS_LENGTH env variable.

Copy link
Member

@Sylviabohnenstengel Sylviabohnenstengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved subject to addressing minor comments

@ukmo-huw-lewis ukmo-huw-lewis merged commit 47c7d3d into main Jul 4, 2025
8 checks passed
@ukmo-huw-lewis ukmo-huw-lewis deleted the 1285-add-summary-statistic-plots branch July 4, 2025 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate case-study or time-period mean/accumulated summary map plots

3 participants