ESM1.6 Read start date and time from restart file

[blimlim]

Opening a draft for discussion.

From #25.

The coupled driver currently initialises the calendar from information in the coupling namelist input_ice.nml. Specifically from inidate (the run start date), init_date (the experiment initialisation date), and runtime0, the seconds between init_date and inidate. This also requires offloading calculations to payu, which recalculate information already stored in the restart file.

This PR initialises the calendar from information in the restart files, since they contain all the information required to setup the calendar (specifically the experiment start date, and time, the seconds since init_date). It also hard codes the init_date to 00010101, which is used as the time origin in the history files.

Testing to follow

[blimlim]

I still need to think through what exactly happens with jobnum=1 after these changes

[anton-seaice]

We don't have a supported usecase for jobnum = 1 , so its not critical

[anton-seaice]

Maybe we should make a seperate issue to investigate this ? Basically to check there are no traps with having jobnum = 1 and runtype = initial trying to have the same meaning.

[blimlim]

Yeah good idea. I've added an issue here

[anton-seaice]

We don't have a supported usecase for jobnum = 1 , so its not critical

[blimlim]

Update on the testing: I've done several 3 month runs starting at different years, (787 non leap, 792 leap, and 9999) with the changes in f9c3978, and also without the changes from this PR.

The restart files at the end are identical, apart from the istep1 global attribute:

DIFFER : VALUES OF GLOBAL ATTRIBUTE : istep1 : 2160 <> 10920

I think the initialisation of istep1 changes when it's read from the restart. At least the physical state hasn't changed anywhere.

Similarly, the ice history files have identical times and data, apart from the frz_onset variable.

With the old code, frz_onset seems to be 1 at several points in the southern hemisphere, and in the new code it varies:

iceh.0787-03.nc
DIFFER : VARIABLE : frz_onset : POSITION : [0,21,91] : VALUES : 1 <> 3
DIFFER : VARIABLE : frz_onset : POSITION : [0,25,94] : VALUES : 1 <> 3
DIFFER : VARIABLE : frz_onset : POSITION : [0,26,94] : VALUES : 1 <> 3
DIFFER : VARIABLE : frz_onset : POSITION : [0,27,92] : VALUES : 1 <> 18
DIFFER : VARIABLE : frz_onset : POSITION : [0,28,93] : VALUES : 1 <> 19
DIFFER : VARIABLE : frz_onset : POSITION : [0,29,94] : VALUES : 1 <> 16
DIFFER : VARIABLE : frz_onset : POSITION : [0,30,93] : VALUES : 1 <> 4
DIFFER : VARIABLE : frz_onset : POSITION : [0,31,87] : VALUES : 1 <> 11
DIFFER : VARIABLE : frz_onset : POSITION : [0,31,293] : VALUES : 1 <> 18
DIFFER : VARIABLE : frz_onset : POSITION : [0,31,294] : VALUES : 1 <> 9
DIFFER : VARIABLE : frz_onset : POSITION : [0,31,305] : VALUES : 1 <> 8
DIFFER : VARIABLE : frz_onset : POSITION : [0,31,306] : VALUES : 1 <> 26
DIFFER : VARIABLE : frz_onset : POSITION : [0,32,303] : VALUES : 1 <> 16
DIFFER : VARIABLE : frz_onset : POSITION : [0,32,338] : VALUES : 1 <> 3
DIFFER : VARIABLE : frz_onset : POSITION : [0,33,34] : VALUES : 1 <> 4
DIFFER : VARIABLE : frz_onset : POSITION : [0,33,48] : VALUES : 1 <> 18
DIFFER : VARIABLE : frz_onset : POSITION : [0,33,327] : VALUES : 1 <> 0
DIFFER : VARIABLE : frz_onset : POSITION : [0,34,54] : VALUES : 1 <> 69
DIFFER : VARIABLE : frz_onset : POSITION : [0,34,333] : VALUES : 1 <> 67
DIFFER : VARIABLE : frz_onset : POSITION : [0,35,333] : VALUES : 1 <> 0

Looking into why this might differ.

[blimlim]

frz_onset is set based on yday, which holds the current day of year (assuming a 1 year run length starting on Jan 1). Through some tests, the difference in istep is unrelated.

Printing yday and each call to calendar for the old and new code:

Old:

 (init_calendar) tday, yday, mday, nyr =    1.00000000000000     
 (calendar) time, yday =   0.000000000000000E+000   1.00000000000000     
 (calendar) time, yday =   0.000000000000000E+000   1.00000000000000     
 (calendar) time, yday =    3600.00000000000        1.00000000000000     
 (calendar) time, yday =    7200.00000000000        1.00000000000000     
 (calendar) time, yday =    10800.0000000000        1.00000000000000     
 (calendar) time, yday =    14400.0000000000        1.00000000000000   

New:

 (init_calendar) tday, yday, mday, nyr =    1.00000000000000     
 (calendar) time, yday =   0.000000000000000E+000   1.00000000000000     
 (calendar) time, yday =    24803712000.0000        287081.000000000     
 (calendar) time, yday =    3600.00000000000        1.00000000000000     
 (calendar) time, yday =    7200.00000000000        1.00000000000000     
 (calendar) time, yday =    10800.0000000000        1.00000000000000     
 (calendar) time, yday =    14400.0000000000        1.00000000000000     

The second call to calendar differs between the two. I believe this comes from this call to calendar under the init_restart subroutine :

cice5/drivers/access/CICE_InitMod.F90

Lines 364 to 369 in 0bd0131

	if (trim(runtype) == 'continue') then
	! start from core restart file
	call restartfile() ! given by pointer in ice_in
	!ars599: 11042014: markout call calendar
	! according to dhb599 initmod at cice4.1_fm
	call calendar(time) ! update time parameters

In the old code, time will equal istep0*dt, which will equal 0 due the value of istep0 in cice_in.nml. At this point, time hasn't yet been reset to runtime0 from input_ice.nml:

cice5/drivers/access/CICE_InitMod.F90

Line 193 in 0bd0131

time = runtime0 !............

Meanwhile in the new code, time will have been initialised from the restart file, and will equal the seconds since init_date. As a result, yday is briefly very large until the next call to calendar where it correctly gets reset to 1.

cice5/drivers/access/CICE_InitMod.F90

Line 215 in 0bd0131

call calendar(time-runtime0)

If replace the calendar call after the restart is read

call calendar(0.0)

we end up with identical frz_onset values between the two versions of the code. Something clearly happens in the following section of the code between L180 and L215 which affects frz_onset but I haven't worked out what it is yet:

cice5/drivers/access/CICE_InitMod.F90

Lines 180 to 215 in 0bd0131

	call init_restart ! initialize restart variables
	#ifdef AusCOM
	if (my_task == master_task) then
	write(il_out,*) 'CICE (cice_init) 2 time = ', my_task, time
	write(il_out,*) 'CICE (cice_init) 2 runtime0 = ', my_task, runtime0
	!write(il_out,*) 'CICE (cice_init) 2 idate = ', my_task, idate
	end if
	if (jobnum == 1 ) then
	time = 0.0 !NOTE, the first job must be set back to 0 and
	idate = idate_save !idate back to the 'initial' value, in any case
	else !BX: 20160720
	time = runtime0 !............
	endif
	#endif
	call init_diags ! initialize diagnostic output points
	call init_history_therm ! initialize thermo history variables
	call init_history_dyn ! initialize dynamic history variables
	! Initialize shortwave components using swdn from previous timestep
	! if restarting. These components will be scaled to current forcing
	! in prep_radiation.
	if (trim(runtype) == 'continue' .or. restart) &
	call init_shortwave ! initialize radiative transfer
	istep = istep + 1 ! update time step counters
	istep1 = istep1 + 1
	time = time + dt ! determine the time and date
	#ifndef AusCOM
	call calendar(time) ! at the end of the first timestep
	#else
	!ars599: 26032014 original code
	! call calendar(time) ! at the end of the first timestep
	call calendar(time-runtime0)

[anton-seaice]

Does frz_onset work if you don't start on Jan 1 ? (I guess it depends what the definition is supposed to be).

With the old code, frz_onset seems to be 1 at several points in the southern hemisphere, and in the new code it varies:

Are the new ones correct now and the old ones were wrong ?

[blimlim]

Does frz_onset work if you don't start on Jan 1 ? (I guess it depends what the definition is supposed to be).

In theory it shouldn't work – frz_onset is initialised to give invalid results if the start day isn't Jan 1:

cice5/source/ice_history.F90

Lines 1500 to 1509 in 0bd0131

	if (restart .and. yday >= c2) then
	! restarting midyear gives erroneous onset dates
	mlt_onset = 999._dbl_kind
	frz_onset = 999._dbl_kind
	else
	mlt_onset = c0
	frz_onset = c0
	endif
	end subroutine init_hist

However in practice, it looks like yday is (incorrectly?) set to 1 just before the init_hist call regardless of the start time.

The init_hist code is run directly after init_calendar:

cice5/drivers/access/CICE_InitMod.F90

Lines 139 to 140 in 0bd0131

	call init_calendar ! initialize some calendar stuff
	call init_hist (dt) ! initialize output history file

and so uses the day-of-year yday calculated by init_calendar. init_calendar calculates yday from istep0 in the namelist, and does not yet have access to the data from the restart file. The calculation looks a bit off, and seems to set yday=1 independent of the value of istep0.

E.g. If I set istep0=4560 (190 days into the year), init_calendar in the old code gives:

 (init_calendar) tday, yday, mday, nyr =    191.000000000000     
   1.00000000000000                1           1

Both the new code and the old code will be using the same value of yday when init_hist is called, so this at least doesn't seem to be the cause of the diverging frz_onset values.

[blimlim]

Are the new ones correct now and the old ones were wrong ?

I'm not sure. From the month 3 output, it's only a few points in the Southern Hemisphere that differ:

Differences start appearing in the second timestep. The following is differences in instantaneous output at timestep 2:

Confusingly, frazil and congel which appear to be the only variables determining frz_onset are identical between the runs.

[blimlim]

The difference in frz_onset doesn't appear to specifically be caused by the changes in the new code, but instead by something strange in this calendar call which affects both the old and new code

cice5/drivers/access/CICE_InitMod.F90

Lines 364 to 369 in 0bd0131

	if (trim(runtype) == 'continue') then
	! start from core restart file
	call restartfile() ! given by pointer in ice_in
	!ars599: 11042014: markout call calendar
	! according to dhb599 initmod at cice4.1_fm
	call calendar(time) ! update time parameters

As mentioned above, the time argument in the new code will come from the restart file, equalling the number of seconds since init_date 01/01/01. In the old code, it will equal 0 since we set istep0=0 in the config (note that this differs from ESM1.5, where istep0 is incorrectly updated by payu each run).

I've added a namelist parameter to the new code test_caltime, to inject different arguments into this call:

cice5/drivers/access/CICE_InitMod.F90

Line 372 in eb23387

call calendar(test_caltime) ! update time parameters

In the old code, we can control the time argument in this call by setting istep0 in the namelist.

Note that anything set by this call will be correctly reset shortly after in the next calendar call:

cice5/drivers/access/CICE_InitMod.F90

Line 214 in eb23387

call calendar(time-runtime0)

I've run the (old, new) code with (istep0, time) values of (0, 0), (1, 3600 [1 timestep]), (4560, 16416000 [190 days]), (8736 ,31449600 [364 days]), (8759 ,31532400 [365 days - 1 timestep]), (8760, 31536000 [365 days]), (3444960, 12401856000 [143540 days]), (6889920, 24803712000, [786 years]).

I ran short 1 day runs for each pair, saving the instantaneous frz_onset_melt. For each pair, the frz_onset_melt at the start of the second day is identical between the old and new code.

All the runs with (istep0, time) less than 1 year have identical output. Likewise all the runs with (istep0, time) greater than one year are identical. However the two groups don't match.

[blimlim]

Since both the old and new code have the same behaviour, where the values of frz_onset change in both depending on the initial time at this calendar call during initialisation

I'm wondering whether the following would make sense?

• Make a separate issue to investigate the behaviour of frz_onset, as it looks like it's not related to the changes made here.
• Hardcode the time argument to 0 in the calendar call in init_restart. Add a comment linking to the above issue. I've tested adding this to the new code and running 5 years. This give identical output and restart files (apart from the istep1 attribute) as the old code with istep0 set to 0.

[anton-seaice]

• Make a separate issue to investigate the behaviour of frz_onset, as it looks like it's not related to the changes made here.

• Hardcode the time argument to 0 in the calendar call in init_restart. Add a comment linking to the above issue. I've tested adding this to the new code and running 5 years. This give identical output and restart files (apart from the istep1 attribute) as the old code with istep0 set to 0.

I think both ?

We should either fix this or consider disabling the diagnostic(s) if we don't need it.

[blimlim]

Apologies for the sidetracks! I think this one's ready for review now. To summarise the changes:

1. init_date hardcoded to 00010101. History output will always have this as the time origin.
2. runtime0, inidate also removed from the input_ice.nml namelist. runtime0 is now set based on the time attribute in the restart file, and inidate is initialised from day, month, year in the restart file.
3. Adds a check at initialisation that runtime0 and the start date read from the restart file are consistent.
4. Hardcode time=0 for the calendar call during init_restart so that frz_onset output doesn't change.

Summary of testing:

• 3 Month runs starting on 1 Jan 787, 792, 9999 using the old and new code. Identical output and restarts to the old code, apart from frz_onset (now fixed), and the istep1 attribute in the restart files.
• 5 year run starting at 787. Identical output and restarts to the old code.

[blimlim]

A payu branch for testing these changes is here: https://github.com/ACCESS-NRI/payu/tree/cice5-resdate-update
Executables here: ACCESS-NRI/ACCESS-ESM1.6#101 (comment)
And a demo config is here: https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/test-cice5-calendar-demo

[anton-seaice]

Example error message:

 CICE: Invalid reference to variable caltype
 CICE: is caltype deprecated ?
 CICE ERROR in input_ice.nml when reading caltype - invalid reference to variabl
 e in NAMELIST input, unit 11, file /scratch/tm70/as2285/access-esm/work/esm1.6-
 preind+c+repeat-dev-preindustrial+concentrations-6a513a44/ice/input_ice.nml, li
 ne 2, position 13                                                     

[anton-seaice]

This looks good! - just waiting for #51

[blimlim]

Example error message:

That looks great, thanks for getting it in!

[anton-seaice]

Thanks @blimlim - we might need another approver again ? :)