The CICE5 tests @anton-seaice is running are producing daily ice outputs. COSIMA found these were very large and use nco to concatenate each month into a single file.
For the example runs daily outputs total 2.3G/year
$ du -shc HISTORY/iceh.0792-??-??.nc | grep total
2.3G total
Looking at just a single month:
$ du -shc HISTORY/iceh.0792-01-??.nc | grep total
194M total
Compressing to level 1 with shuffle reduces the size to 12M
$ ncrcat -O -L 1 -7 HISTORY/iceh.0792-01-??.nc iceh.0792-01-daily.nc
$ du -sh iceh.0792-01-daily.nc
12M iceh.0792-01-daily.nc
this is a 16x reduction in disk usage and should definitely be done as a post-processing step.
The CICE5 tests @anton-seaice is running are producing daily ice outputs. COSIMA found these were very large and use nco to concatenate each month into a single file.
For the example runs daily outputs total 2.3G/year
Looking at just a single month:
Compressing to level 1 with shuffle reduces the size to 12M
this is a 16x reduction in disk usage and should definitely be done as a post-processing step.