Replace CDAT

[forsyth2]

conda-forge/cdtime-feedstock#52 mentions that we need to replace CDAT in zppy. Note that we already have #80, which is specifically about cdscan.

[forsyth2]

@tomvothecoder I'm not quite sure what needs to be done for zppy.

$ git grep -n cdscan
docs/source/dev_guide/new_diags_set.rst:100:        cdscan -x ${xml_name} ${rofDir}/${v}_*.nc
zppy/templates/e3sm_diags.bash:104:        # Add this time series file to the list of files for cdscan to use
zppy/templates/e3sm_diags.bash:111:    cdscan -x ${xml_name} -f ${v}_files.txt
zppy/templates/e3sm_diags.bash:132:  cdscan -x ${xml_name} ${ts_rof_dir_source}/${v}_*.nc
zppy/templates/global_time_series.bash:46:cdscan -x glb.xml *.nc
zppy/templates/global_time_series.bash:70:    cdscan -x glb.xml mpaso.glb*.nc

$ git grep -n cdat
# No matches

$ git grep -n cdms
zppy/templates/readTS.py:1:import cdms2
zppy/templates/readTS.py:10:        self.f = cdms2.open(filename)

$ emacs conda/dev.yml
# No dependencies appear obviously related to cdat

$ emacs conda/meta.yml
# No dependencies appear obviously related to cdat

[tomvothecoder]

@forsyth2 I don't have familiarity with the zppy codebase to give input on what needs to be done specifically.

My general approach to refactoring is to identify what needs to be refactored, understand the implementation, and find alternative solutions that would reproduce the same/similar results.

[tomvothecoder]

It looks like you identified the CDAT modules that are used in zppy, which is a good first step. Luckily, the dependency is not that significant from what I can tell.

[forsyth2]

Ok, great!

https://cdat.llnl.gov/ lists "specific CDAT packages (cdms2, cdat_info, cdutil, cdtime, genutil, libcdms)". Out of those, it looks like zppy has:

$ git grep -n cdms2
zppy/templates/readTS.py:1:import cdms2
zppy/templates/readTS.py:10:        self.f = cdms2.open(filename)

$ git grep -n cdutil
zppy/templates/readTS.py:2:import cdutil
zppy/templates/readTS.py:65:            v = cdutil.YEAR(v)

So, basically I just need replacements for cdcscan, cdms2.open, and cdutil.YEAR.

[tomvothecoder]

Those are the packages whose dependencies are being maintained until end of CY2023.

I doubt zppy uses other modules from CDAT. It might be a good idea to check for other modules that operate on cdms2 objects like MV2 though.

• Example from e3sm_diags: Explore replacement of CDAT modules with xarray/XCDAT e3sm_diags#607
• Complete list of CDAT modules: https://anaconda.org/cdat/repo

[forsyth2]

Thanks, out of the list at https://anaconda.org/cdat/repo, we just have:

$ git grep -n cdms2
zppy/templates/readTS.py:1:import cdms2
zppy/templates/readTS.py:10:        self.f = cdms2.open(filename)

$ git grep -n cdutil
zppy/templates/readTS.py:2:import cdutil
zppy/templates/readTS.py:65:            v = cdutil.YEAR(v)

[forsyth2]

We can probably use xarray to replace cdms2.open, and an XML reader to replace cdscan (mfopen to open multiple files). May have to replace with multiple tools.

[forsyth2]

@tomvothecoder What's the timeline/deadline for this issue? Are we trying not to have anything CDAT-related by the next Unified release?

[tomvothecoder]

@forsyth2 Just CC'd you in an email chain with Xylar. He gave us 6-9 months to refactor any E3SM packages from CDAT to alternative solutions so they can work in E3SM Unified.

It looks like zppy's CDAT dependency is minimal and much more straightforward to refactor compared to e3sm_diags.