Conversation
Collaborator
Author
|
This organization also creates a good foundation for future improvements to the image checker, such as:
|
14 tasks
forsyth2
changed the title
forsyth2
changed the title
forsyth2
force-pushed
the
image-checker
branch
2 times, most recently
from
ac9b418 to
c00d1f9
Compare
Collaborator
Author
|
Check that names/paths were all updated properly: That gives us:
In short, all file paths and names relevant for legacy testing appear to be up-to-date. |
forsyth2
commented
Jun 16, 2025
tests/integration/image_checker.py
Outdated
|
|
||
| # Make diff_dir readable | ||
| if os.path.exists(parameters.diff_dir): | ||
| os.system(f"chmod -R 755 {parameters.diff_dir}") |
Collaborator
Author
There was a problem hiding this comment.
To address these concerns, we could rewrite using os.chmod. LivChat suggests:
import os
import stat
def chmod_recursive(path, mode):
for root, dirs, files in os.walk(path):
for name in dirs:
dir_path = os.path.join(root, name)
os.chmod(dir_path, mode)
for name in files:
file_path = os.path.join(root, name)
os.chmod(file_path, mode)
# Also chmod the root directory itself
os.chmod(path, mode)
# Usage
chmod_recursive(parameters.diff_dir, 0o755)
Collaborator
Author
|
Testing image-checker branch # 1. Set up environments
lcrc_conda # Function to set up conda locally
# 1a. e3sm_diags
# Use previously made environment, e3sm-diags-main-20250613
# 1b. zppy-interfaces
# Use Unified
# 1c. zppy
cd ~/ez/zppy
git branch
# On image-checker branch
git status
# Check for uncommitted changes
git log
# The 6 commits of the image-checker PR #720
# The provenance cfg PR #713
conda clean --all --y
conda env create -f conda/dev.yml -n zppy-image-checker-20250616
conda activate zppy-image-checker-20250616
pip install .
# 2. Run unit tests
pytest tests/test_*.py
# 25 passed, 1 warning in 0.34s
# /gpfs/fs1/home/ac.forsyth2/ez/zppy/zppy/utils.py:191: DeprecationWarning: invalid escape sequence \.
# 3. Set up zppy runs
# Edit tests/integration/utils.py:
# UNIQUE_ID = "test_weekly_20250616"
# "diags_environment_commands": "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm-diags-main-20250613",
# "global_time_series_environment_commands": "source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh",
python tests/integration/utils.py
# 4. Launch zppy jobs to produce actual images for image checker test
# 4a. Legacy tests
zppy -c tests/integration/generated/test_weekly_legacy_3.0.0_comprehensive_v3_chrysalis.cfg
zppy -c tests/integration/generated/test_weekly_legacy_3.0.0_comprehensive_v2_chrysalis.cfg
zppy -c tests/integration/generated/test_weekly_legacy_3.0.0_bundles_chrysalis.cfg # Runs 1st part of bundles cfg
# 4b. Regular tests
zppy -c tests/integration/generated/test_weekly_comprehensive_v3_chrysalis.cfg
zppy -c tests/integration/generated/test_weekly_comprehensive_v2_chrysalis.cfg
zppy -c tests/integration/generated/test_weekly_bundles_chrysalis.cfg # Runs 1st part of bundles cfg
# Wait
zppy -c tests/integration/generated/test_weekly_legacy_3.0.0_bundles_chrysalis.cfg # Runs 2nd part of bundles cfg
zppy -c tests/integration/generated/test_weekly_bundles_chrysalis.cfg # Runs 2nd part of bundles cfg
# Wait
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_legacy_3.0.0_comprehensive_v3_output/test_weekly_20250616/v3.LR.historical_0051/post/scripts/ && grep -v "OK" *status
# No errors
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_legacy_3.0.0_comprehensive_v2_output/test_weekly_20250616/v2.LR.historical_0201/post/scripts/ && grep -v "OK" *status
# No errors
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_legacy_3.0.0_bundles_output/test_weekly_20250616/v3.LR.historical_0051/post/scripts/ && grep -v "OK" *status
# No errors
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_comprehensive_v3_output/test_weekly_20250616/v3.LR.historical_0051/post/scripts/ && grep -v "OK" *status
# No errors
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_comprehensive_v2_output/test_weekly_20250616/v2.LR.historical_0201/post/scripts/ && grep -v "OK" *status
# No errors
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_bundles_output/test_weekly_20250616/v3.LR.historical_0051/post/scripts/ && grep -v "OK" *status
# No errors
# 5. Run zppy integration tests
cd ~/ez/zppy
pytest tests/integration/test_images.py
# 1 failed in 2965.67s (0:49:25)
# See results of `cat test_images_summary.md` below.
pytest tests/integration/test_bundles.py
# 2 passed in 0.65s
pytest tests/integration/test_bash_generation.py
# 1 failed in 1.31s
# Only in test_bash_generation_output/post/scripts: global_time_series_None_0001-0020.bash
# Only in test_bash_generation_output/post/scripts: global_time_series_None_0001-0020.settings
# Only in test_bash_generation_output/post/scripts: provenance.20250610_210055_333029.cfg
pytest tests/integration/test_campaign.py
# 6 failed in 1.77s
# Only in test_campaign_water_cycle_override_output/post/scripts: provenance.20250610_210140_098761.cfg
pytest tests/integration/test_defaults.py
# 1 failed in 0.39s
# Only in test_defaults_output/post/scripts: global_time_series_None_0001-0020.settings
# Only in test_defaults_output/post/scripts: provenance.20250610_210239_847055.cfg
pytest tests/integration/test_last_year.py
# 1 failed in 0.32s
# ['climo_atm_monthly_180x360_aave_0001-0010.bash', 'climo_atm_monthly_diurnal_8xdaily_180x360_aave_0001-0010.bash', 'e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_0001-0010.bash', 'mpas_analysis_ts_0001-0010_climo_0001-0010.bash', 'provenance.20250610_210436_001646.cfg', 'tc_analysis_0001-0010.bash', 'ts_atm_daily_180x360_aave_0001-0010-0010.bash', 'ts_atm_monthly_180x360_aave_0001-0010-0010.bash', 'ts_atm_monthly_glb_0001-0010-0010.bash', 'ts_land_monthly_0001-0010-0010.bash', 'ts_rof_monthly_0001-0010-0010.bash']
# ['climo_atm_monthly_180x360_aave_0001-0010.bash', 'climo_atm_monthly_diurnal_8xdaily_180x360_aave_0001-0010.bash', 'e3sm_diags_atm_monthly_180x360_aave_model_vs_obs_0001-0010.bash', 'mpas_analysis_ts_0001-0010_climo_0001-0010.bash', 'tc_analysis_0001-0010.bash', 'ts_atm_daily_180x360_aave_0001-0010-0010.bash', 'ts_atm_monthly_180x360_aave_0001-0010-0010.bash', 'ts_atm_monthly_glb_0001-0010-0010.bash', 'ts_land_monthly_0001-0010-0010.bash', 'ts_rof_monthly_0001-0010-0010.bash']
# The first one (actual) has 'provenance.20250610_210436_001646.cfg'
# The "None" subsection in paths might be from stale files in the "actual" dir
# Let's delete those and rerun.
rm -rf test_bash_generation_output test_campaign_cryosphere_output test_campaign_cryosphere_override_output test_campaign_high_res_v1_output test_campaign_none_output test_campaign_water_cycle_output test_campaign_water_cycle_override_output test_defaults_output test_last_year_output
pytest tests/integration/test_bash_generation.py
# 1 failed in 1.13s
# Only in test_bash_generation_output/post/scripts: provenance.20250617_005905_907190.cfg
# Yes, "None" in name is gone
pytest tests/integration/test_campaign.py
# Only in test_campaign_cryosphere_output/post/scripts: provenance.20250617_005930_327882.cfg
# (And similar for other tests)
pytest tests/integration/test_defaults.py
# Only in test_defaults_output/post/scripts: provenance.20250617_010015_773183.cfg
# Yes, "None" in name is gone
pytest tests/integration/test_last_year.py
# 1 failed in 0.32s
# provenance.cfg in the "actual"All non-image diffs are simply because of the addition of the provenance cfg (#713) Summary of test resultsDiff subdir is where to find the lists of missing/mismatched images, the image diff grid, and the individual diffs.
|
Collaborator
Author
UpdateTo address the Codacy errors: # Update chmod function
# Add line to remove existing diff_dirs
pytest tests/integration/test_images.py
# 1 failed in 2936.95s (0:48:56)Gives: Summary of test resultsDiff subdir is where to find the lists of missing/mismatched images, the image diff grid, and the individual diffs.
No changes, and I can access image check failures for: |
Collaborator
Author
|
The first two commits form the bulk of this PR and were reviewed by @chengzhuzhang as part of the review for #722. The remaining commits are small fixes, that I've run the full test suite on. All the GitHub checks pass now as well. So, this is ready to merge. |
Collaborator
Author
Collaborator
Author
Well they appear to... |
Collaborator
Author
|
It appears this might be more of a warning than a direct error; but perhaps worthwhile to see if just running with read (and no execute) permission will work. |
Collaborator
Author
|
Here's what LivChat has to say: Codacity's Typical Policy
@chengzhuzhang Do you think (And after this merges, I can rebase #722 on the updated |
forsyth2
commented
Jun 17, 2025
tests/integration/image_checker.py
Outdated
|
|
||
| # Make diff_dir readable | ||
| if os.path.exists(parameters.diff_dir): | ||
| _chmod_recursive(parameters.diff_dir, 0o644) |
Collaborator
Author
There was a problem hiding this comment.
0o644 does seem to work.
Collaborator
Author
|
I've determined that |
Collaborator
Author
|
I've squashed the chmod fixes into a single commit, so the commit history is now:
This should be good to merge now. It looks like all the GitHub checks pass. |
This was referenced Aug 8, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Objectives:
tests/integration/image_checker.py.zppy-testing specifics into higher-level functions.Select one: This pull request is...
Big Change
1. Does this do what we want it to do?
Required:
2. Are the implementation details accurate & efficient?
Required:
If applicable:
zppy-interfaces/conda, not just animportstatement.3. Is this well documented?
Required:
4. Is this code clean?
Required:
If applicable: