Add test for tar deletion

[forsyth2]

Summary

Objectives:

• Add test for tar deletion. It should fail on main when using --non-blocking, given [Bug]: tar files are not deleted after successful globus transfer #374. After merging Delete transferred files #405, it should pass. (For this reason, I specifically made this a separate pull request, allowing us to test both on main and with the fixes of Delete transferred files #405).

Issue resolution:

• This adds an explicit test for [Bug]: tar files are not deleted after successful globus transfer #374.
• This will allow us to check if Delete transferred files #405 solves the issue.

Select one: This pull request is...

• a bug fix: increment the patch version
• a small improvement: increment the minor version
• a new feature: increment the minor version
• an incompatible (non-backwards compatible) API change: increment the major version

Small Change

• To merge, I will use "Squash and merge". That is, this change should be a single commit.
• Logic: I have visually inspected the entire pull request myself.
• Pre-commit checks: All the pre-commits checks have passed.

[forsyth2]

Passes for blocking, fails for non-blocking, confirming #374.

[forsyth2]

Using the code of #405, all 4 tests pass. Using the code of main the following failures occur:

• Test 2 (test_globus_tar_deletion ${path_to_repo} LCRC_IMPROV_DTN_ENDPOINT ${chrysalis_dst_dir} "non-blocking" "non-keep") fails with Not-expected grep '000000.tar' was found in ls_non-blocking_non-keep_src3_output.log. Test failed., which reproduces the error of [Bug]: tar files are not deleted after successful globus transfer #374. That is, tar files were not being deleted after successful transfer when --non-blocking (but not --keep) was set.
• Test 4 (test_globus_tar_deletion ${path_to_repo} LCRC_IMPROV_DTN_ENDPOINT ${chrysalis_dst_dir} "non-blocking" "keep") fails with Expected grep 'non-blocking_keep' not found in ls_non-blocking_keep_dst_output.log. Test failed. That's an interesting failure. I can check:

> ls /home/ac.forsyth2/zstash_tests/test_globus_tar_deletion_test_main_try2/
blocking_keep  blocking_non-keep  non-blocking_keep
> ls /home/ac.forsyth2/zstash_tests/test_globus_tar_deletion_test_main_try2/non-blocking_keep/
000000.tar  index.db

# Yet, we only get:
> cat /home/ac.forsyth2/ez/zstash/tests/utils/globus_tar_deletion/non-blocking_keep/ls_non-blocking_keep_dst_output.log
blocking_keep
blocking_non-keep

[forsyth2]

Remaining TODO:

• Determine why Test 4 above is failing on main (plus this PR). It's a strange failure, but since the tests pass in Delete transferred files #405, maybe it's irrelevant.
• Code review & confirm design decisions of Delete transferred files #405.
• Merge this PR
• Merge Delete transferred files #405

[forsyth2]

@chengzhuzhang This is just a test addition, but let me know if you would like to review before I merge it.

The results are:

==========================================
TEST RESULTS
==========================================
✓ blocking_non-keep PASSED
✗ non-blocking_non-keep FAILED
✓ blocking_keep PASSED
✓ non-blocking_keep PASSED
==========================================
TEST SUMMARY
==========================================
Total tests: 4
Passed: 3
Failed: 1
==========================================

This is exactly as expected, given #374. I decided to make this test addition a distinct PR from the fix #405 because I wanted to make sure it showed this failure when testing on main. Alternatively, this test could simply be merged as part of that PR.

[chengzhuzhang]

@forsyth2 I I think the change looks okay. my preference would be having this type of specific testing change together with the actual code change to reduce the number of PR and review cycle.

[forsyth2]

my preference would be having this type of specific testing change together with the actual code change to reduce the number of PR and review cycle.

Ok, I can include it as part of #405 then. I mainly just wanted to make sure it did in fact fail on main, which is of course not quite possible if it's included on a branch with fixes to main.

[chengzhuzhang]

Ok, I can include it as part of #405 then. I mainly just wanted to make sure it did in fact fail on main, which is of course not quite possible if it's included on a branch with fixes to main.

I think you can just merge this pull request as is. For future testing PRs, we can consider to bundle with the PR with actual code change.

[forsyth2]

Sounds good, thanks, merging.