Update auth tests

[forsyth2]

Summary

Objectives:

• Improve Globus authentication testing by adding the ability to test on more machines and Globus endpoints.
• Resolve Globus auth issue noted in E3SM Unified 1.12.0 testing #391 (comment)

Select one: This pull request is...

• a bug fix: increment the patch version
• a small improvement: increment the minor version
• a new feature: increment the minor version
• an incompatible (non-backwards compatible) API change: increment the major version

Small Change

• To merge, I will use "Squash and merge". That is, this change should be a single commit.
• Logic: I have visually inspected the entire pull request myself.
• Pre-commit checks: All the pre-commits checks have passed.

[forsyth2]

TODO:

• Run full test suite, including the new interactive globus_auth.bash, on Chrysalis, Perlmutter, Compy

[forsyth2]

This is the fix for the issue noted in the E3SM Unified 1.12.0rc2 error

[forsyth2]

@chengzhuzhang @golaz A few notes on zstash behavior:

Case 1: no external changes

test_different_endpoint1

1. Use zstash with destination endpoint A (accomplished in the test by previously running test_single_auth_code)
2. Use zstash with destination endpoint B
3. Fails with TransferAPIError

Case 2: revoke consents then remove token file

test_different_endpoint2

1. Use zstash with destination endpoint A (accomplished in the test by previously running test_different_endpoint1)
2. Revoke consents
3. Use zstash with destination endpoint B
4. Fails with AuthAPIError 'Bad Request' (This is the source of the error encountered in Unified testing)
5. Remove the token file.
6. Try again: zstash with destination endpoint B
7. Success

Case 3: Just remove the token file

test_different_endpoint3

1. Use zstash with destination endpoint A (accomplished in the test by previously running test_different_endpoint2)
2. Remove the token file (This is the fix to the error encountered in Unified testing)
3. Use zstash with destination endpoint B
4. Success

Conclusion

As you can see, case 3 is the simplest successful solution.

Good: It means that we can have multiple Globus consents simultaneously registered at https://auth.globus.org/v2/web/consents > Globus Endpoint Performance Monitoring, perhaps associated with different Globus endpoints.

Possible future improvements: At the moment, zstash can only support one saved local token at a time. That means if you're using zstash on multiple destination endpoints, you'll have to re-authenticate every time you switch. I think this should be possible to change in the future, but I imagine it is out of scope for the Unified testing period. Please let me know if that should be priortized.

[chengzhuzhang]

The fix for the failed test looks fine to me. The test uses ~/.zstash_globus_tokens.json and currently deletes it, it might be safer to use a tmp directory for storing the token for the test, but this can be addressed later.

[forsyth2]

if you're using zstash on multiple destination endpoints, you'll have to re-authenticate every time you switch. I think this should be possible to change in the future

Addressed in Point 1 of #398.

The test uses ~/.zstash_globus_tokens.json and currently deletes it, it might be safer to use a tmp directory for storing the token for the test, but this can be addressed later.

Addressed in Point 2 of #398.

[forsyth2]

Run full test suite, including the new interactive globus_auth.bash, on Chrysalis, Perlmutter, Compy

• Ran machine-independent tests + Chrysalis-specific tests on Chrysalis
• Ran machine-independent tests + Perlmutter-specific tests on Perlmutter
• Ran machine-independent tests on Compy. The globus_auth.bash test is difficult to run there because the Compy Globus endpoint is finicky about what file paths it can transfer to/from. I think trying to get it working perfectly on Compy as a 3rd machine has diminishing returns on time spent debugging.

The GitHub Actions are passing, so merging.