test: Fix flaky test TestRingHash_RequestHashKeyConnecting in ringhash_e2e_test.go.#9079
Open
Pranjali-2501 wants to merge 4 commits intogrpc:masterfrom
Open
test: Fix flaky test TestRingHash_RequestHashKeyConnecting in ringhash_e2e_test.go.#9079Pranjali-2501 wants to merge 4 commits intogrpc:masterfrom
Pranjali-2501 wants to merge 4 commits intogrpc:masterfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9079 +/- ##
==========================================
- Coverage 80.52% 77.32% -3.21%
==========================================
Files 413 413
Lines 33543 43463 +9920
==========================================
+ Hits 27012 33607 +6595
- Misses 4316 7283 +2967
- Partials 2215 2573 +358 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #9030
Problem
The flakiness was caused by the following race condition in the test execution flow:
The Fix
Refactored the test to use a deterministic negative assertion flow that eliminates background races:
The second RPC is now sent with a short timeout and WaitForReady(true). Because the first connection is held, the second RPC blocks and fails with DeadlineExceeded as expected. We verify the failure and assert that no new connection attempts were triggered during this time. Then, we resume the hold in the main thread to let the test finish.
Verified it on forge for a million times: link
RELEASE NOTES: N/A