[BUG] org.opensearch.search.SearchTimeoutIT.testSimpleTimeout fails intermittently #9401
Description
References
#9378 (comment)
#7357 (comment)
https://build.ci.opensearch.org/job/gradle-check/22854/testReport/junit/org.opensearch.search/SearchTimeoutIT/testSimpleTimeout__p0___search_concurrent_segment_search_enabled___true___/
To Reproduce
Not able to reproduce locally, however this is the reproduction seed provided by the jenkins job.
./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.SearchTimeoutIT" -Dtests.method="testSimpleTimeout {p0={"search.concurrent_segment_search.enabled":"true"}}" -Dtests.seed=4A4EAA35C9A6ED23 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=id -Dtests.timezone=Africa/Algiers -Druntime.java=20
Additional Details
The assertion that is failing is asserting that there were no shard failures in the search request. This assertion failing means that for whatever reason one a shard encountered an exception executing the search request.