[AUTOCUT] Gradle Check Flaky Test Report for SearchServiceTests

[opensearch-ci-bot]

Flaky Test Report for SearchServiceTests

Noticed the SearchServiceTests has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
f2cc655	18104	57849	org.opensearch.search.SearchServiceTests.testConcurrentSegmentSearchWithRandomizedModeSettings
fb77cc7	16908	57852	org.opensearch.search.SearchServiceTests.testConcurrentSegmentSearchWithRandomizedModeSettings

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the SearchServiceTests class are:

• 16366
• 17881
• 17978
• 15016
• 15289
• 17329
• 13172
• 15363
• 16843
• 17752
• 17759
• 18082
• 18085
• 18090
• 18102
• 18103
• 18161
• 18249
• 18277
• 18280
• 18290

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

This can be reproduced pretty easily by running on repeat:

./gradlew ':server:test' --tests "org.opensearch.search.SearchServiceTests.testConcurrentSegmentSearchWithRandomizedModeSettings" -Dtests.iters=100

@Vikasht34 @bowenlan-amzn @jed326 I'm assuming this is related to enabling concurrent search by default. Can someone take a look here?

[sandeshkr419]

https://build.ci.opensearch.org/job/gradle-check/58060/consoleText

Capturing the failure error:

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.search.SearchServiceTests.testConcurrentSegmentSearchWithRandomizedModeSettings" -Dtests.seed=95D69893E135E51F -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=luy -Dtests.timezone=Etc/GMT+2 -Druntime.java=21

SearchServiceTests > testConcurrentSegmentSearchWithRandomizedModeSettings FAILED
    java.lang.AssertionError: expected:<false> but was:<true>
        at __randomizedtesting.SeedInfo.seed([95D69893E135E51F:FBDD89F429A0843D]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at org.opensearch.search.SearchServiceTests.testConcurrentSegmentSearchWithRandomizedModeSettings(SearchServiceTests.java:1571)

Probable related PR: #15363

[kkewwei]

It seems be related to #17978.

Previously the default value was false, it had nothing to do with aggregation. Now default value is auto in case mode setting is not set, it is also decided by aggregationSupportsConcurrent, We should take aggregationSupportsConcurrent into account.