[AUTOCUT] Gradle Check Flaky Test Report for IngestFromKafkaIT

[opensearch-ci-bot]

Flaky Test Report for IngestFromKafkaIT

Noticed the IngestFromKafkaIT has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
8ffa576	20940	73372	org.opensearch.plugin.kafka.IngestFromKafkaIT.testDynamicUpdateKafkaParams
main	null	62752 62902 63631 64397 66445 67850 69104 69182 69208 69571 73021 73130	org.opensearch.plugin.kafka.IngestFromKafkaIT.testAllActiveIngestion org.opensearch.plugin.kafka.IngestFromKafkaIT.testAllActiveOffsetBasedLag org.opensearch.plugin.kafka.IngestFromKafkaIT.testKafkaIngestion_RewindByOffset org.opensearch.plugin.kafka.IngestFromKafkaIT.testUpdateWithoutIDField org.opensearch.plugin.kafka.IngestFromKafkaIT.testAllActiveIngestionBatchStartPointerOnReplicaPromotion org.opensearch.plugin.kafka.IngestFromKafkaIT.testDynamicConfigUpdateOnNoMessages org.opensearch.plugin.kafka.IngestFromKafkaIT.testKafkaIngestion org.opensearch.plugin.kafka.IngestFromKafkaIT.testMessageOperationTypes
08ed9ee	19480	64831	org.opensearch.plugin.kafka.IngestFromKafkaIT.testUpdateWithoutIDField
b9c5bc7	19184	63789	org.opensearch.plugin.kafka.IngestFromKafkaIT.testCloseIndex
e25f332	18031	56911	org.opensearch.plugin.kafka.IngestFromKafkaIT.testCloseIndex
c472314	17640	54929	org.opensearch.plugin.kafka.IngestFromKafkaIT.testCloseIndex
852011a	16021	53136	org.opensearch.plugin.kafka.IngestFromKafkaIT.classMethod
20a58eb	17194	52946	org.opensearch.plugin.kafka.IngestFromKafkaIT.classMethod
d601c37	17121	52942	org.opensearch.plugin.kafka.IngestFromKafkaIT.classMethod

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the IngestFromKafkaIT class are:

• 16958
• 18333
• 17631
• 20526
• 18332
• 18693
• 17752
• 17240
• 17241
• 17638
• 18249
• 19046
• 19320
• 19635
• 14912
• 17101
• 17132
• 17252
• 17273
• 17359
• 17365
• 17538
• 17565
• 17575
• 17771
• 17922
• 17925
• 18036
• 18106
• 18193
• 18255
• 18451
• 18454
• 18509
• 18530
• 18683
• 18827
• 19009
• 19335
• 19489
• 19540
• 19769
• 19878
• 20874
• 20929
• 20936
• 20949
• 20987

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

@yupeng9 Unfortunately it looks like there are still cases where the thread leak failure happens :(

[yupeng9]

ack, can take on this issue and investigate

[andrross]

Another failure here: https://build.ci.opensearch.org/job/gradle-check/56900/consoleText

REPRODUCE WITH: ./gradlew ':plugins:ingestion-kafka:internalClusterTest' --tests "org.opensearch.plugin.kafka.IngestFromKafkaIT.testPluginsAreInstalled" -Dtests.seed=5D3C7ADEF6C7F804 -Dtests.security.manager=false -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=uz-Arab -Dtests.timezone=Africa/Timbuktu -Druntime.java=21

IngestFromKafkaIT > testPluginsAreInstalled FAILED
    org.awaitility.core.ConditionTimeoutException: Condition org.opensearch.plugin.kafka.KafkaUtils$$Lambda/0x000071e77c743558 was not fulfilled within 3 seconds.
        at __randomizedtesting.SeedInfo.seed([5D3C7ADEF6C7F804:2A88A93E0A1AC475]:0)
        at app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
        at app//org.opensearch.plugin.kafka.KafkaUtils.createTopic(KafkaUtils.java:56)
        at app//org.opensearch.plugin.kafka.KafkaIngestionBaseIT.setupKafka(KafkaIngestionBaseIT.java:79)
        at app//org.opensearch.plugin.kafka.KafkaIngestionBaseIT.setup(KafkaIngestionBaseIT.java:63)
        at java.****@21.0.6/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.****@21.0.6/java.lang.reflect.Method.invoke(Method.java:580)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:972)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at app//org.opensearch.test.OpenSearchTestClusterRule$1.evaluate(OpenSearchTestClusterRule.java:369)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
        at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
        at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
        at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.****@21.0.6/java.lang.Thread.run(Thread.java:1583)

        Caused by:
        java.util.concurrent.TimeoutException
            at java.****/java.util.concurrent.FutureTask.get(FutureTask.java:204)
            at org.awaitility.core.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:101)
            at org.awaitility.core.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:81)
            at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:103)
            ... 45 more