[BUG] Flaky test SearchBackpressureIT.testSearchTaskCancellationWithHighCpu

[raghuvanshraj]

Describe the bug

java.lang.AssertionError: 
Expected: a string containing "cpu usage exceeded"
     but: was null

Even with the given seed, the test passes, both on the PR branch as well as on main, but fails in the gradle ci run.

./gradlew ':server:internalClusterTest' --tests "org.opensearch.search.backpressure.SearchBackpressureIT.testSearchTaskCancellationWithHighCpu" -Dtests.seed=C07B948307D5098C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=zh-TW -Dtests.timezone=America/Thunder_Bay
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.1.1
  OS Info               : Linux 5.19.0-1025-aws (amd64)
  JDK Version           : 11 (Ubuntu JDK)
  JAVA_HOME             : /usr/lib/jvm/java-11-openjdk-amd64
  Random Testing Seed   : C07B948307D5098C
  In FIPS 140 mode      : false
=======================================

> Task :server:internalClusterTest
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.BootstrapForTesting (file:/home/ubuntu/workplace/OpenSearchMain/test/framework/build/distributions/framework-3.0.0-SNAPSHOT.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.BootstrapForTesting
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.gradle.api.internal.tasks.testing.worker.TestWorker (file:/home/ubuntu/.gradle/wrapper/dists/gradle-8.1.1-all/bs1rrjki8hh9bujwbsqnxtuzr/gradle-8.1.1/lib/plugins/gradle-testing-base-8.1.1.jar)
WARNING: Please consider reporting this to the maintainers of org.gradle.api.internal.tasks.testing.worker.TestWorker
WARNING: System::setSecurityManager will be removed in a future release

BUILD SUCCESSFUL in 11s
46 actionable tasks: 1 executed, 45 up-to-date

To Reproduce
Build Failure: https://build.ci.opensearch.org/job/gradle-check/16209/testReport/org.opensearch.search.backpressure/SearchBackpressureIT/testSearchTaskCancellationWithHighCpu/

Expected behavior
Test should always pass

[stephen-crawford]

Resolved with #7978