[CI] UberModuleClassLoaderTests failing

[astefan]

CI Link

https://gradle-enterprise.elastic.co/s/jfyygdphnhmu6

Repro line

./gradlew ':server:test' --tests "org.elasticsearch.plugins.UberModuleClassLoaderTests" -Dtests.seed=A39D03284C84C6A2 -Dtests.locale=en-US -Dtests.timezone=Atlantic/Reykjavik -Druntime.java=19

Does it reproduce?

No

Applicable branches

main

Failure history

https://gradle-enterprise.elastic.co/scans/tests?search.timeZoneId=Europe/Bucharest&tests.container=org.elasticsearch.plugins.UberModuleClassLoaderTests&tests.test=classMethod

Failure excerpt

  2> java.io.IOException: Could not remove the following files (in the order of attempts):
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\parent.jar: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\parent.jar
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\optional.jar: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\optional.jar
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\modular.jar: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\modular.jar
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\non-modular.jar: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001\libs-002\non-modular.jar
       C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\platform-support\479\server\build\testrun\test\temp\org.elasticsearch.plugins.UberModuleClassLoaderTests_A39D03284C84C6A2-001
        at __randomizedtesting.SeedInfo.seed([A39D03284C84C6A2]:0)
        at org.apache.lucene.util.IOUtils.rm(IOUtils.java:341)
        at org.apache.lucene.tests.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:209)
        at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
        at java.base/java.lang.Thread.run(Thread.java:1589)
  2> REPRODUCE WITH: gradlew ':server:test' --tests "org.elasticsearch.plugins.UberModuleClassLoaderTests" -Dtests.seed=A39D03284C84C6A2 -Dtests.locale=en-US -Dtests.timezone=Atlantic/Reykjavik -Druntime.java=19
  2> NOTE: test params are: codec=Lucene94, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=sq-AL, timezone=Asia/Manila
  2> NOTE: Windows Server 2012 R2 6.3 amd64/Oracle Corporation 19.0.1 (64-bit)/cpus=32,threads=1,free=331172344,total=536870912

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[luigidellaquila]

This keeps failing and seems to be Windows-specific

[thecoop]

Windows is a lot more picky about deleting files - Linux lets you delete files that have open handles on them, Windows does not.

[thecoop]

There's no open handles for those files during cleanup on a linux system, something really odd is going on here.

[n1v0lg]

More failures today: https://gradle-enterprise.elastic.co/s/lv6cbblo7magu

[thecoop]

I've been unable to reproduce this on a windows VM so far...

[valeriy42]

Another failure here https://gradle-enterprise.elastic.co/s/epvcq7zatnlys

[slobodanadamovic]

This failed again and it looks like the same issue as in #88275 - which was solved by disabling the cache for JAR URLConnections (#88586).

[williamrandolph]

Looks like I introduced this in #91217. I don't see an obvious spot to disable the jar cache but I'll take another look tomorrow.