[BUG] Security plugin snapshot install fails due to JAR hell

[Xtansia]

Cross-post of opensearch-project/security#2799 for visibility and due to my noticing the incorrect LICENSE.

What is the bug?
Due to the addition of a dependency on org.jboss.spec.javax.annotation:jboss-annotations-api_1.2_spec in #7604 , attempting to install a snapshot build of opensearch-security on a build of OpenSearch 2.x or main now results in a JAR hell error:

2023-05-28T22:42:45.3242078Z Exception in thread "main" java.lang.IllegalStateException: failed to load plugin opensearch-security due to jar hell
2023-05-28T22:42:45.3242712Z 	at org.opensearch.plugins.PluginsService.checkBundleJarHell(PluginsService.java:681)
2023-05-28T22:42:45.3243361Z 	at org.opensearch.plugins.InstallPluginCommand.jarHellCheck(InstallPluginCommand.java:862)
2023-05-28T22:42:45.3244201Z 	at org.opensearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:830)
2023-05-28T22:42:45.3244876Z 	at org.opensearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:875)
2023-05-28T22:42:45.3245519Z 	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:276)
2023-05-28T22:42:45.3246132Z 	at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:250)
2023-05-28T22:42:45.3246757Z 	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
2023-05-28T22:42:45.3247344Z 	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
2023-05-28T22:42:45.3247864Z 	at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
2023-05-28T22:42:45.3248386Z 	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
2023-05-28T22:42:45.3248883Z 	at org.opensearch.cli.Command.main(Command.java:101)
2023-05-28T22:42:45.3249340Z 	at org.opensearch.plugins.PluginCli.main(PluginCli.java:66)
2023-05-28T22:42:45.3249804Z Caused by: java.lang.IllegalStateException: jar hell!
2023-05-28T22:42:45.3250205Z class: javax.annotation.Generated
2023-05-28T22:42:45.3251037Z jar1: /tmp/OpenSearchManaged/opensearch-3.0.0-SNAPSHOT/ephemeral-cluster-4ac246/home/plugins/.installing-10479747099297960195/jakarta.annotation-api-1.3.5.jar
2023-05-28T22:42:45.3252137Z jar2: /tmp/OpenSearchManaged/opensearch-3.0.0-SNAPSHOT/ephemeral-cluster-4ac246/home/lib/jboss-annotations-api_1.2_spec-1.0.2.Final.jar

Further it appears the jboss-annotations-api_1.2_spec-LICENSE.txt and jboss-annotations-api_1.2_spec-NOTICE.txt that were added are copies of OpenSearch's own LICENSE and NOTICE, not jboss-annotations-api's LICENSE which is GPLv2 not Apache 2.

As a few projects in the org are already using the Jakarta implementation of the javax annotations: https://github.com/search?q=org%3Aopensearch-project+%22jakarta.annotation-api%22&type=code potentially it makes sense to use that implementation here and drop the dependency inside the security plugin?

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Clone opensearch-project/OpenSearch at main branch
2. Run ./gradlew :distribution:archives:linux-tar:assemble
3. Extract archive in distribution/archives/linux-tar/build/distributions/
4. Run ${opensearch_home}/bin/opensearch-plugin install --batch https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-security/3.0.0.0-SNAPSHOT/opensearch-security-3.0.0.0-20230525.223929-104.zip

What is the expected behavior?
Plugin to successfully install

What is your host/environment?

• OS: Ubuntu
• Version: 22.04

[cwperks]

@Xtansia thank you for the PR on the security repo: opensearch-project/security#2801

The jar hell issue should now be resolved for installation of the security plugin.

[dblock]

Close this?

[cwperks]

The security plugin is still facing issues with stale artifacts: opensearch-project/security-dashboards-plugin#1449

The last minimum distribution of core was run with 2.x instead of 2.8, but there is no dev work left on this issue. It can be closed.

[cwperks]

@Xtansia Please re-open or file a new issue regarding the licenses. The main description of this issue has been resolved in the security repo, but the issue with the license for the new dependency still remains.