Migrate from BC to BCFIPS libraries

[cwperks]

Description

bouncycastle was removed from the gradle version catalog in https://github.com/opensearch-project/OpenSearch/pull/17507/files#diff-697f70cdd88ba88fe77eebda60c7e143f6ad1286bca75017421e93ad84fb87df.

This PR migrates from BC to BCFIPS libraries for this repo

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.30%. Comparing base (e49812b) to head (5cc0cb4).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1087   +/-   ##
=========================================
  Coverage     76.30%   76.30%           
  Complexity     1076     1076           
=========================================
  Files           101      101           
  Lines          5276     5276           
  Branches        504      504           
=========================================
  Hits           4026     4026           
  Misses         1001     1001           
  Partials        249      249           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dbwiddis]

Hey @cwperks we previously had 1.8.0 which caused jar hell. We downgraded to the version catalog to avoid that. (See #1072)

But I expect our dependency manager will suggest an upgrade again.

Can you be more clear on the reason for this downgrade if it is not for consistency with other dependencies? Can we not make this 1.8.0 again by reverting #1078?

[cwperks]

@dbwiddis I can try to update the version, but the purpose of this PR is to put the version inline instead of referencing the version catalog because versions.bouncycastle was removed from the version catalog.

@beanuwave has been working on a large effort around FIPS-140-3 compliance for OpenSearch which includes swapping BC non-FIPS jars with FIPS jars so non-FIPS references were removed from the core.

Currently the jar swap is only for the core and has not been performed in the plugins. Plugins that referenced the version of bouncycastle from core's version catalog now need to put the version inline.

[cwperks]

@dbwiddis I think the update will work, but core also changed the qualifier to beta1 and we now need to wait for all of FF's dependent plugins to update accordingly.

Switching back to 1.78 in the interim

[cwperks]

@dbwiddis I think the update will work, but core also changed the qualifier to beta1 and we now need to wait for all of FF's dependent plugins to update accordingly.

Confirmed it would work.

When checking out core you need to specify the qualifier when building. i.e. ./gradlew publishToMavenLocal -Dbuild.version_qualifier=alpha1

[dbwiddis]

The bigger question is why we are not aligning on a common version. I’d like to prevent future churn.

[cwperks]

One option could be to do the swap in this repo from non-FIPS -> FIPS deps for BC dependencies.

CC @beanuwave

[dbwiddis]

One option could be to do the swap in this repo from non-FIPS -> FIPS deps for BC dependencies.

CC @beanuwave

Sure, happy to look into this. Still trying to understand what happened here.

• we'd been bumping versions normally and had gone from 1.78 to 1.80 over the last few months
• our pipeline suddenly broke with 1.80 due to jar hell with 1.78 ([BUG] Jar hell from org.bouncycastle:bcprov-jdk18on #1072)
• we updated to the version catalog to hopefully eliminate that
• if we merge this PR we'll be back where we were a few months ago and we'll get a version bump to 1.80
• rather than merging this PR I'd prefer to just revert Get bouncycastle dependency version from catalog #1078 if it's no longer needed
• but I don't understand yet why it was a problem to begin with.
  • Was whatever was causing the jar hell removed and 1.80 will work fine? (In which case let's revert Get bouncycastle dependency version from catalog #1078)
  • Or is there some problem with this dependency where we should switch to something else?

[cwperks]

Was whatever was causing the jar hell removed and 1.80 will work fine?

It used to collide with the dependencies in opensearch-test-framework here. With this PR those deps changed to BCFIPS.

> Task :dependencyInsight
Dependency resolution failed because of conflict on the following module:
   - org.bouncycastle:bcprov-jdk18on between versions 1.80 and 1.78

org.bouncycastle:bcprov-jdk18on:1.80
  Variant compile:
    | Attribute Name                 | Provided | Requested    |
    |--------------------------------|----------|--------------|
    | org.gradle.status              | release  |              |
    | org.gradle.category            | library  | library      |
    | org.gradle.libraryelements     | jar      | classes      |
    | org.gradle.usage               | java-api | java-api     |
    | org.gradle.dependency.bundling |          | external     |
    | org.gradle.jvm.environment     |          | standard-jvm |
    | org.gradle.jvm.version         |          | 21           |
   Selection reasons:
      - By conflict resolution: between versions 1.80 and 1.78

org.bouncycastle:bcprov-jdk18on:1.80
\--- testCompileClasspath

org.bouncycastle:bcprov-jdk18on:1.78 -> 1.80
\--- org.opensearch.test:framework:3.0.0-alpha1-SNAPSHOT:20250319.180915-157
     \--- testCompileClasspath

[dbwiddis]

ncies in opensearch-test-framework here. With this PR those deps changed to BCFIPS.

So I think the best action for us is to do the same

[dbwiddis]

@cwperks Thanks for updating the PR! We'll merge this and switch to beta1 once all our upstream repos do...

One curious question remains:

It used to collide with the dependencies in opensearch-test-framework here.

Yeah, but we had bumped to 1.8.0 on Jan 15, and from 1.78 to 1.79 back on Nov 1 last year. But the Jar hell only started relatively recently. Maybe it was whatever security fix we needed that caused the 2.19.1 release since it happened around that time...