Upgrade opensearch-protobufs dependency to 0.7.0 and update transport…

[sakrah]

…-grpc module compatibility

• Upgrade protobufs dependency from 0.6.0 to 0.7.0 in transport-grpc module
• Update all protobuf utility classes to handle API changes in 0.7.0
• Fix enum structure changes (simple enums to message with oneof fields)
• Update field name changes (setIndex -> setUnderscoreIndex, etc.)
• Handle new OperationContainer pattern for bulk operations
• Update all corresponding unit and integration tests
• Add new SHA for protobufs-0.7.0.jar and remove old 0.6.0 SHA

Description

Updates transport-grpc module to support opensearch-protobufs version 0.7.0

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 29ca685: TIMEOUT

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 2233a13: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[karenyrx]

nit: add PR number for changelog?

[andrross]

@sakrah All commits must have the signed off by tag to pass the DCO check. Can you fixup or squash that latest commit and make sure everything is signed off correctly?

Need to fix DCO

[github-actions]

❌ Gradle check result for 08572d2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[finnegancarroll]

Can you expand on this change?

Handle new OperationContainer pattern for bulk operations

I see OperationContainer added as a wrapper for the previous operations types. Is there benefit to wrapping these operations in this way or does this change just align with the spec? I notice the new version gives the OperationContainer as optional, is there a case where a user would not include this field in BulkRequestBody?

[github-actions]

❕ Gradle check result for a6a14fd: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

❌ Patch coverage is 67.90123% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.88%. Comparing base (64da5f1) to head (a6a14fd).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...est/document/bulk/BulkRequestParserProtoUtils.java	38.88%	6 Missing and 16 partials ⚠️
...esponse/document/common/VersionTypeProtoUtils.java	0.00%	1 Missing and 1 partial ⚠️
...o/request/common/FetchSourceContextProtoUtils.java	90.00%	0 Missing and 1 partial ⚠️
...onse/document/bulk/BulkItemResponseProtoUtils.java	87.50%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19003      +/-   ##
============================================
- Coverage     72.91%   72.88%   -0.03%     
+ Complexity    69391    69294      -97     
============================================
  Files          5645     5645              
  Lines        318789   318774      -15     
  Branches      46126    46120       -6     
============================================
- Hits         232442   232343      -99     
- Misses        67551    67565      +14     
- Partials      18796    18866      +70     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[karenyrx]

Can you expand on this change?

Handle new OperationContainer pattern for bulk operations

I see OperationContainer added as a wrapper for the previous operations types. Is there benefit to wrapping these operations in this way or does this change just align with the spec? I notice the new version gives the OperationContainer as optional, is there a case where a user would not include this field in BulkRequestBody?

Thanks for the feedback, this is a great callout.

The unique challenge to the Bulk API's is its NDJSON instead of JSON format in the spec. If we translate the NDJSON spec to proto directly, the current version in 0.8.0 protobufs is what we get. https://github.com/opensearch-project/opensearch-api-specification/blame/c5ca97a26dbeb089aebc4c946a3e0e22449a75c6/spec/namespaces/_core.yaml#L2292-L2297

OperationContainer is "optional" because in the HTTP BUlk API, some lines which include the document itself is not an "OperationContainer" type, e.g. { "title": "Rush", "year": 2013 } in this request:

POST _bulk
{ "delete": { "_index": "movies", "_id": "tt2229499" } }
{ "index": { "_index": "movies", "_id": "tt1979320" } }
{ "title": "Rush", "year": 2013 }
{ "create": { "_index": "movies", "_id": "tt1392214" } }
{ "title": "Prisoners", "year": 2013 }
{ "update": { "_index": "movies", "_id": "tt0816711" } }
{ "doc" : { "title": "World War Z" } }
{ "title": "Rush", "year": 2013 }

I agree it would be great to explore treating BulkRequests with an exception in the proto conversion tooling to handle NDJSON appropriate. Can this be taken up in a separate PR? Created opensearch-project/opensearch-protobufs#164 for tracking.

[sakrah]

Closing this PR in favor of consolidating all changes into PR #19007 to avoid running CI twice. All protobufs 0.8.0 upgrade work and additional unit tests will be included there.