[GRPC] Make transport-grpc extensible to other plugins

[karenyrx]

Description

This PR refactors the GRPC transport module to provide extension points for plugins. The goal is to make it possible for third-party and internal plugins to register custom GRPC services, enhancing the flexibility and extensibility of the transport layer.

Test plan

1. New and old unit tests still passing per ./gradlew :plugins:transport-grpc:test
2. Existing GRPC query types still working. Example for MatchAll:

grpcurl -import-path ~/OpenSearch -d '{"size": 10000, "request_body":{"query":{"match_all":{}}}}' -proto ~/OpenSearch/protos/services/search_service.proto -plaintext localhost:9400 org.opensearch.protobufs.services.SearchService/Search

{
  "responseBody": {
    "took": "15",
    "timedOut": false,
    "shards": {},
    "hits": {
      "total": {
        "totalHits": {
          "relation": "TOTAL_HITS_RELATION_EQ"
        }
      },
      "maxScore": {
        "floatValue": 0
      }
    }
  }
}

3. Implementations of this interface (e.g. from the KNN plugin) are working, with some custom changes to opensearch-build. *The transport-grpc plugin may have to be moved to an external plugin (in a separate diff) in order to get this working out of the box without custom opensearch-build changes.

'{
    "index": ["test_idx"],
    "size": 10,
    "request_body": {
      "query": {
        "knn": {
          "vector_field": {
            "vector": [0.1, 0.2, 0.3, 0.4, 0.5],
            "k": 5,
            "boost": 1.5,
            "method_parameters": {
              "ef_search": 100
            }
          }
        }
      }
    }
  }'

{
  "body": {
    "response_body": {
      "took": "9",
      "timed_out": false,
      "shards": {
        "successful": 1,
        "total": 1
      },
      "hits": {
        "total": {
          "total_hits": {
            "relation": "TOTAL_HITS_RELATION_EQ",
            "value": "1"
          }
        },
        "hits": [
          {
            "index": "test_idx",
            "id": "20",
            "score": {
              "float_value": 1.4999999
            },
            "source": "eyJ0aXRsZSI6ICJPcGVuU2VhcmNoIEJhc2ljcyIsICJ2ZWN0b3JfZmllbGQiOiBbMC4xLCAwLjIsIDAuMywgMC40LCAwLjVdfQ==",
            "meta_fields": {}
          }
        ],
        "max_score": {
          "float_value": 1.4999999
        }
      }
    }
  }
}

Related Issues

#18513

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

✅ Gradle check result for 968cc2f: SUCCESS

[codecov]

Codecov Report

❌ Patch coverage is 77.77778% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.76%. Comparing base (3b7603e) to head (69e2eef).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...quest/search/query/TermQueryBuilderProtoUtils.java	54.05%	16 Missing and 1 partial ⚠️
...arch/query/QueryBuilderProtoConverterRegistry.java	82.00%	8 Missing and 1 partial ⚠️
...g/opensearch/plugin/transport/grpc/GrpcPlugin.java	80.00%	2 Missing and 2 partials ⚠️
...arch/query/MatchAllQueryBuilderProtoConverter.java	83.33%	0 Missing and 1 partial ⚠️
...rch/query/MatchNoneQueryBuilderProtoConverter.java	83.33%	0 Missing and 1 partial ⚠️
...t/search/query/TermQueryBuilderProtoConverter.java	83.33%	0 Missing and 1 partial ⚠️
.../search/query/TermsQueryBuilderProtoConverter.java	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18516      +/-   ##
============================================
+ Coverage     72.67%   72.76%   +0.08%     
- Complexity    68610    68683      +73     
============================================
  Files          5577     5582       +5     
  Lines        315375   315510     +135     
  Branches      45772    45789      +17     
============================================
+ Hits         229209   229568     +359     
+ Misses        67613    67357     -256     
- Partials      18553    18585      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for f5a2243: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for f5a2243: SUCCESS

[github-actions]

✅ Gradle check result for 3d3b570: SUCCESS

[github-actions]

✅ Gradle check result for b15e640: SUCCESS

[github-actions]

✅ Gradle check result for 380ef21: SUCCESS

[github-actions]

❌ Gradle check result for 5732236: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f959e3e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 69e2eef: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.