[Feature/extensions] Return consumed params and content from extensions

[dbwiddis]

Companion PR: opensearch-project/opensearch-sdk-java#169

Note: merging this PR will break SDK main due to changes in transport parameters. Merge the two together.

Description

Moves the ExtensionRestResponse class from the SDK to o.o.extensions.rest.

• Updates the class to store the consumed parameters and content as fields
• Updates the RestExecuteOnExtensionResponse to copy over the content from ExtensionRestResponse and use it for transport
  • These classes are somewhat the same but cannot be combined like we did the ExtensionRestRequest because they extend different superclasses (RestResponse and TransportResponse)
    • RestResponse is an interface and we could extend it but we'd lose all the goodness of the BytesRestResponse implementation that we extend, so we'd have to copy all that. Having two very similar classes here is better.
• Simplified the logic in RestSendToExtensionAction enabled by the above two steps
• Updated tests for the new methods

Issues Resolved

Fixes SDK #165

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4102/
• CommitID: 4c45537

[dbwiddis]

Gradle check failure associated with #4661, need to catch up feature/extension branch to main

[saratvemulapalli]

Trying to learn, how do regular HTTP servers respond?
Is returning back consumed params a standard?

[dbwiddis]

I'm not sure how it is outside of OpenSearch. All I know is that OpenSearch spits out an error if you include parameters which aren't included.

[saratvemulapalli]

Which is pretty interesting, Is there a code pointer in OpenSearch. I would love to see why we put it in place.

[dbwiddis]

Well, it's part of the RestRequest class that we have to use to forward the relevant bits to the extension (and track the consuming).

Here's where it throws an exception on unconsumed params:

OpenSearch/server/src/main/java/org/opensearch/rest/BaseRestHandler.java

Lines 105 to 118 in 60d7a09

	// validate unconsumed params, but we must exclude params used to format the response
	// use a sorted set so the unconsumed parameters appear in a reliable sorted order
	final SortedSet<String> unconsumedParams = request.unconsumedParams()
	.stream()
	.filter(p -> !responseParams().contains(p))
	.collect(Collectors.toCollection(TreeSet::new));
	// validate the non-response params
	if (!unconsumedParams.isEmpty()) {
	final Set<String> candidateParams = new HashSet<>();
	candidateParams.addAll(request.consumedParams());
	candidateParams.addAll(responseParams());
	throw new IllegalArgumentException(unrecognized(request, unconsumedParams, candidateParams, "parameter"));
	}

[dbwiddis]

Here's the PR from 6 years ago with more info:
9a83ded

Looks like the intent is to provide a "you typed blah, did you mean bleh?" functionality

[saratvemulapalli]

The changes look good to me.
Just a couple of questions for my learning. Thanks @dbwiddis !

[owaiskazi19]

Started gradle check again

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4308/
• CommitID: 4c45537

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/4318/
• CommitID: 845d77d

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/4326/
• CommitID: 3fdaf6b

[codecov-commenter]

Codecov Report

Merging #4705 (3fdaf6b) into feature/extensions (eee59c5) will increase coverage by 0.16%.
The diff coverage is 77.02%.

@@                   Coverage Diff                    @@
##             feature/extensions    #4705      +/-   ##
========================================================
+ Coverage                 70.52%   70.68%   +0.16%     
- Complexity                57898    57900       +2     
========================================================
  Files                      4708     4708              
  Lines                    278528   277873     -655     
  Branches                  40661    40395     -266     
========================================================
+ Hits                     196420   196421       +1     
+ Misses                    65740    65215     -525     
+ Partials                  16368    16237     -131     

Impacted Files	Coverage Δ
...main/java/org/opensearch/gradle/PublishPlugin.java	43.43% <0.00%> (-0.45%)	⬇️
.../java/org/opensearch/gradle/pluginzip/Publish.java	11.62% <0.00%> (-0.57%)	⬇️
...earch/analysis/common/NGramTokenFilterFactory.java	75.00% <0.00%> (+10.71%)	⬆️
...gregations/bucket/geogrid/BucketPriorityQueue.java	75.00% <ø> (ø)
...arch/aggregations/bucket/geogrid/CellIdSource.java	69.23% <ø> (ø)
...ions/bucket/geogrid/GeoGridAggregationBuilder.java	75.67% <ø> (ø)
.../bucket/geogrid/GeoHashGridAggregationBuilder.java	100.00% <ø> (ø)
.../bucket/geogrid/GeoTileGridAggregationBuilder.java	100.00% <ø> (ø)
...ions/bucket/geogrid/InternalGeoHashGridBucket.java	100.00% <ø> (ø)
...ions/bucket/geogrid/InternalGeoTileGridBucket.java	100.00% <ø> (ø)
... and 646 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.