Skip to content

[GRPC] Add gRPC support for Min, Max, and Terms aggregations #21205

Merged
karenyrx merged 13 commits into
opensearch-project:mainfrom
lucy66hw:aggregationGrpc
Apr 19, 2026
Merged

[GRPC] Add gRPC support for Min, Max, and Terms aggregations #21205
karenyrx merged 13 commits into
opensearch-project:mainfrom
lucy66hw:aggregationGrpc

Conversation

@lucy66hw
Copy link
Copy Markdown
Contributor

@lucy66hw lucy66hw commented Apr 10, 2026

Description

Based on #20676, #20970

  • Min/Max metric aggregations — converts InternalMin/InternalMax to SingleMetricAggregateBase proto
  • Terms bucket aggregations — converts all five terms variants to their respective proto types:
    • StringTerms → sterms
    • LongTerms → lterms
    • DoubleTerms → dterms
    • UnsignedLongTerms → ulterms
    • UnmappedTerms → umterms

TEST

Index Mapping

{
  "properties": {
    "status":      { "type": "keyword"       },
    "category_id": { "type": "long"          },
    "price":       { "type": "double"        },
    "big_id":      { "type": "unsigned_long" }
  }
}

Document count: 5


Test 1 — Keyword field (status)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "status_terms": {
        "terms_aggregation": { "terms": { "field": "status" } }
      }
    }
  }
}

REST Response (aggregations only)

{
  "status_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      { "key": "active",   "doc_count": 3 },
      { "key": "inactive", "doc_count": 2 }
    ]
  }
}

gRPC Response (aggregations only)

{
  "status_terms": {
    "sterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0",
      "buckets": [
        { "key": "active",   "docCount": "3" },
        { "key": "inactive", "docCount": "2" }
      ]
    }
  }
}

Test 2 — Long field (category_id)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "category_terms": {
        "terms_aggregation": { "terms": { "field": "category_id" } }
      }
    }
  }
}

REST Response (aggregations only)

{
  "category_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      { "key": 1, "doc_count": 2 },
      { "key": 2, "doc_count": 2 },
      { "key": 3, "doc_count": 1 }
    ]
  }
}

gRPC Response (aggregations only)

{
  "category_terms": {
    "lterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0",
      "buckets": [
        { "key": { "signed": "1" }, "docCount": "2" },
        { "key": { "signed": "2" }, "docCount": "2" },
        { "key": { "signed": "3" }, "docCount": "1" }
      ]
    }
  }
}

Test 3 — Double field (price)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "price_terms": {
        "terms_aggregation": { "terms": { "field": "price" } }
      }
    }
  }
}

REST Response (aggregations only)

{
  "price_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      { "key": 5.2,  "doc_count": 1 },
      { "key": 10.5, "doc_count": 1 },
      { "key": 15.0, "doc_count": 1 },
      { "key": 25.0, "doc_count": 1 },
      { "key": 30.0, "doc_count": 1 }
    ]
  }
}

gRPC Response (aggregations only)

{
  "price_terms": {
    "dterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0",
      "buckets": [
        { "key": 5.2,  "docCount": "1" },
        { "key": 10.5, "docCount": "1" },
        { "key": 15,   "docCount": "1" },
        { "key": 25,   "docCount": "1" },
        { "key": 30,   "docCount": "1" }
      ]
    }
  }
}

Test 4 — Unsigned Long field (big_id)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "bigid_terms": {
        "terms_aggregation": { "terms": { "field": "big_id" } }
      }
    }
  }
}

REST Response (aggregations only)

{
  "bigid_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      { "key": 9223372036854775808, "doc_count": 2 },
      { "key": 9223372036854775809, "doc_count": 2 },
      { "key": 9223372036854775810, "doc_count": 1 }
    ]
  }
}

gRPC Response (aggregations only)

{
  "bigid_terms": {
    "ulterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0",
      "buckets": [
        { "key": "9223372036854775808", "docCount": "2" },
        { "key": "9223372036854775809", "docCount": "2" },
        { "key": "9223372036854775810", "docCount": "1" }
      ]
    }
  }
}

Test 5 — Unmapped field (does_not_exist)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "unmapped_terms": {
        "terms_aggregation": { "terms": { "field": "does_not_exist" } }
      }
    }
  }
}

REST Response (aggregations only)

{
  "unmapped_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": []
  }
}

gRPC Response (aggregations only)

{
  "unmapped_terms": {
    "umterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0"
    }
  }
}

Test 6 — Keyword field with sub-aggregation (status + max_price)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "status_terms": {
        "terms_aggregation": {
          "terms": { "field": "status" },
          "aggregations": {
            "max_price": { "max": { "field": "price" } }
          }
        }
      }
    }
  }
}

REST Response (aggregations only)

{
  "status_terms": {
    "doc_count_error_upper_bound": 0,
    "sum_other_doc_count": 0,
    "buckets": [
      { "key": "active",   "doc_count": 3, "max_price": { "value": 25.0 } },
      { "key": "inactive", "doc_count": 2, "max_price": { "value": 30.0 } }
    ]
  }
}

gRPC Response (aggregations only)

{
  "status_terms": {
    "sterms": {
      "doc_count_error_upper_bound": "0",
      "sum_other_doc_count": "0",
      "buckets": [
        {
          "key": "active",
          "docCount": "3",
          "aggregate": {
            "max_price": { "max": { "value": { "double": 25 } } }
          }
        },
        {
          "key": "inactive",
          "docCount": "2",
          "aggregate": {
            "max_price": { "max": { "value": { "double": 30 } } }
          }
        }
      ]
    }
  }
}


Test 7 — Min aggregation (price, category_id, big_id)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "min_price": { "min": { "field": "price" } },
      "min_catid": { "min": { "field": "category_id" } },
      "min_bigid": { "min": { "field": "big_id" } }
    }
  }
}

REST Response (aggregations only)

{
  "min_price": { "value": 5.2  },
  "min_catid": { "value": 1.0  },
  "min_bigid": { "value": 9.223372036854776e+18, "value_as_string": "9.223372036854776E18" }
}

gRPC Response (aggregations only)

{
  "min_price": { "min": { "value": { "double": 5.2  } } },
  "min_catid": { "min": { "value": { "double": 1    } } },
  "min_bigid": { "min": { "value": { "double": 9.223372036854776e+18 }, "valueAsString": "9.223372036854776E18" } }
}

Test 8 — Max aggregation (price, category_id, big_id)

gRPC Request

{
  "index": ["test-terms-agg"],
  "search_request_body": {
    "size": 0,
    "aggregations": {
      "max_price": { "max": { "field": "price" } },
      "max_catid": { "max": { "field": "category_id" } },
      "max_bigid": { "max": { "field": "big_id" } }
    }
  }
}

REST Response (aggregations only)

{
  "max_price": { "value": 30.0 },
  "max_catid": { "value": 3.0  },
  "max_bigid": { "value": 9.223372036854776e+18, "value_as_string": "9.223372036854776E18" }
}

gRPC Response (aggregations only)

{
  "max_price": { "max": { "value": { "double": 30   } } },
  "max_catid": { "max": { "value": { "double": 3    } } },
  "max_bigid": { "max": { "value": { "double": 9.223372036854776e+18 }, "valueAsString": "9.223372036854776E18" } }
}

Test 9 — Min/Max on unmapped field (does_not_exist)

REST Response (aggregations only)

{
  "min_missing": { "value": null },
  "max_missing": { "value": null }
}

gRPC Response (aggregations only)

{
  "min_missing": { "min": { "value": { "nullValue": "NULL_VALUE_NULL" } } },
  "max_missing": { "max": { "value": { "nullValue": "NULL_VALUE_NULL" } } }
}

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 10, 2026

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 6c7e5b4.

PathLineSeverityDescription
gradle/libs.versions.toml30highDependency version bump: opensearchprotobufs upgraded from 1.3.0 to 1.4.0. Per mandatory flagging rule, all dependency version changes must be flagged regardless of apparent legitimacy. Maintainers should verify the new artifact at version 1.4.0 against the updated SHA1 (c09d98cf3c06fe49f77c5090309ba1769ba9b3c1) from a trusted source before merging.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 1 | Medium: 0 | Low: 0


Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.


⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

@lucy66hw lucy66hw force-pushed the aggregationGrpc branch 2 times, most recently from 3f875fd to b393fa1 Compare April 15, 2026 04:15
Copy link
Copy Markdown
Contributor

@karenyrx karenyrx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall

Comment thread gradle/libs.versions.toml Outdated
yiyuabc and others added 4 commits April 16, 2026 21:15
This commit adds complete gRPC support for Min and Max metric aggregations,
including request parsing, response conversion, and integration with the search pipeline.

**Request Conversion (OpenSearch ← gRPC):**

- AggregationContainerProtoUtils: Central dispatcher routing aggregation types
  to specific converters, validates aggregation names
- ValuesSourceAggregationProtoUtils: Shared utilities for parsing common
  ValuesSource fields (field, missing, value_type, format, script)

- MinAggregationProtoUtils: Converts proto MinAggregation → MinAggregationBuilder
- MaxAggregationProtoUtils: Converts proto MaxAggregation → MaxAggregationBuilder

- Updated SearchSourceBuilderProtoUtils to parse aggregations map from
  proto SearchRequestBody and add to SearchSourceBuilder

**Response Conversion (gRPC ← OpenSearch):**

- SearchResponseSectionsProtoUtils: **CRITICAL FIX** - Added aggregation response
  conversion to search response pipeline, enabling aggregation results to be
  returned via gRPC. This connects the Min/Max aggregate converters to the
  search response builder.

- AggregateProtoUtils: Central dispatcher for converting InternalAggregation
  to proto Aggregate, with metadata and sub-aggregation helpers

- MinAggregateProtoUtils: Converts InternalMin → proto MinAggregate
- MaxAggregateProtoUtils: Converts InternalMax → proto MaxAggregate
- Handles special values (infinity, NaN), formatting, and metadata

**OpenSearch Core Changes:**

- InternalNumericMetricsAggregation: Added getFormat() getter to expose
  format information for gRPC converters

**Request Conversion Tests:**

- AggregationContainerProtoUtilsTests: 11 tests
- MinAggregationProtoUtilsTests: 108 tests
- MaxAggregationProtoUtilsTests: 108 tests
- ValuesSourceAggregationProtoUtilsTests: 40+ tests
- SearchSourceBuilderProtoUtilsTests: 2 new aggregation tests

**Response Conversion Tests:**

- AggregateProtoUtilsTests: 10 tests
- MinAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- MaxAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- SearchResponseSectionsProtoUtilsTests: 2 new aggregation tests (null/present)
- InternalMinTests: 32 tests
- InternalMaxTests: 32 tests

**Total: ~680 test cases**

**Design Principles:**

- Mirrors REST API patterns for consistency
- Maintains behavioral parity with REST layer
- Comprehensive error handling and validation
- Special value support (infinity, NaN) matching REST behavior
- Metadata handling consistent across all aggregations

**Implementation Summary:**

- **New Implementation Files**: 11 (converters + infrastructure)
- **New Test Files**: 8 (comprehensive test coverage)
- **Modified Files**: 4 (SearchSourceBuilder + SearchResponseSections + server changes)
- **Package Documentation**: 5 package-info.java files
- **Total Lines**: ~1,850 (implementation + tests)

**Testing & Validation:**

- Comprehensive integration testing via REST and gRPC
- Verified behavioral parity (see min_max_aggregation_comparison_report.md)
- All core functionality confirmed working correctly
- Values match exactly between REST and gRPC responses

**API Differences (By Design):**

- REST uses simple JSON values; gRPC uses typed protobuf structures
- Parameter formats differ (e.g., missing parameter requires FieldValue in gRPC)
- Field naming follows protobuf conventions (camelCase vs snake_case)

Co-Authored-By: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Signed-off-by: Yiyu Pan <yypan14@gmail.com>
Signed-off-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: xil <fridalu66@gmail.com>
Signed-off-by: xil <fridalu66@gmail.com>
Signed-off-by: xil <fridalu66@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 9338dd0

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 9338dd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 9338dd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 9338dd0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

✅ Gradle check result for 9338dd0: SUCCESS

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.21%. Comparing base (c93eb90) to head (249661e).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #21205      +/-   ##
============================================
- Coverage     73.22%   73.21%   -0.01%     
- Complexity    73325    73455     +130     
============================================
  Files          5910     5911       +1     
  Lines        334422   334936     +514     
  Branches      48207    48257      +50     
============================================
+ Hits         244880   245236     +356     
- Misses        69964    70107     +143     
- Partials      19578    19593      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: xil <fridalu66@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 5af5162

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 5af5162: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit b15b75c

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for b15b75c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: xil <fridalu66@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 249661e

@github-actions
Copy link
Copy Markdown
Contributor

✅ Gradle check result for 249661e: SUCCESS

@karenyrx karenyrx merged commit 077fde2 into opensearch-project:main Apr 19, 2026
24 checks passed
abhishek00159 pushed a commit to abhishek00159/OpenSearch that referenced this pull request Apr 23, 2026
…rch-project#21205)

* Add gRPC support for Min and Max metric aggregations

This commit adds complete gRPC support for Min and Max metric aggregations,
including request parsing, response conversion, and integration with the search pipeline.

**Request Conversion (OpenSearch ← gRPC):**

- AggregationContainerProtoUtils: Central dispatcher routing aggregation types
  to specific converters, validates aggregation names
- ValuesSourceAggregationProtoUtils: Shared utilities for parsing common
  ValuesSource fields (field, missing, value_type, format, script)

- MinAggregationProtoUtils: Converts proto MinAggregation → MinAggregationBuilder
- MaxAggregationProtoUtils: Converts proto MaxAggregation → MaxAggregationBuilder

- Updated SearchSourceBuilderProtoUtils to parse aggregations map from
  proto SearchRequestBody and add to SearchSourceBuilder

**Response Conversion (gRPC ← OpenSearch):**

- SearchResponseSectionsProtoUtils: **CRITICAL FIX** - Added aggregation response
  conversion to search response pipeline, enabling aggregation results to be
  returned via gRPC. This connects the Min/Max aggregate converters to the
  search response builder.

- AggregateProtoUtils: Central dispatcher for converting InternalAggregation
  to proto Aggregate, with metadata and sub-aggregation helpers

- MinAggregateProtoUtils: Converts InternalMin → proto MinAggregate
- MaxAggregateProtoUtils: Converts InternalMax → proto MaxAggregate
- Handles special values (infinity, NaN), formatting, and metadata

**OpenSearch Core Changes:**

- InternalNumericMetricsAggregation: Added getFormat() getter to expose
  format information for gRPC converters

**Request Conversion Tests:**

- AggregationContainerProtoUtilsTests: 11 tests
- MinAggregationProtoUtilsTests: 108 tests
- MaxAggregationProtoUtilsTests: 108 tests
- ValuesSourceAggregationProtoUtilsTests: 40+ tests
- SearchSourceBuilderProtoUtilsTests: 2 new aggregation tests

**Response Conversion Tests:**

- AggregateProtoUtilsTests: 10 tests
- MinAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- MaxAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- SearchResponseSectionsProtoUtilsTests: 2 new aggregation tests (null/present)
- InternalMinTests: 32 tests
- InternalMaxTests: 32 tests

**Total: ~680 test cases**

**Design Principles:**

- Mirrors REST API patterns for consistency
- Maintains behavioral parity with REST layer
- Comprehensive error handling and validation
- Special value support (infinity, NaN) matching REST behavior
- Metadata handling consistent across all aggregations

**Implementation Summary:**

- **New Implementation Files**: 11 (converters + infrastructure)
- **New Test Files**: 8 (comprehensive test coverage)
- **Modified Files**: 4 (SearchSourceBuilder + SearchResponseSections + server changes)
- **Package Documentation**: 5 package-info.java files
- **Total Lines**: ~1,850 (implementation + tests)

**Testing & Validation:**

- Comprehensive integration testing via REST and gRPC
- Verified behavioral parity (see min_max_aggregation_comparison_report.md)
- All core functionality confirmed working correctly
- Values match exactly between REST and gRPC responses

**API Differences (By Design):**

- REST uses simple JSON values; gRPC uses typed protobuf structures
- Parameter formats differ (e.g., missing parameter requires FieldValue in gRPC)
- Field naming follows protobuf conventions (camelCase vs snake_case)

Co-Authored-By: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Signed-off-by: Yiyu Pan <yypan14@gmail.com>

* Add TermsAggregation implementation for grpc

Signed-off-by: Patrick Zhai <pzhai@uber.com>

* update aggregation response based on new protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* use release protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* add missing javadoc

Signed-off-by: xil <fridalu66@gmail.com>

* Add java doc to spi

Signed-off-by: xil <fridalu66@gmail.com>

* Address PR Code Suggestions

Signed-off-by: xil <fridalu66@gmail.com>

* Address suggestion

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* increase test coverage

Signed-off-by: xil <fridalu66@gmail.com>

* spotlessApply

Signed-off-by: xil <fridalu66@gmail.com>

---------

Signed-off-by: Yiyu Pan <yypan14@gmail.com>
Signed-off-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: xil <fridalu66@gmail.com>
Co-authored-by: Yiyu Pan <yypan14@gmail.com>
Co-authored-by: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Co-authored-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: Abhishek Som <abhissom@amazon.com>
divyaruhil pushed a commit to divyaruhil/OpenSearch that referenced this pull request Apr 23, 2026
…rch-project#21205)

* Add gRPC support for Min and Max metric aggregations

This commit adds complete gRPC support for Min and Max metric aggregations,
including request parsing, response conversion, and integration with the search pipeline.

**Request Conversion (OpenSearch ← gRPC):**

- AggregationContainerProtoUtils: Central dispatcher routing aggregation types
  to specific converters, validates aggregation names
- ValuesSourceAggregationProtoUtils: Shared utilities for parsing common
  ValuesSource fields (field, missing, value_type, format, script)

- MinAggregationProtoUtils: Converts proto MinAggregation → MinAggregationBuilder
- MaxAggregationProtoUtils: Converts proto MaxAggregation → MaxAggregationBuilder

- Updated SearchSourceBuilderProtoUtils to parse aggregations map from
  proto SearchRequestBody and add to SearchSourceBuilder

**Response Conversion (gRPC ← OpenSearch):**

- SearchResponseSectionsProtoUtils: **CRITICAL FIX** - Added aggregation response
  conversion to search response pipeline, enabling aggregation results to be
  returned via gRPC. This connects the Min/Max aggregate converters to the
  search response builder.

- AggregateProtoUtils: Central dispatcher for converting InternalAggregation
  to proto Aggregate, with metadata and sub-aggregation helpers

- MinAggregateProtoUtils: Converts InternalMin → proto MinAggregate
- MaxAggregateProtoUtils: Converts InternalMax → proto MaxAggregate
- Handles special values (infinity, NaN), formatting, and metadata

**OpenSearch Core Changes:**

- InternalNumericMetricsAggregation: Added getFormat() getter to expose
  format information for gRPC converters

**Request Conversion Tests:**

- AggregationContainerProtoUtilsTests: 11 tests
- MinAggregationProtoUtilsTests: 108 tests
- MaxAggregationProtoUtilsTests: 108 tests
- ValuesSourceAggregationProtoUtilsTests: 40+ tests
- SearchSourceBuilderProtoUtilsTests: 2 new aggregation tests

**Response Conversion Tests:**

- AggregateProtoUtilsTests: 10 tests
- MinAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- MaxAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- SearchResponseSectionsProtoUtilsTests: 2 new aggregation tests (null/present)
- InternalMinTests: 32 tests
- InternalMaxTests: 32 tests

**Total: ~680 test cases**

**Design Principles:**

- Mirrors REST API patterns for consistency
- Maintains behavioral parity with REST layer
- Comprehensive error handling and validation
- Special value support (infinity, NaN) matching REST behavior
- Metadata handling consistent across all aggregations

**Implementation Summary:**

- **New Implementation Files**: 11 (converters + infrastructure)
- **New Test Files**: 8 (comprehensive test coverage)
- **Modified Files**: 4 (SearchSourceBuilder + SearchResponseSections + server changes)
- **Package Documentation**: 5 package-info.java files
- **Total Lines**: ~1,850 (implementation + tests)

**Testing & Validation:**

- Comprehensive integration testing via REST and gRPC
- Verified behavioral parity (see min_max_aggregation_comparison_report.md)
- All core functionality confirmed working correctly
- Values match exactly between REST and gRPC responses

**API Differences (By Design):**

- REST uses simple JSON values; gRPC uses typed protobuf structures
- Parameter formats differ (e.g., missing parameter requires FieldValue in gRPC)
- Field naming follows protobuf conventions (camelCase vs snake_case)

Co-Authored-By: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Signed-off-by: Yiyu Pan <yypan14@gmail.com>

* Add TermsAggregation implementation for grpc

Signed-off-by: Patrick Zhai <pzhai@uber.com>

* update aggregation response based on new protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* use release protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* add missing javadoc

Signed-off-by: xil <fridalu66@gmail.com>

* Add java doc to spi

Signed-off-by: xil <fridalu66@gmail.com>

* Address PR Code Suggestions

Signed-off-by: xil <fridalu66@gmail.com>

* Address suggestion

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* increase test coverage

Signed-off-by: xil <fridalu66@gmail.com>

* spotlessApply

Signed-off-by: xil <fridalu66@gmail.com>

---------

Signed-off-by: Yiyu Pan <yypan14@gmail.com>
Signed-off-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: xil <fridalu66@gmail.com>
Co-authored-by: Yiyu Pan <yypan14@gmail.com>
Co-authored-by: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Co-authored-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: Divya <divyruhil999@gmail.com>
krishna-ggk pushed a commit to krishna-ggk/OpenSearch that referenced this pull request Apr 28, 2026
…rch-project#21205)

* Add gRPC support for Min and Max metric aggregations

This commit adds complete gRPC support for Min and Max metric aggregations,
including request parsing, response conversion, and integration with the search pipeline.

**Request Conversion (OpenSearch ← gRPC):**

- AggregationContainerProtoUtils: Central dispatcher routing aggregation types
  to specific converters, validates aggregation names
- ValuesSourceAggregationProtoUtils: Shared utilities for parsing common
  ValuesSource fields (field, missing, value_type, format, script)

- MinAggregationProtoUtils: Converts proto MinAggregation → MinAggregationBuilder
- MaxAggregationProtoUtils: Converts proto MaxAggregation → MaxAggregationBuilder

- Updated SearchSourceBuilderProtoUtils to parse aggregations map from
  proto SearchRequestBody and add to SearchSourceBuilder

**Response Conversion (gRPC ← OpenSearch):**

- SearchResponseSectionsProtoUtils: **CRITICAL FIX** - Added aggregation response
  conversion to search response pipeline, enabling aggregation results to be
  returned via gRPC. This connects the Min/Max aggregate converters to the
  search response builder.

- AggregateProtoUtils: Central dispatcher for converting InternalAggregation
  to proto Aggregate, with metadata and sub-aggregation helpers

- MinAggregateProtoUtils: Converts InternalMin → proto MinAggregate
- MaxAggregateProtoUtils: Converts InternalMax → proto MaxAggregate
- Handles special values (infinity, NaN), formatting, and metadata

**OpenSearch Core Changes:**

- InternalNumericMetricsAggregation: Added getFormat() getter to expose
  format information for gRPC converters

**Request Conversion Tests:**

- AggregationContainerProtoUtilsTests: 11 tests
- MinAggregationProtoUtilsTests: 108 tests
- MaxAggregationProtoUtilsTests: 108 tests
- ValuesSourceAggregationProtoUtilsTests: 40+ tests
- SearchSourceBuilderProtoUtilsTests: 2 new aggregation tests

**Response Conversion Tests:**

- AggregateProtoUtilsTests: 10 tests
- MinAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- MaxAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- SearchResponseSectionsProtoUtilsTests: 2 new aggregation tests (null/present)
- InternalMinTests: 32 tests
- InternalMaxTests: 32 tests

**Total: ~680 test cases**

**Design Principles:**

- Mirrors REST API patterns for consistency
- Maintains behavioral parity with REST layer
- Comprehensive error handling and validation
- Special value support (infinity, NaN) matching REST behavior
- Metadata handling consistent across all aggregations

**Implementation Summary:**

- **New Implementation Files**: 11 (converters + infrastructure)
- **New Test Files**: 8 (comprehensive test coverage)
- **Modified Files**: 4 (SearchSourceBuilder + SearchResponseSections + server changes)
- **Package Documentation**: 5 package-info.java files
- **Total Lines**: ~1,850 (implementation + tests)

**Testing & Validation:**

- Comprehensive integration testing via REST and gRPC
- Verified behavioral parity (see min_max_aggregation_comparison_report.md)
- All core functionality confirmed working correctly
- Values match exactly between REST and gRPC responses

**API Differences (By Design):**

- REST uses simple JSON values; gRPC uses typed protobuf structures
- Parameter formats differ (e.g., missing parameter requires FieldValue in gRPC)
- Field naming follows protobuf conventions (camelCase vs snake_case)

Co-Authored-By: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Signed-off-by: Yiyu Pan <yypan14@gmail.com>

* Add TermsAggregation implementation for grpc

Signed-off-by: Patrick Zhai <pzhai@uber.com>

* update aggregation response based on new protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* use release protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* add missing javadoc

Signed-off-by: xil <fridalu66@gmail.com>

* Add java doc to spi

Signed-off-by: xil <fridalu66@gmail.com>

* Address PR Code Suggestions

Signed-off-by: xil <fridalu66@gmail.com>

* Address suggestion

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* increase test coverage

Signed-off-by: xil <fridalu66@gmail.com>

* spotlessApply

Signed-off-by: xil <fridalu66@gmail.com>

---------

Signed-off-by: Yiyu Pan <yypan14@gmail.com>
Signed-off-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: xil <fridalu66@gmail.com>
Co-authored-by: Yiyu Pan <yypan14@gmail.com>
Co-authored-by: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Co-authored-by: Patrick Zhai <pzhai@uber.com>
imRishN pushed a commit to imRishN/OpenSearch that referenced this pull request May 8, 2026
…rch-project#21205)

* Add gRPC support for Min and Max metric aggregations

This commit adds complete gRPC support for Min and Max metric aggregations,
including request parsing, response conversion, and integration with the search pipeline.

**Request Conversion (OpenSearch ← gRPC):**

- AggregationContainerProtoUtils: Central dispatcher routing aggregation types
  to specific converters, validates aggregation names
- ValuesSourceAggregationProtoUtils: Shared utilities for parsing common
  ValuesSource fields (field, missing, value_type, format, script)

- MinAggregationProtoUtils: Converts proto MinAggregation → MinAggregationBuilder
- MaxAggregationProtoUtils: Converts proto MaxAggregation → MaxAggregationBuilder

- Updated SearchSourceBuilderProtoUtils to parse aggregations map from
  proto SearchRequestBody and add to SearchSourceBuilder

**Response Conversion (gRPC ← OpenSearch):**

- SearchResponseSectionsProtoUtils: **CRITICAL FIX** - Added aggregation response
  conversion to search response pipeline, enabling aggregation results to be
  returned via gRPC. This connects the Min/Max aggregate converters to the
  search response builder.

- AggregateProtoUtils: Central dispatcher for converting InternalAggregation
  to proto Aggregate, with metadata and sub-aggregation helpers

- MinAggregateProtoUtils: Converts InternalMin → proto MinAggregate
- MaxAggregateProtoUtils: Converts InternalMax → proto MaxAggregate
- Handles special values (infinity, NaN), formatting, and metadata

**OpenSearch Core Changes:**

- InternalNumericMetricsAggregation: Added getFormat() getter to expose
  format information for gRPC converters

**Request Conversion Tests:**

- AggregationContainerProtoUtilsTests: 11 tests
- MinAggregationProtoUtilsTests: 108 tests
- MaxAggregationProtoUtilsTests: 108 tests
- ValuesSourceAggregationProtoUtilsTests: 40+ tests
- SearchSourceBuilderProtoUtilsTests: 2 new aggregation tests

**Response Conversion Tests:**

- AggregateProtoUtilsTests: 10 tests
- MinAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- MaxAggregateProtoUtilsTests: 190 tests (values, infinity, NaN, formatting)
- SearchResponseSectionsProtoUtilsTests: 2 new aggregation tests (null/present)
- InternalMinTests: 32 tests
- InternalMaxTests: 32 tests

**Total: ~680 test cases**

**Design Principles:**

- Mirrors REST API patterns for consistency
- Maintains behavioral parity with REST layer
- Comprehensive error handling and validation
- Special value support (infinity, NaN) matching REST behavior
- Metadata handling consistent across all aggregations

**Implementation Summary:**

- **New Implementation Files**: 11 (converters + infrastructure)
- **New Test Files**: 8 (comprehensive test coverage)
- **Modified Files**: 4 (SearchSourceBuilder + SearchResponseSections + server changes)
- **Package Documentation**: 5 package-info.java files
- **Total Lines**: ~1,850 (implementation + tests)

**Testing & Validation:**

- Comprehensive integration testing via REST and gRPC
- Verified behavioral parity (see min_max_aggregation_comparison_report.md)
- All core functionality confirmed working correctly
- Values match exactly between REST and gRPC responses

**API Differences (By Design):**

- REST uses simple JSON values; gRPC uses typed protobuf structures
- Parameter formats differ (e.g., missing parameter requires FieldValue in gRPC)
- Field naming follows protobuf conventions (camelCase vs snake_case)

Co-Authored-By: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Signed-off-by: Yiyu Pan <yypan14@gmail.com>

* Add TermsAggregation implementation for grpc

Signed-off-by: Patrick Zhai <pzhai@uber.com>

* update aggregation response based on new protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* use release protobuf

Signed-off-by: xil <fridalu66@gmail.com>

* add missing javadoc

Signed-off-by: xil <fridalu66@gmail.com>

* Add java doc to spi

Signed-off-by: xil <fridalu66@gmail.com>

* Address PR Code Suggestions

Signed-off-by: xil <fridalu66@gmail.com>

* Address suggestion

Signed-off-by: xil <fridalu66@gmail.com>

* address comment

Signed-off-by: xil <fridalu66@gmail.com>

* increase test coverage

Signed-off-by: xil <fridalu66@gmail.com>

* spotlessApply

Signed-off-by: xil <fridalu66@gmail.com>

---------

Signed-off-by: Yiyu Pan <yypan14@gmail.com>
Signed-off-by: Patrick Zhai <pzhai@uber.com>
Signed-off-by: xil <fridalu66@gmail.com>
Co-authored-by: Yiyu Pan <yypan14@gmail.com>
Co-authored-by: Claude (claude-sonnet-4-5) <noreply@anthropic.com>
Co-authored-by: Patrick Zhai <pzhai@uber.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants