Skip to content

Refactor and optimize the aggregation part of grouping stages#100

Open
zhaih wants to merge 1 commit intoopensearch-project:mainfrom
zhaih:grouping
Open

Refactor and optimize the aggregation part of grouping stages#100
zhaih wants to merge 1 commit intoopensearch-project:mainfrom
zhaih:grouping

Conversation

@zhaih
Copy link
Copy Markdown
Contributor

@zhaih zhaih commented Mar 12, 2026

Description

Similar approach as #43 (using a fixed size array instead of hashmap), but apply to general grouping stages as well as reduce phase

Benchmark

Showing roughly 15 - 30% gain
Baseline

Benchmark                                       (cardinality)  (labelCount)  (method)  (sampleCount)  Mode  Cnt    Score    Error  Units
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       sum            100  avgt    3    0.376 ±  0.040  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       sum           1000  avgt    3    2.264 ±  0.695  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       avg            100  avgt    3    0.386 ±  0.040  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       avg           1000  avgt    3    2.296 ±  1.805  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       sum            100  avgt    3   25.300 ±  9.160  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       sum           1000  avgt    3  209.194 ± 20.337  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       avg            100  avgt    3   26.020 ±  6.213  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       avg           1000  avgt    3  206.281 ± 35.858  ms/op

PR

Benchmark                                       (cardinality)  (labelCount)  (method)  (sampleCount)  Mode  Cnt    Score    Error  Units
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       sum            100  avgt    3    0.291 ±  0.038  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       sum           1000  avgt    3    1.320 ±  0.072  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       avg            100  avgt    3    0.333 ±  0.025  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping            100            20       avg           1000  avgt    3    1.708 ±  0.263  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       sum            100  avgt    3   15.921 ±  1.266  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       sum           1000  avgt    3  119.415 ±  4.327  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       avg            100  avgt    3   20.678 ±  2.058  ms/op
GroupingSampleStageBenchmark.benchmarkGrouping          10000            20       avg           1000  avgt    3  170.672 ± 10.899  ms/op

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Patrick Zhai <pzhai@uber.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 12, 2026

Codecov Report

❌ Patch coverage is 93.26923% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.16%. Comparing base (8cdcce1) to head (4232d45).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...va/org/opensearch/tsdb/lang/m3/stage/AvgStage.java 83.33% 0 Missing and 1 partial ⚠️
...va/org/opensearch/tsdb/lang/m3/stage/MaxStage.java 83.33% 0 Missing and 1 partial ⚠️
...va/org/opensearch/tsdb/lang/m3/stage/MinStage.java 83.33% 0 Missing and 1 partial ⚠️
...g/opensearch/tsdb/lang/m3/stage/MultiplyStage.java 83.33% 0 Missing and 1 partial ⚠️
...ch/tsdb/lang/m3/stage/PercentileOfSeriesStage.java 91.66% 0 Missing and 1 partial ⚠️
.../org/opensearch/tsdb/lang/m3/stage/RangeStage.java 83.33% 0 Missing and 1 partial ⚠️
...va/org/opensearch/tsdb/lang/m3/stage/SumStage.java 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #100      +/-   ##
============================================
+ Coverage     89.13%   89.16%   +0.03%     
+ Complexity     5035     5030       -5     
============================================
  Files           322      324       +2     
  Lines         15518    15547      +29     
  Branches       2337     2344       +7     
============================================
+ Hits          13832    13863      +31     
+ Misses         1017     1009       -8     
- Partials        669      675       +6     
Flag Coverage Δ
unittests 89.16% <93.26%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...g/m3/stage/AbstractGroupingDoubleBucketsStage.java 100.00% <100.00%> (ø)
...g/m3/stage/AbstractGroupingSampleBucketsStage.java 100.00% <100.00%> (ø)
...sdb/lang/m3/stage/AbstractGroupingSampleStage.java 100.00% <100.00%> (+1.14%) ⬆️
...va/org/opensearch/tsdb/lang/m3/stage/AvgStage.java 94.11% <83.33%> (-2.76%) ⬇️
...va/org/opensearch/tsdb/lang/m3/stage/MaxStage.java 95.45% <83.33%> (+0.21%) ⬆️
...va/org/opensearch/tsdb/lang/m3/stage/MinStage.java 95.45% <83.33%> (+0.21%) ⬆️
...g/opensearch/tsdb/lang/m3/stage/MultiplyStage.java 95.45% <83.33%> (+0.21%) ⬆️
...ch/tsdb/lang/m3/stage/PercentileOfSeriesStage.java 93.10% <91.66%> (-0.65%) ⬇️
.../org/opensearch/tsdb/lang/m3/stage/RangeStage.java 97.05% <83.33%> (-2.95%) ⬇️
...va/org/opensearch/tsdb/lang/m3/stage/SumStage.java 95.45% <83.33%> (+0.21%) ⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant