WLM group custom search settings#20536
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis change implements support for custom search settings at the WLM group level. It introduces a searchSettings field to workload groups, adds validation logic for supported settings, enables serialization/deserialization with version awareness, and applies group-level settings to search requests during request processing. Changes
Sequence DiagramsequenceDiagram
participant Client as Client/Request
participant Listener as WorkloadGroupRequestOperationListener
participant ClusterState as Cluster State
participant WorkloadGroup as WorkloadGroup
participant SearchReq as SearchRequest
participant SearchService as SearchService
participant Aggregator as MultiBucketConsumer
Client->>Listener: onRequestStart(searchRequestContext)
Listener->>ClusterState: getMetadata().workloadGroups()
ClusterState->>Listener: WorkloadGroup
Listener->>WorkloadGroup: getSearchSettings()
WorkloadGroup->>Listener: Map<String, String> searchSettings
loop For each setting (batched_reduce_size, timeout, etc.)
Listener->>SearchReq: apply setting (respecting min/max logic)
Note over Listener,SearchReq: Handle BATCHED_REDUCE_SIZE,<br/>TIMEOUT, CANCEL_AFTER_TIME_INTERVAL,<br/>MAX_CONCURRENT_SHARD_REQUESTS,<br/>MAX_BUCKET, PHASE_TOOK
end
Listener->>SearchService: proceed with updated SearchRequest
SearchService->>SearchService: parseSource (check MAX_BUCKET header)
SearchService->>Aggregator: create(limit) or create()
Aggregator->>SearchService: MultiBucketConsumer configured
SearchService->>Client: execute search with WLM settings applied
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Suggested labels
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
❌ Gradle check result for ebc4077: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for ffeb770: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
PR Reviewer Guide 🔍(Review updated until commit 965d2f1)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 965d2f1 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 0287c7a
Suggestions up to commit 2b3d1b4
Suggestions up to commit 4dde46a
Suggestions up to commit 0ee5538
Suggestions up to commit 2cad3c6
|
2cad3c6 to
0ee5538
Compare
|
Persistent review updated to latest commit 0ee5538 |
|
❌ Gradle check result for 0ee5538: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
Persistent review updated to latest commit 4dde46a |
|
❌ Gradle check result for 4dde46a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
kaushalmahi12
left a comment
There was a problem hiding this comment.
Lets resolve the merge conflict, rest LGTM!
|
Persistent review updated to latest commit 2b3d1b4 |
server/src/main/java/org/opensearch/wlm/MutableWorkloadGroupFragment.java
Show resolved
Hide resolved
|
❕ Gradle check result for 2b3d1b4: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #20536 +/- ##
============================================
- Coverage 73.35% 73.33% -0.03%
- Complexity 72253 72265 +12
============================================
Files 5794 5795 +1
Lines 329944 330042 +98
Branches 47620 47641 +21
============================================
- Hits 242037 242020 -17
- Misses 68483 68631 +148
+ Partials 19424 19391 -33 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: David Zane <davizane@amazon.com>
|
Persistent review updated to latest commit 0287c7a |
|
❌ Gradle check result for 0287c7a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
jainankitk
left a comment
There was a problem hiding this comment.
Can you add few more tests to increase the code coverage?
Signed-off-by: David Zane <davizane@amazon.com>
|
Persistent review updated to latest commit 965d2f1 |
Description
This PR adds the foundational infrastructure for defining custom search settings within a WLM group, along with the first setting implementation (
timeout).What's included
WorkloadGroupSearchSettingsenum with validation framework for search settingssearch_settingsfield inMutableWorkloadGroupFragmentandWorkloadGroupWorkloadGroupRequestOperationListenerintegration to apply settings during request starttimeoutsetting implementationSupported setting (this PR)
timeout- Enforces a hard upper bound on how long a search is allowed to execute, helping keep latency predictable for a workload group and avoiding situations where slow or stalled queries tie up search threads. The WLM timeout is only applied when the request does not already have an explicit timeout set.Usage
Create WLM group with
search_settings:Update WLM group:
Related Issues
Part of #20555
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.