-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Is your feature request related to a problem? Please describe.
Currently, OpenSearch lacks a direct means of providing insights into “top queries” that have a significant impact on latency, we want to build the in-memory aggregator to capture those queries on the coordinator node level.
(Parent rfc: #11186)
Describe the solution you'd like
Implement a priority queue-based in-memory data store, with configurable window size, on the coordinator node, designed to efficiently store the top N queries. The data model of the stored query attributes should be highly extensible for different types of resources and metrics.
Describe alternatives you've considered
We can instead export the data and offload the query instrumentation logic to external sinks. But it would require us to retain all necessary data for every request. Additionally, as of now we lack a state-of-the-art on-node store capable of holding all request information for aggregation and calculating the top N when user requested. Moreover, we refrain from introducing dependencies on a specific sink in this feature.
Additional context
Please see parent rfc: #11186
Metadata
Metadata
Assignees
Labels
Type
Projects
Status