Skip to content

Add additional details to search backpressure stats #11982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kolchfa-aws wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add additional details to search backpressure stats #11982

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions _tuning-your-cluster/availability-and-recovery/search-backpressure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,12 +223,12 @@ The response contains the following fields.
Field Name | Data type | Description
:--- | :--- | :---
search_backpressure | Object | Statistics about search backpressure.
search_backpressure.search_task | Object | Statistics specific to the search task.
search_backpressure.search_task.[resource_tracker_stats](#resource_tracker_stats) | Object | Statistics about the current search tasks.
search_backpressure.search_task.[cancellation_stats](#cancellation_stats) | Object | Statistics about the search tasks canceled since the node last restarted.
search_backpressure.search_shard_task | Object | Statistics specific to the search shard task.
search_backpressure.search_shard_task.[resource_tracker_stats](#resource_tracker_stats) | Object | Statistics about the current search shard tasks.
search_backpressure.search_shard_task.[cancellation_stats](#cancellation_stats) | Object | Statistics about the search shard tasks canceled since the node last restarted.
search_backpressure.search_task | Object | Statistics for search tasks. Contains resource tracker statistics for individual cancellation criteria (heap, CPU, elapsed time) and a summary of all cancellation activity across these trackers.
search_backpressure.search_task.[resource_tracker_stats](#resource_tracker_stats) | Object | Per-tracker statistics showing cancellation counts for each resource type (heap usage, CPU usage, elapsed time) and current resource consumption metrics.
search_backpressure.search_task.[cancellation_stats](#cancellation_stats) | Object | Aggregated cancellation statistics across all resource trackers. The sum of `cancellation_count` and `cancellation_limit_reached_count` equals the total of all resource tracker cancellation counts.
search_backpressure.search_shard_task | Object | Statistics for search shard tasks. Contains resource tracker statistics for individual cancellation criteria (heap, CPU, elapsed time) and a summary of all cancellation activity across these trackers.
search_backpressure.search_shard_task.[resource_tracker_stats](#resource_tracker_stats) | Object | Per-tracker statistics showing cancellation counts for each resource type (heap usage, CPU usage, elapsed time) and current resource consumption metrics.
search_backpressure.search_shard_task.[cancellation_stats](#cancellation_stats) | Object | Aggregated cancellation statistics across all resource trackers. The sum of `cancellation_count` and `cancellation_limit_reached_count` equals the total of all resource tracker cancellation counts.
search_backpressure.mode | String | The [mode](#search-backpressure-modes) for search backpressure.

### `resource_tracker_stats`
Expand Down Expand Up @@ -274,3 +274,6 @@ Field Name | Data type | Description
:--- | :--- | :---
cancellation_count | Integer | The total number of tasks marked for cancellation since the node last restarted.
cancellation_limit_reached_count | Integer | The number of times when the number of tasks eligible for cancellation exceeded the set cancellation threshold.

Each resource tracker (heap, CPU, elapsed time) independently identifies tasks that breach its thresholds and increments its own `cancellation_count`. Since a single task may breach multiple resource thresholds, the sum of resource tracker `cancellation_count` values may exceed the top-level `cancellation_count`, which represents the actual number of unique tasks that were cancelled. The `cancellation_limit_reached_count` increments when the cancellation rate limit is reached during an observer iteration, preventing additional cancellations in that iteration.
{: .note}