Choose the best performing node when writing with append only index

[kkewwei]

Description

We will use the following settings to enable adaptive shard selection in automatic routing for bulk.

PUT index1
{
    "settings": {
        "index.append_only.enabled": "true",
        "index.bulk.adaptive_shard_selection.enabled": "true"
    }
}

OR

For append_only index:

PUT index1/_settings
{
    "settings": {
        "index.bulk.adaptive_shard_selection.enabled": "true"
    }
}

Related Issues

Resolves #18307 #18306 #9219

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Summary by CodeRabbit

• New Features

  • Adaptive shard selection for append‑only indices to prefer higher‑performing nodes for bulk writes
  • New index setting to enable/disable adaptive shard selection (default: disabled)
  • Bulk responses include per‑node metrics (service‑time EWMA and queue size)

• Bug Fixes

  • Validation prevents enabling adaptive selection on non‑append‑only indices
  • Bulk operations return validation errors when constraints are violated

• Tests / Documentation

  • Added REST test scenario and changelog entry with version gate and error cases

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[kkewwei]

@reta @gaobinlong Would you mind have a look in your spare time?

[github-actions]

❌ Gradle check result for bc2e09f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for f891d9b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for cbe9bb1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for de7ac01: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for de7ac01: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for de7ac01: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for bf5e1fa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for bf5e1fa: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for cb5e5d3: SUCCESS

[kkewwei]

@gaobinlong @andrross Thanks for your reviews! I’ll merge this PR in two days. Feel free to feedback if you have any concerns.

[andrross]

@gaobinlong @andrross Thanks for your reviews! I’ll merge this PR in two days. Feel free to feedback if you have any concerns.

Thanks @kkewwei, I just merged it!