[Backport 2.x] Force merge API supports performing only on primary shards (#11269) by gaobinlong · Pull Request #12609 · opensearch-project/OpenSearch

gaobinlong · 2024-03-12T07:47:50Z

Description

Backport #11269 to 2.x.

Related Issues

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…h-project#11269) * Force merge API supports performing on primary shards only Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify change log Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix test failure Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Fix typo Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify skip version Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add version check and more tests Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Format code Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Modify supported version and add more test Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Change the supported version to 3.0.0 Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Add test case in SegmentReplicationIT Signed-off-by: Gao Binlong <gbinlong@amazon.com> * Optimize the test code Signed-off-by: Gao Binlong <gbinlong@amazon.com> --------- Signed-off-by: Gao Binlong <gbinlong@amazon.com> (cherry picked from commit e6eec36)

github-actions · 2024-03-12T07:51:12Z

Compatibility status:

Checks if related components are compatible with change 7d1c255

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security-analytics.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/sql.git]

github-actions · 2024-03-12T07:52:42Z

❌ Gradle check result for 51d9bb3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

github-actions · 2024-03-12T08:09:01Z

❌ Gradle check result for a3465d6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

github-actions · 2024-03-12T11:03:02Z

❌ Gradle check result for 1340aa8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

github-actions · 2024-03-12T14:31:23Z

❌ Gradle check result for e9b194d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

sohami · 2024-03-12T23:38:59Z

❌ Gradle check result for e9b194d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Failures are related to: #12593

github-actions · 2024-03-13T00:31:19Z

❕ Gradle check result for e9b194d: UNSTABLE

TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests.testContainerCreationAndDeletion
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.get_field_mapping/20_missing_field/Return empty object if field doesn't exist, but index does}
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.forcemerge/20_wait_for_completion/Force merge index with wait_for_completion}

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2024-03-13T00:32:01Z

Codecov Report

Attention: Patch coverage is 66.66667% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 71.15%. Comparing base (0dd892c) to head (7d1c255).
Report is 11 commits behind head on 2.x.

Files	Patch %	Lines
.../indices/forcemerge/TransportForceMergeAction.java	0.00%	3 Missing ⚠️
...n/indices/forcemerge/ForceMergeRequestBuilder.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x   #12609      +/-   ##
============================================
- Coverage     71.28%   71.15%   -0.13%     
+ Complexity    60145    60084      -61     
============================================
  Files          4957     4957              
  Lines        282799   282834      +35     
  Branches      41409    41414       +5     
============================================
- Hits         201591   201264     -327     
- Misses        64189    64577     +388     
+ Partials      17019    16993      -26

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sohami · 2024-03-13T05:10:03Z

❕ Gradle check result for e9b194d: UNSTABLE

TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests.testContainerCreationAndDeletion
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.get_field_mapping/20_missing_field/Return empty object if field doesn't exist, but index does}
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.forcemerge/20_wait_for_completion/Force merge index with wait_for_completion}

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@gaobinlong It looks like below test is flaky. Can you check this please ?

>       1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.forcemerge/20_wait_for_completion/Force merge index with wait_for_completion}

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

...api-spec/src/main/resources/rest-api-spec/test/indices.forcemerge/20_wait_for_completion.yml

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

github-actions · 2024-03-13T08:20:32Z

❌ Gradle check result for a25b43d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-03-13T08:23:21Z

❕ Gradle check result for 83fb554: UNSTABLE

TEST FAILURES:

      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.get_field_mapping/20_missing_field/Return empty object if field doesn't exist, but index does}

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

gaobinlong · 2024-03-13T09:13:38Z

❕ Gradle check result for 83fb554: UNSTABLE

TEST FAILURES:
      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.get_field_mapping/20_missing_field/Return empty object if field doesn't exist, but index does}
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

The failed test seems not related to this PR, and there's an issue tracking it but seems we haven't resolved it yet.

…269-to-2.x

github-actions · 2024-03-13T10:10:14Z

❌ Gradle check result for 7d1c255: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-03-13T17:55:57Z

❕ Gradle check result for 7d1c255: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testRestartPrimary_NoReplicas

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

sohami · 2024-03-13T20:49:56Z

❕ Gradle check result for 7d1c255: UNSTABLE

TEST FAILURES:
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testRestartPrimary_NoReplicas
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Known issue: #11993

sohami · 2024-03-13T20:51:40Z

@gaobinlong We will need a forward port to change the version checks in main branch. Also probably we can update the yml tests as done in this PR. Can you also create a documentation issue or PR for this new option ?

gaobinlong · 2024-03-14T02:36:38Z

@gaobinlong We will need a forward port to change the version checks in main branch. Also probably we can update the yml tests as done in this PR. Can you also create a documentation issue or PR for this new option ?

Thanks, I'll create a following PR for the main branch, and both of the related document issue and PR have been created yet.

Modify supported version to 2.13.0

a3465d6

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

Fix test failure

1340aa8

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

Fix yml test failure

e9b194d

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

sohami approved these changes Mar 12, 2024

View reviewed changes

Fix yaml test failure

a25b43d

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

sohami reviewed Mar 13, 2024

View reviewed changes

...api-spec/src/main/resources/rest-api-spec/test/indices.forcemerge/20_wait_for_completion.yml Outdated Show resolved Hide resolved

...api-spec/src/main/resources/rest-api-spec/test/indices.forcemerge/20_wait_for_completion.yml Outdated Show resolved Hide resolved

Fix typo and test failure

83fb554

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

Merge remote-tracking branch 'upstream/2.x' into backport/backport-11…

7d1c255

…269-to-2.x

kkmr approved these changes Mar 13, 2024

View reviewed changes

sohami merged commit 0dfc846 into opensearch-project:2.x Mar 13, 2024

sohami added the documentation pending Tracks issues which have PRs merged but documentation changes pending label Mar 13, 2024

gaobinlong deleted the backport/backport-11269-to-2.x branch June 19, 2024 15:20

Conversation

gaobinlong commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Compatibility status:

Incompatible components

Skipped components

Compatible components

Uh oh!

github-actions bot commented Mar 12, 2024

Uh oh!

github-actions bot commented Mar 12, 2024

Uh oh!

github-actions bot commented Mar 12, 2024

Uh oh!

github-actions bot commented Mar 12, 2024

Uh oh!

sohami commented Mar 12, 2024

Uh oh!

github-actions bot commented Mar 13, 2024

Uh oh!

codecov bot commented Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sohami commented Mar 13, 2024

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 13, 2024

Uh oh!

github-actions bot commented Mar 13, 2024

Uh oh!

gaobinlong commented Mar 13, 2024

Uh oh!

github-actions bot commented Mar 13, 2024

Uh oh!

github-actions bot commented Mar 13, 2024

Uh oh!

sohami commented Mar 13, 2024

Uh oh!

sohami commented Mar 13, 2024

Uh oh!

gaobinlong commented Mar 14, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gaobinlong commented Mar 12, 2024 •

edited

Loading

github-actions bot commented Mar 12, 2024 •

edited

Loading

codecov bot commented Mar 13, 2024 •

edited

Loading