Fix flakiness of testRemoteCleanupDeleteStale, bug fix in RemoteMetadataManifest and RemoteReadResult by shiv0408 · Pull Request #14230 · opensearch-project/OpenSearch

shiv0408 · 2024-06-12T14:56:18Z

Description

We see that Cleanup interval setting set before the might not be applied before the interval for assert is over.
Added a check to ensure that setting is updated before checking if files are deleted.

Further, addressing some bugs found after merging #14089

Addressing a bug in getManifestCodecVersion where if path for files have Delimiter __ we are throwing an error.
Changed that expected object in RemoteReadResult to move away from XContent, as this was decided in Add toXContent and fromXContent for DiscoveryNode and DiscoveryNodes #13576 but not updated in Add POJO classes required for cluster state publication from remote #14006

Ran the test 1000 times to ensure that flakiness is not there now.

Related Issues

Resolves #14205
Resolves #13968

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-06-12T15:14:46Z

❌ Gradle check result for a8bf4b0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/gateway/remote/model/RemoteClusterMetadataManifest.java

github-actions · 2024-06-13T11:42:11Z

❌ Gradle check result for f5831ac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-06-13T12:07:31Z

❌ Gradle check result for 376eb8c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

prudhvigodithi · 2024-06-13T23:25:34Z

From this AUTOCUT issue #14315 (comment) I can see the testEndToEndRemoteMigration has failed on some other PR's and post merge actions.

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

github-actions · 2024-06-14T10:11:30Z

❌ Gradle check result for af9c6be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

shiv0408 · 2024-06-14T10:46:14Z

❌ Gradle check result for af9c6be: FAILURE

org.opensearch.cache.common.tier.TieredSpilloverCacheStatsIT.testIndicesAndTierLevelAggregation - #14281

github-actions · 2024-06-14T11:53:23Z

❌ Gradle check result for af9c6be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

shwetathareja · 2024-06-14T11:55:51Z

❌ Gradle check result for af9c6be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

shiv0408 · 2024-06-14T12:05:48Z

org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock - #10006
org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex - #11547

github-actions · 2024-06-14T12:55:23Z

❌ Gradle check result for af9c6be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-06-14T14:00:16Z

✅ Gradle check result for af9c6be: SUCCESS

…ataManifest and RemoteReadResult (#14230) * Address flakiness of testRemoteCleanupDeleteStale - Make RemoteReadResult have Object rather than ToXContent - Fix getManifestCodecVersion in RemoteClusterMetadataManifest Signed-off-by: Shivansh Arora <hishiv@amazon.com> (cherry picked from commit 0c2ff03) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ataManifest and RemoteReadResult (#14230) (#14354) (cherry picked from commit 0c2ff03) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ataManifest and RemoteReadResult (#14230) (#14353) * Address flakiness of testRemoteCleanupDeleteStale - Make RemoteReadResult have Object rather than ToXContent - Fix getManifestCodecVersion in RemoteClusterMetadataManifest (cherry picked from commit 0c2ff03) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ataManifest and RemoteReadResult (opensearch-project#14230) * Address flakiness of testRemoteCleanupDeleteStale - Make RemoteReadResult have Object rather than ToXContent - Fix getManifestCodecVersion in RemoteClusterMetadataManifest Signed-off-by: Shivansh Arora <hishiv@amazon.com>

…ataManifest and RemoteReadResult (opensearch-project#14230) (opensearch-project#14353) * Address flakiness of testRemoteCleanupDeleteStale - Make RemoteReadResult have Object rather than ToXContent - Fix getManifestCodecVersion in RemoteClusterMetadataManifest (cherry picked from commit 0c2ff03) Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: kkewwei <kkewwei@163.com>

…ataManifest and RemoteReadResult (opensearch-project#14230) * Address flakiness of testRemoteCleanupDeleteStale - Make RemoteReadResult have Object rather than ToXContent - Fix getManifestCodecVersion in RemoteClusterMetadataManifest Signed-off-by: Shivansh Arora <hishiv@amazon.com>

shiv0408 requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, dblock, dbwiddis, dreamer-89, gbbafna, kotwanikunal, mch2, msfroh, nknize, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja, sohami and tlfeng as code owners June 12, 2024 14:56

github-actions bot added >test-failure Test failure from CI, local build, etc. autocut bug Something isn't working labels Jun 12, 2024

soosinha reviewed Jun 13, 2024

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/model/RemoteClusterMetadataManifest.java Outdated Show resolved Hide resolved

shiv0408 force-pushed the fix_14205 branch from a8bf4b0 to f5831ac Compare June 13, 2024 11:14

shiv0408 force-pushed the fix_14205 branch from f5831ac to 376eb8c Compare June 13, 2024 11:42

Add unit test

af9c6be

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

shiv0408 force-pushed the fix_14205 branch from 87b05a7 to af9c6be Compare June 14, 2024 09:50

shwetathareja approved these changes Jun 14, 2024

View reviewed changes

shwetathareja added backport 2.x Backport to 2.x branch backport 2.15 labels Jun 14, 2024

shwetathareja merged commit 0c2ff03 into opensearch-project:main Jun 14, 2024

shiv0408 deleted the fix_14205 branch June 14, 2024 14:04

opensearch-trigger-bot bot mentioned this pull request Jun 14, 2024

[Backport 2.x] Fix flakiness of testRemoteCleanupDeleteStale, bug fix in RemoteMetadataManifest and RemoteReadResult #14353

Merged

opensearch-trigger-bot bot mentioned this pull request Jun 14, 2024

[Backport 2.15] Fix flakiness of testRemoteCleanupDeleteStale, bug fix in RemoteMetadataManifest and RemoteReadResult #14354

Merged

finnegancarroll mentioned this pull request Jun 14, 2024

Remove composeBuild fixture tasks when docker support not found #14357

Merged

3 tasks

This was referenced Jun 13, 2024

[AUTOCUT] Gradle Check Flaky Test Report for MinimumClusterManagerNodesIT #14289

Open

[AUTOCUT] Gradle Check Flaky Test Report for TieredSpilloverCacheStatsIT #14300

Closed

prudhvigodithi mentioned this pull request Jun 14, 2024

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheIT prudhvigodithi/OpenSearch#27

Closed

shiv0408 mentioned this pull request Jun 17, 2024

[AUTOCUT] Gradle Check Flaky Test Report for RemoteStoreClusterStateRestoreIT #14326

Open

opensearch-ci-bot mentioned this pull request Jun 17, 2024

[AUTOCUT] Gradle Check Flaky Test Report for IndexServiceTests #14407

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix flakiness of testRemoteCleanupDeleteStale, bug fix in RemoteMetadataManifest and RemoteReadResult#14230

Fix flakiness of testRemoteCleanupDeleteStale, bug fix in RemoteMetadataManifest and RemoteReadResult#14230
shwetathareja merged 3 commits intoopensearch-project:mainfrom
shiv0408:fix_14205

shiv0408 commented Jun 12, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jun 12, 2024

Uh oh!

Uh oh!

github-actions bot commented Jun 13, 2024

Uh oh!

github-actions bot commented Jun 13, 2024

Uh oh!

prudhvigodithi commented Jun 13, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

shiv0408 commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

shwetathareja commented Jun 14, 2024

Uh oh!

shiv0408 commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

shiv0408 commented Jun 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Jun 12, 2024

Uh oh!

Uh oh!

github-actions bot commented Jun 13, 2024

Uh oh!

github-actions bot commented Jun 13, 2024

Uh oh!

prudhvigodithi commented Jun 13, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

shiv0408 commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

shwetathareja commented Jun 14, 2024

Uh oh!

shiv0408 commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

github-actions bot commented Jun 14, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

shiv0408 commented Jun 12, 2024 •

edited

Loading