Add version details in remote index path file with code enhancements by ashking94 · Pull Request #13386 · opensearch-project/OpenSearch

ashking94 · 2024-04-25T13:03:20Z

Description

This is a follow up PR after #13150. In this PR, we are doing the following things -

Refactored the IndexMetadataUploadListener to support a more generic hook for index metadata upload.
Add index metadata version and file version in the filename of the remote index path file.

Related Issues

Resolves #13302

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-04-25T13:14:19Z

❌ Gradle check result for 522e527: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2024-04-25T13:59:11Z

❌ Gradle check result for 1fb08f0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ashish Singh <ssashish@amazon.com>

github-actions · 2024-04-25T17:47:09Z

❌ Gradle check result for bd3ab77: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ashking94 · 2024-04-25T18:05:14Z

❌ Gradle check result for bd3ab77: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky tests - #10006 and #10152.

github-actions · 2024-04-26T05:53:42Z

❌ Gradle check result for cacc26d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ashking94 · 2024-04-26T06:09:33Z

❌ Gradle check result for cacc26d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #12197

Signed-off-by: Ashish Singh <ssashish@amazon.com>

rramachand21 · 2024-04-26T06:19:28Z

Triage: Updated assignee and project board fields.

github-actions · 2024-04-26T07:07:46Z

❌ Gradle check result for e92892c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ashking94 · 2024-04-26T08:26:34Z

❌ Gradle check result for e92892c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #12197

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java

server/src/main/java/org/opensearch/index/remote/RemoteStoreUtils.java

server/src/main/java/org/opensearch/index/remote/RemoteIndexPathUploader.java

Signed-off-by: Ashish Singh <ssashish@amazon.com>

github-actions · 2024-04-29T12:26:22Z

❌ Gradle check result for de1f70f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ashking94 · 2024-04-29T12:33:16Z

❌ Gradle check result for de1f70f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky tests - #12651 and #13437.

github-actions · 2024-04-29T13:29:47Z

✅ Gradle check result for de1f70f: SUCCESS

…13386) Signed-off-by: Ashish Singh <ssashish@amazon.com> (cherry picked from commit 2dc3f74) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…13386) (#13444) (cherry picked from commit 2dc3f74) Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

ashking94 requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, dblock, dbwiddis, dreamer-89, gbbafna, kotwanikunal, mch2, msfroh, nknize, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja, sohami and tlfeng as code owners April 25, 2024 13:03

github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Remote v2.14.0 labels Apr 25, 2024

ashking94 added the skip-changelog label Apr 25, 2024

ashking94 force-pushed the 13302 branch from 522e527 to 1fb08f0 Compare April 25, 2024 13:26

Add version details in remote index path file with code enhancements

bd3ab77

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 force-pushed the 13302 branch from 1fb08f0 to bd3ab77 Compare April 25, 2024 16:54

rramachand21 assigned ashking94 Apr 26, 2024

Merge remote-tracking branch 'upstream/main' into 13302

e92892c

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 force-pushed the 13302 branch from cacc26d to e92892c Compare April 26, 2024 06:19

ashking94 mentioned this pull request Apr 27, 2024

Upload remote paths during index creation or full cluster upload #13150

Merged

6 tasks

gbbafna reviewed Apr 29, 2024

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java Outdated Show resolved Hide resolved

server/src/main/java/org/opensearch/index/remote/RemoteStoreUtils.java Outdated Show resolved Hide resolved

gbbafna reviewed Apr 29, 2024

View reviewed changes

server/src/main/java/org/opensearch/index/remote/RemoteIndexPathUploader.java Show resolved Hide resolved

ashking94 mentioned this pull request Apr 29, 2024

[META] Optimised prefix pattern per shard for remote store data and metadata files for higher throughput #12589

Closed

13 tasks

ashking94 added 2 commits April 29, 2024 16:57

Incoporate PR review feedback

c0bcbb7

Signed-off-by: Ashish Singh <ssashish@amazon.com>

Merge remote-tracking branch 'upstream/main' into 13302

de1f70f

Signed-off-by: Ashish Singh <ssashish@amazon.com>

gbbafna approved these changes Apr 29, 2024

View reviewed changes

gbbafna merged commit 2dc3f74 into opensearch-project:main Apr 29, 2024

gbbafna added the backport 2.x Backport to 2.x branch label Apr 29, 2024

opensearch-trigger-bot bot mentioned this pull request Apr 29, 2024

[Backport 2.x] Add version details in remote index path file with code enhancements #13444

Merged

BrewTestBot mentioned this pull request May 14, 2024

opensearch 2.14.0 Homebrew/homebrew-core#171722

Merged

Conversation

ashking94 commented Apr 25, 2024

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Apr 25, 2024

Uh oh!

github-actions bot commented Apr 25, 2024

Uh oh!

github-actions bot commented Apr 25, 2024

Uh oh!

ashking94 commented Apr 25, 2024

Uh oh!

github-actions bot commented Apr 26, 2024

Uh oh!

ashking94 commented Apr 26, 2024

Uh oh!

rramachand21 commented Apr 26, 2024

Uh oh!

github-actions bot commented Apr 26, 2024

Uh oh!

ashking94 commented Apr 26, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 29, 2024

Uh oh!

ashking94 commented Apr 29, 2024

Uh oh!

github-actions bot commented Apr 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants