[Backport to 2.x] [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. by dreamer-89 · Pull Request #4450 · opensearch-project/OpenSearch

dreamer-89 · 2022-09-07T18:42:35Z

Manual backport of #4402 to 2.x

dreamer-89 · 2022-09-07T18:43:33Z

Looks like something is missing. There is only one file change in this backport PR.

github-actions · 2022-09-07T18:53:24Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2850/
CommitID: 469228a

dreamer-89 · 2022-09-07T18:58:25Z

Java compilation error :(

> Task :rest-api-spec:validatePom

        TranslogManager translogManager = engine.translogManager();
> Task :test:framework:compileJava
                                                ^
  symbol:   method translogManager()
  location: variable engine of type Engine

> Task :server:processTestResources
> Task :server:dependencyLicenses
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error

> Task :test:framework:compileJava

> Task :test:framework:compileJava FAILED
> Task :server:filepermissions

github-actions · 2022-09-07T19:03:02Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2851/
CommitID: 62049fd

github-actions · 2022-09-07T19:03:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2849/
CommitID: 712cb71

github-actions · 2022-09-07T19:20:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2856/
CommitID: 5467140

dreamer-89 · 2022-09-07T19:24:37Z

Java spotless failure.

github-actions · 2022-09-07T19:47:23Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2857/
CommitID: 407dc92

dreamer-89 · 2022-09-07T19:56:05Z

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':distribution:bwc:maintenance:buildBwcLinuxTar'.
> Building 1.3.5 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/maintenance/build/bwc/checkout-1.3/distribution/archives/linux-tar/build/distributions/opensearch-min-1.3.5-SNAPSHOT-linux-x64.tar.gz

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
org.gradle.api.GradleException: Reaper process failed. Check log at /var/jenkins/workspace/gradle-check/search/.gradle/reaper/build-13277/error.log for details
> Reaper process failed. Check log at /var/jenkins/workspace/gradle-check/search/.gradle/reaper/build-13277/error.log for details

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================

* Get more help at https://help.gradle.org/

BUILD FAILED in 16m 29s
2432 actionable tasks: 2418 executed, 2 from cache, 12 up-to-date
Gradle Check Failed!

…uting metadata snapshot on primary shards. (opensearch-project#4366) * Segment Replication - Fix NoSuchFileException errors caused when computing metadata snapshot on primary shards. This change fixes the errors that occur when computing metadata snapshots on primary shards from the latest in-memory SegmentInfos. The error occurs when a segments_N file that is referenced by the in-memory infos is deleted as part of a concurrent commit. The segments themselves are incref'd by IndexWriter.incRefDeleter but the commit file (Segments_N) is not. This change resolves this by ignoring the segments_N file when computing metadata for CopyState and only sending incref'd segment files to replicas. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix spotless. Signed-off-by: Marc Handalian <handalm@amazon.com> * Update StoreTests.testCleanupAndPreserveLatestCommitPoint to assert additional segments are deleted. Signed-off-by: Marc Handalian <handalm@amazon.com> * Rename snapshot to metadataMap in CheckpointInfoResponse. Signed-off-by: Marc Handalian <handalm@amazon.com> * Refactor segmentReplicationDiff method to compute off two maps instead of MetadataSnapshots. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix spotless. Signed-off-by: Marc Handalian <handalm@amazon.com> * Revert catchall in SegmentReplicationSourceService. Signed-off-by: Marc Handalian <handalm@amazon.com> * Revert log lvl change. Signed-off-by: Marc Handalian <handalm@amazon.com> * Fix SegmentReplicationTargetTests Signed-off-by: Marc Handalian <handalm@amazon.com> * Cleanup unused logger. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Suraj Singh <surajrider@gmail.com>

…d of relying on segments_N from primary shards. (opensearch-project#4402) * Segment Replication - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards. This change updates replicas to commit SegmentInfos before the shard is closed, on receiving a new commit point from a primary, and when a new primary is detected. This change also makes the public commitSegmentInfos on NRTEngine obsolete, refactoring IndexShard to simply call reset on the engine. Signed-off-by: Marc Handalian <handalm@amazon.com> * Remove noise & extra log statement. Signed-off-by: Marc Handalian <handalm@amazon.com> * PR feedback. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Marc Handalian <handalm@amazon.com>

…anslog() Signed-off-by: Suraj Singh <surajrider@gmail.com>

Signed-off-by: Suraj Singh <surajrider@gmail.com>

mch2

thanks for the backport @dreamer-89

github-actions · 2022-09-07T22:52:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/2866/
CommitID: 8a18787

codecov-commenter · 2022-09-07T22:55:05Z

Codecov Report

❌ Patch coverage is 96.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 70.62%. Comparing base (ceb0e17) to head (8a18787).
⚠️ Report is 2741 commits behind head on 2.x.

Files with missing lines	Patch %	Lines
...va/org/opensearch/index/engine/EngineTestCase.java	66.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x    #4450      +/-   ##
============================================
- Coverage     70.67%   70.62%   -0.05%     
+ Complexity    57179    57153      -26     
============================================
  Files          4585     4585              
  Lines        274506   274507       +1     
  Branches      40227    40229       +2     
============================================
- Hits         193996   193878     -118     
- Misses        64278    64389     +111     
- Partials      16232    16240       +8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dreamer-89 requested review from a team and reta as code owners September 7, 2022 18:42

dreamer-89 force-pushed the mch2_replicaCommits_2x branch from 469228a to 62049fd Compare September 7, 2022 18:52

dreamer-89 requested a review from mch2 September 7, 2022 18:53

dreamer-89 mentioned this pull request Sep 7, 2022

Add bwcVersion 1.3.6 to 2.x #4452

Merged

6 tasks

mch2 and others added 4 commits September 7, 2022 15:15

[Segment Replicaiton] Fix merge conflict. Update EngineTestCase.getTr…

bb6db6d

…anslog() Signed-off-by: Suraj Singh <surajrider@gmail.com>

Spotless check apply fixes

8a18787

Signed-off-by: Suraj Singh <surajrider@gmail.com>

dreamer-89 force-pushed the mch2_replicaCommits_2x branch from 407dc92 to 8a18787 Compare September 7, 2022 22:15

mch2 approved these changes Sep 7, 2022

View reviewed changes

dreamer-89 merged commit 4170d37 into opensearch-project:2.x Sep 7, 2022

This was referenced Sep 7, 2022

Add bwcVersion 1.3.6 to 2.2 #4453

Merged

Add 1.3.6 bwc version to 2.0 branch #4491

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport to 2.x] [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards.#4450

[Backport to 2.x] [Segment Replication] - Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards.#4450
dreamer-89 merged 4 commits into
opensearch-project:2.xfrom
dreamer-89:mch2_replicaCommits_2x

dreamer-89 commented Sep 7, 2022

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

mch2 left a comment

Uh oh!

github-actions Bot commented Sep 7, 2022

Uh oh!

codecov-commenter commented Sep 7, 2022 •

edited by codecov Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

dreamer-89 commented Sep 7, 2022

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

dreamer-89 commented Sep 7, 2022

Uh oh!

mch2 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 7, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

codecov-commenter commented Sep 7, 2022 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Sep 7, 2022 •

edited by codecov Bot

Loading