[Remote Store] Changes to perform repository registration during bootstrap via node attributes.#9105
Conversation
Gradle Check (Jenkins) Run Completed with:
|
|
Compatibility status: |
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/RemoteStoreJoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/RemoteStoreJoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinHelper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/RemoteStoreJoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
|
Compatibility status: |
Gradle Check (Jenkins) Run Completed with:
|
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
|
How about simplifying the configurations to |
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
|
Compatibility status: |
Gradle Check (Jenkins) Run Completed with:
|
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
...rch/action/admin/cluster/remotestore/repository/RemoteStoreRepositoryRegistrationHelper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/coordination/JoinTaskExecutor.java
Outdated
Show resolved
Hide resolved
ashking94
left a comment
There was a problem hiding this comment.
Pls add more comments on the places where we are updating the cluster state. Specifically where a condition evaluates to true and we update the state. It would help understand under what circumstances it would update the state.
e89fbcf to
bcf8ef3
Compare
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 6a5b464 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
bcf8ef3 to
5ada3cd
Compare
Signed-off-by: Dharmesh 💤 <sdharms@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 8c54a86 Incompatible componentsIncompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
Gradle Check (Jenkins) Run Completed with:
|
|
The test |
|
A different test is flaky now - |
|
The backport to To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9105-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9119b6dc20ea11d95a399c68505f1d858b78e30e
# Push it to GitHub
git push --set-upstream origin backport/backport-9105-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.xThen, create a pull request where the |
…on during bootstrap via node attributes. (opensearch-project#9105) Signed-off-by: Dharmesh 💤 <sdharms@amazon.com> (cherry picked from commit 9119b6d)
…strap via node attributes. (opensearch-project#9105) Signed-off-by: Dharmesh 💤 <sdharms@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
…strap via node attributes. (opensearch-project#9105) Signed-off-by: Dharmesh 💤 <sdharms@amazon.com> Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
…strap via node attributes. (opensearch-project#9105) Signed-off-by: Dharmesh 💤 <sdharms@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
| // use these joins to try and become the cluster-manager. | ||
| // Note that we don't have to do any validation of the amount of joining nodes - the commit | ||
| // during the cluster state publishing guarantees that we have enough | ||
| newState = becomeClusterManagerAndTrimConflictingNodes(currentState, joiningNodes); |
There was a problem hiding this comment.
This could update routingTable and metadata.indices["index-name"].primaryTerms for the primary shards in deleted nodes. Are we overwriting this metadata update in this PR at multiple places by using currentState.metadata() instead of newState.metadata()?
This breaks the assertion at https://github.com/opensearch-project/OpenSearch/blob/2.19/server/src/main/java/org/opensearch/index/shard/IndexShard.java#L846-L850 in the node containing the replica that is to be promoted (according to the routingTable update. But the metadata is stale)
Description
Solves #8623
The changes are to perform repository registration via the values supplied in node attributes during node bootstap.
Reject or accept the join request if they have matchin node attributes.
Below are the format in which the node attributes will be supplied.
Config 1
Config 2
Related
Resolves #8623
Pending Items
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.