[Close Index API] Add TransportShardCloseAction for pre-closing verifications#36249
Conversation
tlrx
added
>enhancement
v7.0.0
:Distributed/Distributed
|
Pinging @elastic/es-distributed |
There was a problem hiding this comment.
I hesitated on the name of the action and request as it does not really close the shard. Suggestions welcome.
There was a problem hiding this comment.
TransportVerifyShardBeforeCloseAction? I know it's long, but it's OK, I think.
There was a problem hiding this comment.
Thanks for the suggestion - verbose but better than TransportShardCloseAction
There was a problem hiding this comment.
TransportVerifyShardBeforeCloseAction? I know it's long, but it's OK, I think.
There was a problem hiding this comment.
can we just inline this in the transport action file? I don't see the need for another file.
There was a problem hiding this comment.
I think we want a management thread pool here?
There was a problem hiding this comment.
can we put the shardId first in the log line so it will look like [index][0] shard is ready for closing ?
tlrx
force-pushed
the
add-transport-shard-close-action
branch
from
c5a0dff to
d3e786d
Compare
tlrx
force-pushed
the
add-transport-shard-close-action
branch
from
d3e786d to
feac528
Compare
|
Thanks @bleskes |
The commit changes how indices are closed in the MetaDataIndexStateService. It now uses a 3 steps process where writes are blocked on indices to be closed, then some verifications are done on shards using the TransportVerifyShardBeforeCloseAction added in #36249, and finally indices states are moved to CLOSE and their routing tables removed. The closing process also takes care of using the pre-7.0 way to close indices if the cluster contains mixed version of nodes and a node does not support the TransportVerifyShardBeforeCloseAction. It also closes unassigned indices. Related to #33888
This commit backports to 6.x of the Close Index API refactoring. It cherry-picks the following commits from master: 3ca885e [Close Index API] Add TransportShardCloseAction for pre-closing verifications (#36249) 8e5dd20 [Close Index API] Refactor MetaDataIndexStateService (#36354) 7372529 [Tests] Reduce randomization in CloseWhileRelocatingShardsIT (#36694) 103c4d4 [Close Index API] Mark unavailable shard copy as stale during verification (#36755) 1959388 [Close Index API] Propagate tasks ids between Freeze, Close and Verify(#36630) e149b08 [Close Index API] Add unique UUID to ClusterBlock (#36775) dc371ef [Tests] Fix ReopenWhileClosingIT with correct min num shards The following two commits were needed to adapt the change to 6.x: ef6ae69 [Close Index API] Adapt MetaDataIndexStateServiceTests after merge 21b7653 [Tests] Adapt CloseIndexIT tests for 6.x Related to #33888
4 participants
Note: this pull request will be merged in the
close-index-api-refactoringbranchThis pull request adds the
TransportShardCloseActionwhich is a transport replication action that acquires all index shard permits for its execution. This action will be used in the future by theMetaDataIndexStateServicein a new index closing process, where we need to execute some sanity checks before closing an index.The action executes the following verifications on the primary and replicas:
When the verifications are done and successful, the shard is flushed.
Once this pull request is merged, a follow up pull request will change the
MetaDataIndexStateServiceto make use of this new transport action.