-
Notifications
You must be signed in to change notification settings - Fork 2.5k
[Enhancement] Add a batch allocator for building allocation decisions for multiple primary & replica shards #8960
Description
Describe the bug
Original Issue opened : #5098 . This issue is a sub task for the overall project defined #8098.
Description
As part of node drop and joins, for allocation of unassigned primary and replica shards, PrimaryShardAllocator and ReplicaShardAllocator(in short PSA & RSA) build an AllocateUnassignedDecision. This decision is used for actual allocation of shards by BaseGatewayShardAllocator(i.e. GatewayAllocator).
OpenSearch/server/src/main/java/org/opensearch/gateway/PrimaryShardAllocator.java
Lines 93 to 97 in 96630f0
| public AllocateUnassignedDecision makeAllocationDecision( | |
| final ShardRouting unassignedShard, | |
| final RoutingAllocation allocation, | |
| final Logger logger | |
| ) { |
OpenSearch/server/src/main/java/org/opensearch/gateway/BaseGatewayShardAllocator.java
Line 76 in 96630f0
| final AllocateUnassignedDecision allocateUnassignedDecision = makeAllocationDecision(shardRouting, allocation, logger); |
We need to add new classes similar to PSA & RSA for building decisions for a batch of shards instead of just one. As our fetching of data is moving to batch of shards. Once we have the metadata for those set of shards, we're in a position to build the allocation decisions for all those shards. New classes will return a map of <ShardRouting, AllocateUnassignedDecision>.
Additional context
Please go through the discussion #5098 to understand the overall enhancement approach and check #8098 to see the sub tasks of overall project.