[SOLR-16438] Shard split should be able to set preferred leaders on other replicas

• Type: Improvement
• Status: Reopened
• Priority: Major
• Resolution: Abandoned
• Assignee: Bruno Roustant
• Reporter: Bruno Roustant
• Created: 29/Sep/22 14:55
• Updated: 09/Oct/24 01:28
• Resolved: 03/Mar/23 10:07

Description

Currently, shard split always create a first replica for each sub-shard on the current host. Then it creates other replicas and their corresponding sub-shards are in RECOVERY state. The effect is that the first replica (on the current host) is always the leader, meaning that if the sub-shards are split themselves, their sub-sub-shards leaders are also on the same host.
This can lead to very unbalanced situation where the same host is the leader for a whole set of shards.

A solution to distribute evenly the leaders is to flag some other replicas with the preferredLeader property during the split. Then a rebalance-leaders command can elect the appropriate leaders. If we do that for each split, then all the sub-shards have their leaders correctly balanced.

To go further, we can improve CollectionsHandler#CollectionOperation to support combined operations. That way a CollectionOperation#SPLITSHARD_OP can trigger a split op, then a wait for split completion op, and then a rebalance leaders op.

Issue Links

• links to: GitHub Pull Request #1048