[SOLR-7569] Create an API to force a leader election between nodes

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.4, 6.0
• Component/s: SolrCloud
• Labels: difficulty-medium impact-high
• Assignee: Noble Paul
• Reporter: Shalin Shekhar Mangar
• Created: 19/May/15 17:19
• Updated: 02/Oct/19 17:25
• Resolved: 17/Nov/15 18:38

Description

There are many reasons why Solr will not elect a leader for a shard e.g. all replicas' last published state was recovery or due to bugs which cause a leader to be marked as 'down'. While the best solution is that they never get into this state, we need a manual way to fix this when it does get into this state. Right now we can do a series of dance involving bouncing the node (since recovery paths between bouncing and REQUESTRECOVERY are different), but that is difficult when running a large cluster. Although it is possible that such a manual API may lead to some data loss but in some cases, it is the only possible option to restore availability.

This issue proposes to build a new collection API which can be used to force replicas into recovering a leader while avoiding data loss on a best effort basis.

Issue Links

• contains: SOLR-8233 Core admin API for overriding last published state
• depends upon: SOLR-7989 After a new leader is elected it, it should ensure it's state is ACTIVE if it has already registered with ZK.
• relates to: SOLR-9305 ForceLeaderTest.testReplicasInLIRNoLeader(): connection refused
• relates to: SOLR-5991 SolrCloud: Add API to move leader off a Solr instance