[SOLR-10006] Cannot do a full sync (fetchindex) if the replica can't open a searcher

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.3.1, 6.4
• Assignee: Unassigned
• Reporter: Erick Erickson
• Created: 19/Jan/17 22:18
• Updated: 08/Jun/19 15:33

Description

Doing a full sync or fetchindex requires an open searcher and if you can't open the searcher those operations fail.

For discussion. I've seen a situation in the field where a replica's index became corrupt. When the node was restarted, the replica tried to do a full sync but fails because the core can't open a searcher. The replica went into an endless sync/fail/sync cycle.

I couldn't reproduce that exact scenario, but it's easy enough to get into a similar situation. Create a 2x2 collection and index some docs. Then stop one of the instances and go in and remove a couple of segments files and restart.

The replica stays in the "down" state, fine so far.

Manually issue a fetchindex. That fails because the replica can't open a searcher. Sure, issuing a fetchindex is abusive.... but I think it's the same underlying issue: why should we care about the state of a replica's current index when we're going to completely replace it anyway?

Issue Links

• depends upon: LUCENE-7662 Index with missing files should throw CorruptIndexException
• is related to: SOLR-9836 Add more graceful recovery steps when failing to create SolrCore
• is related to: SOLR-10259 admin/cores?action=STATUS returns 500 when a single core has init failures