[SOLR-13789] Solr core reload with config change upstream put solr instance in bad state

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 6.6.5, 8.1.1
• Component/s: replication (java)
• Labels: patch
• Assignee: Unassigned
• Reporter: Nilesh Singh
• Created: 24/Sep/19 15:45
• Updated: 21/Jan/20 16:59

Description

In a master/slave setup if the schema.xml is changed on master, during replication on slave solr instance, solr tries to reload the core with the new config.

During the reload operation solr creates a new searcher but it is not checking the status of the searcher and make this new core active.

Problem arises when the custom caches attached to the searcher has not been warmed up. In some scenarios custom caches might take some times to warm up in the background. 

 

This issue becomes critical during the schema changes in live solr instances, that makes solr to return bad responses until the searcher caches are fully ready.

Another issue arises due to reload and openNewSearcherAndUpdateCommitPoint methods by warming two searcher`s in parallel that is bad for cpu/memory.