[SOLR-5509] ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.7, 6.0
• Component/s: SolrCloud
• Assignee: Mark Miller
• Reporter: Mark Miller
• Created: 27/Nov/13 18:41
• Updated: 09/May/16 18:44
• Resolved: 24/Feb/14 16:22

Description

   [junit4]   2> 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&CONTROL=TRUE&version=2} {add=[50086 (1452880907553734656)]} 0 142

   [junit4]   2> 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50086 (1452880908206997504)]} 0 254

   [junit4]   2> 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&CONTROL=TRUE&version=2} {delete=[50086 (-1452880908537298944)]} 0 2

   [junit4]   2> 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[50086 (-1452880908542541824)]} 0 1

   [junit4]   2> 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {add=[50086 (1452880908850823168)]} 0 1

   [junit4]   2> 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50086]} 0 1223

   [junit4]   2> ###### Only in cloudDocList: [{id=50086}]

   [junit4]   2> 		cloudClient :{numFound=1,start=0,docs=[SolrDocument{id=50086, _version_=1452880908850823168}]}

h

Issue Links

• breaks: SOLR-6983 SocketExceptions no longer trigger retries when processing distributed updates
• depends upon: SOLR-5727 LBHttpSolrServer should only retry on Connection exceptions when sending updates.
• is related to: SOLR-6931 We should do a limited retry when using HttpClient.
• relates to: SOLR-5650 When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.
• relates to: SOLR-5503 Retry 'forward to leader' requests less aggressively - rather than on IOException, ConnectException.
• relates to: SOLR-5504 We need better testing for SolrCmdDistributor retry logic.