[SOLR-4929] (ChaosMonkey)ShardSplitTest fails often on jenkins

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Shalin Shekhar Mangar
• Reporter: Shalin Shekhar Mangar
• Created: 15/Jun/13 16:26
• Updated: 18/Jun/13 16:16
• Resolved: 18/Jun/13 16:13

Description

ChaosMonkeyShardSplitTest and ShardSplitTest both fail on jenkins quite often with the same message always:

Error Message:
Server at http://127.0.0.1:20986 returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:20986 returned non ok status:500, message:Server Error
        at __randomizedtesting.SeedInfo.seed([7262B9B042D2C205:F38437A8358DA239]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
        at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
        at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
        at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

The logs show that the WaitForState action fails saying I am not the leader:

[junit4:junit4]   2> 1023092 T1943 oasc.SolrCore.registerSearcher [collection1_shard1_1_replica1] Registered new searcher Searcher@4afb5184 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1023093 T1944 oasc.SolrCore.registerSearcher [collection1_shard1_0_replica1] Registered new searcher Searcher@67d0de96 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1023095 T1939 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1023095 T1869 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1023095 T1939 oasc.CoreContainer.registerCore registering core: collection1_shard1_1_replica1
[junit4:junit4]   2> 1023096 T1869 oasc.CoreContainer.registerCore registering core: collection1_shard1_0_replica1
[junit4:junit4]   2> 1023096 T1939 oasc.ZkController.register Register replica - core:collection1_shard1_1_replica1 address:http://127.0.0.1:41605 collection:collection1 shard:shard1_1
[junit4:junit4]   2> 1023096 T1869 oasc.ZkController.register Register replica - core:collection1_shard1_0_replica1 address:http://127.0.0.1:41605 collection:collection1 shard:shard1_0
[junit4:junit4]   2> 1023097 T1939 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1_1/election
[junit4:junit4]   2> 1023098 T1869 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1_0/election
[junit4:junit4]   2> 1023129 T1939 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1023130 T1869 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1023147 T1871 oasc.SolrException.log ERROR org.apache.solr.common.SolrException: We are not the leader
[junit4:junit4]   2> 		at org.apache.solr.handler.admin.CoreAdminHandler.handleWaitForStateAction(CoreAdminHandler.java:914)
[junit4:junit4]   2> 		at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:190)
[junit4:junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
[junit4:junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:608)
[junit4:junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:206)

Issue Links

• duplicates: SOLR-4933 org.apache.solr.cloud.ShardSplitTest.testDistribSearch fails often with a 500 error.