[SOLR-6213] StackOverflowException in Solr cloud's leader election - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Bulk Copy Attachments

Bulk Move Attachments

Add vote

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Critical
Resolution: Unresolved
Affects Version/s: 4.10, 6.0
Fix Version/s: None
Component/s: SolrCloud
Labels:
None

Description

This is what's causing test hangs (at least on FreeBSD, ~~LUCENE-5786~~), possibly on other machines too. The problem is stack overflow from looped calls in:

  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)

These routines attempt to log information to loggers, which in turn attempts to serialize messages back to the master (test process). When the stack is exhausted the serialization process fails and breaks the communication with the master test node.