[SOLR-6213] StackOverflowException in Solr cloud's leader election

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 4.10, 6.0
• Component/s: SolrCloud
• Assignee: Unassigned
• Reporter: Dawid Weiss
• Created: 30/Jun/14 06:52
• Updated: 02/Nov/23 17:34

Description

This is what's causing test hangs (at least on FreeBSD, LUCENE-5786), possibly on other machines too. The problem is stack overflow from looped calls in:

  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)
  > org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:212)
  > org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
  > org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:313)
  > org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
  > org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:448)

These routines attempt to log information to loggers, which in turn attempts to serialize messages back to the master (test process). When the stack is exhausted the serialization process fails and breaks the communication with the master test node.

Issue Links

• is duplicated by: SOLR-5692 StackOverflowError during SolrCloud leader election process
• is duplicated by: SOLR-6285 StackOverflowError in SolrCloud test on jenkins