[SPARK-15943] Spark driver hangs up periodically (cannot receive any reply in 120 seconds)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.6.0
• Component/s: Spark Core
• Labels: spark

Environment

Ubuntu Server 14.04
Spark 1.6.0
Hadoop 2.6.0

• Assignee: Unassigned
• Reporter: Sergey Zhemzhitsky
• Created: 14/Jun/16 13:58
• Updated: 14/Jun/16 16:42
• Resolved: 14/Jun/16 16:42

Description

Spark driver hangs up periodically with the following stacktrace on driver driver-stacktrace.log and threaddump on the executor that causes this issue executor-threaddump.log

The attached executor-threaddump.log has been retrieved by means of spark user interface. Trying to dump the threads by means of jstack -l -F <pid> allowed us to retrieve only the following stacktrace executor-threaddump-jstack.log

Issue Links

• duplicates: SPARK-13566 Deadlock between MemoryStore and BlockManager