[HDFS-9684] DataNode stopped sending heartbeat after getting OutOfMemoryError form DataTransfer thread.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 2.7.1
• Component/s: datanode
• Assignee: Surendra Singh Lilhore
• Reporter: Surendra Singh Lilhore
• Created: 22/Jan/16 11:56
• Updated: 19/Apr/16 02:41
• Resolved: 29/Jan/16 18:46

Description

java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:714)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.transferBlock(DataNode.java:1999)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.transferBlocks(DataNode.java:2008)
	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:657)
	at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:615)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:857)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:671)
	at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:823)
	at java.lang.Thread.run(Thread.java:745)

Issue Links

• duplicates: HDFS-9046 Any Error during BPOfferService run can leads to Missing DN.
• is related to: HDFS-2182 Exceptions in DataXceiver#run can result in a zombie datanode
• is related to: HDFS-2452 OutOfMemoryError in DataXceiverServer takes down the DataNode