Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Bug
-
2.6.0
-
None
-
None
-
CDH5.12
hadoop2.6
Description
I configure the max user processes 65535 with any user ,and the datanode memory is 8G.
When a log of data was been writeen,the datanode was been shutdown.
But I can see the memory use only < 1000M.
Please to see the attachment.
DataNode shutdown error log:
2017-12-17 23:58:14,422 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-1437036909-192.168.17.36-1509097205664:blk_1074725940_987917, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 2017-12-17 23:58:31,425 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DataNode is out of memory. Will retry in 30 seconds. java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:154) at java.lang.Thread.run(Thread.java:745) 2017-12-17 23:59:01,426 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DataNode is out of memory. Will retry in 30 seconds. java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:154) at java.lang.Thread.run(Thread.java:745) 2017-12-17 23:59:05,520 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DataNode is out of memory. Will retry in 30 seconds. java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:154) at java.lang.Thread.run(Thread.java:745) 2017-12-17 23:59:31,429 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-1437036909-192.168.17.36-1509097205664:blk_1074725951_987928 src: /192.168.17.54:40478 dest: /192.168.17.48:50010