[HDFS-10340] data node sudden killed

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Invalid
• Affects Version/s: 2.6.0
• Component/s: datanode

Environment

Ubuntu 16.04 LTS , RAM 16g , cpu core : 8 , hdd 100gb, hadoop 2.6.0

• Assignee: Unassigned
• Reporter: tu nguyen khac
• Created: 28/Apr/16 07:40
• Updated: 24/Aug/16 02:52
• Resolved: 28/Apr/16 14:51

Description

I tried to setup a new data node using ubuntu 16
and get it join to an existed Hadoop Hdfs cluster ( there are 10 nodes in this cluster and they all run on centos Os 6 )
But when i try to boostrap this node , after about 10 or 20 minutes i get this strange errors :

2016-04-26 20:12:09,394 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.3.24.65:55323, dest: /10.3.24.197:50010, bytes: 79902, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1379996362_1, offset: 0, srvID: 225f5b43-1dd3-4ac6-88d2-1e8d27dba55b, blockid: BP-352432948-10.3.24.65-1433821675295:blk_1074038505_789832, duration: 15331628
2016-04-26 20:12:09,394 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder: BP-352432948-10.3.24.65-1433821675295:blk_1074038505_789832, type=LAST_IN_PIPELINE, downstreams=0:[] terminating
2016-04-26 20:12:25,410 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification succeeded for BP-352432948-10.3.24.65-1433821675295:blk_1074038502_789829
2016-04-26 20:12:25,411 INFO org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification succeeded for BP-352432948-10.3.24.65-1433821675295:blk_1074038505_789832
2016-04-26 20:13:18,546 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Scheduling blk_1074038502_789829 file /data/hadoop_data/backup/data/current/BP-352432948-10.3.24.65-1433821675295/current/finalized/subdir4/subdir134/blk_1074038502 for deletion
2016-04-26 20:13:18,562 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-352432948-10.3.24.65-1433821675295 blk_1074038502_789829 file /data/hadoop_data/backup/data/current/BP-352432948-10.3.24.65-1433821675295/current/finalized/subdir4/subdir134/blk_1074038502
2016-04-26 20:15:46,481 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: RECEIVED SIGNAL 15: SIGTERM
2016-04-26 20:15:46,504 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at bigdata-dw-24-197/10.3.24.197
************************************************************/