[HDFS-5503] Datanode#checkDiskError also should check for ClosedChannelException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Vinayakumar B
• Reporter: Vinayakumar B
• Created: 12/Nov/13 08:45
• Updated: 15/Aug/14 05:26
• Resolved: 16/Jul/14 07:56

Description

out file
==============

Exception in thread "PacketResponder: BP-52063768-xxxxxxxxx-1383447451718:blk_1073755206_1099511661730, type=LAST_IN_PIPELINE, downstreams=0:[]" java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.datanode.DataNode.checkDiskError(DataNode.java:1363)
        at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1233)
        at java.lang.Thread.run(Thread.java:662)

log file
=======

2013-11-08 21:23:36,622 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: IOException in BlockReceiver.run():
java.nio.channels.ClosedChannelException
        at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
        at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63)
        at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
        at java.io.DataOutputStream.flush(DataOutputStream.java:106)
        at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1212)
        at java.lang.Thread.run(Thread.java:662)
2013-11-08 21:23:36,622 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: checkDiskError: exception:
java.nio.channels.ClosedChannelException
        at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
        at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63)
        at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159)
        at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
        at java.io.DataOutputStream.flush(DataOutputStream.java:106)
        at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1212)
        at java.lang.Thread.run(Thread.java:662)

Issue Links

• is part of: HDFS-5522 Datanode disk error check may be incorrectly skipped
• relates to: HDFS-5745 Unnecessary disk check triggered when socket operation has problem.