Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9190

VolumeScanner throwing NPE while scanning suspect block.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: datanode
    • Labels:
      None

      Description

      Volume scanner NPEs while scanning suspect Block.
      Following is the stack trace:

      2015-10-02 06:45:30,333 [VolumeScannerThread(dataDir)] ERROR datanode.VolumeScanner: VolumeScanner(dataDir, DS-5fc4263e-7a5c-4463-9f82-842108c0ab3b) exiting because of exception 
      java.lang.NullPointerException
              at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:539)
              at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:619)
      2015-10-02 06:45:30,333 [org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer@7768ca5] WARN datanode.DataNode: DatanodeRegistration(sourceDN:1004, datanodeUuid=f554982f-7c45-4fd4-ad57-9d472a39729e, infoPort=1006, infoSecurePort=0, ipcPort=8020, storageInfo=lv=-56;cid=CID-ddc217ab-5203-48ef-9695-a348feb4dac2;nsid=1872110141;c=1443758672580):Failed to transfer BP-1749317823-<nnIp>-1443758669533:blk_1073742231_1407 to destDN:1004 got 
      java.net.SocketException: Original Exception : java.io.IOException: Connection reset by peer
              at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
              at sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:443)
              at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:575)
              at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:223)
              at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:579)
              at org.apache.hadoop.hdfs.server.datanode.BlockSender.doSendBlock(BlockSender.java:759)
              at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:706)
              at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:2124)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.io.IOException: Connection reset by peer
              ... 9 more
      

      It is NPEing at the code in file: VolumeScanner#runLoop

            long saveDelta = monotonicMs - curBlockIter.getLastSavedMs();
      

      curBlockIter is not initialized.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                shahrs87 Rushabh S Shah
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: