Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6581 Write to single replica in memory
  3. HDFS-7066

LazyWriter#evictBlocks misses a null check for replicaState

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: HDFS-6581
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      LazyWriter#evictBlocks (added for HDFS-6581) misses a null check for replicaState. As a result, there are many NPEs in the debug log under certain conditions.

      2014-09-15 14:27:10,820 DEBUG impl.FsDatasetImpl (FsDatasetImpl.java:evictBlocks(2335)) - Evicting block null
      2014-09-15 14:27:10,821 WARN  impl.FsDatasetImpl (FsDatasetImpl.java:run(2409)) - Ignoring exception in LazyWriter:
      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.evictBlocks(FsDatasetImpl.java:2343)
      	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl$LazyWriter.run(FsDatasetImpl.java:2396)
      	at java.lang.Thread.run(Thread.java:745)
      

      The proposed fix is to break if there is no candidate available to evict.

      	while (iterations++ < MAX_BLOCK_EVICTIONS_PER_ITERATION &&
                  transientFreeSpaceBelowThreshold()) {
                 LazyWriteReplicaTracker.ReplicaState replicaState =
                     lazyWriteReplicaTracker.getNextCandidateForEviction();
      
                 ++++++++++++++++++++
      	    if (replicaState == null) {
                    break;
                 }
                 ++++++++++++++++++++
      
                 if (LOG.isDebugEnabled()) {
                   LOG.debug("Evicting block " + replicaState);
                 }
      
      

        Attachments

        1. HDFS-7066.0.patch
          0.9 kB
          Xiaoyu Yao

          Activity

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              xyao Xiaoyu Yao
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: