Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14942

DistCp#cleanup() should check whether jobFS is null

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Over in HBASE-18975, we observed the following:

      2017-10-10 17:22:53,211 DEBUG [main] mapreduce.MapReduceBackupCopyJob(313): Doing COPY_TYPE_DISTCP
      2017-10-10 17:22:53,272 DEBUG [main] mapreduce.MapReduceBackupCopyJob(322): DistCp options: [hdfs://localhost:55247/backupUT/.tmp/backup_1507681285309, hdfs://localhost:55247/   backupUT]
      2017-10-10 17:22:53,283 ERROR [main] tools.DistCp(167): Exception encountered
      java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hbase.backup.mapreduce.MapReduceBackupCopyJob$BackupDistCp.execute(MapReduceBackupCopyJob.java:234)
        at org.apache.hadoop.tools.DistCp.run(DistCp.java:153)
        at org.apache.hadoop.hbase.backup.mapreduce.MapReduceBackupCopyJob.copy(MapReduceBackupCopyJob.java:331)
        at org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.incrementalCopyHFiles(IncrementalTableBackupClient.java:286)
      ...
      Caused by: java.lang.NullPointerException
        at org.apache.hadoop.tools.DistCp.cleanup(DistCp.java:460)
        ... 45 more
      

      NullPointerException came from second line below:

            if (metaFolder == null) return;
      
            jobFS.delete(metaFolder, true);
      

      in which case jobFS was null.
      A check against null should be added.

        Attachments

          Activity

            People

            • Assignee:
              boky01 Andras Bokor
              Reporter:
              yuzhihong@gmail.com Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: