Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6213

NullPointerException caused by job history server addr not resolvable

    Details

    • Hadoop Flags:
      Reviewed

      Description

      When DNS failed for a time, all MapReduce jobs which completed during that time got failed. Log as below:

      2015-01-08 01:26:44,968 ERROR [Thread-856] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Exception while unregistering 
      java.lang.NullPointerException
          at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithoutScheme(MRWebAppUtil.java:135)
          at org.apache.hadoop.mapreduce.v2.util.MRWebAppUtil.getApplicationWebURLOnJHSWithScheme(MRWebAppUtil.java:150)
          at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.doUnregistration(RMCommunicator.java:212)
          at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.unregister(RMCommunicator.java:182)
          at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStop(RMCommunicator.java:255)
          at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStop(RMContainerAllocator.java:257)
          at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
          at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
          at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStop(MRAppMaster.java:823)
          at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
          at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
          at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
          at org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
          at org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
          at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1471)
          at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
          at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1085)
          at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
          at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
      

        Activity

        Hide
        peng.zhang Peng Zhang added a comment -

        I found MAPREDUCE-5721 has similar error log. But this is not caused by wrong configuration but temporary DNS down. So I think this is a bug.

        Show
        peng.zhang Peng Zhang added a comment - I found MAPREDUCE-5721 has similar error log. But this is not caused by wrong configuration but temporary DNS down. So I think this is a bug.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12690745/MAPREDUCE-6213.patch
        against trunk revision a6ed489.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5100//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5100//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12690745/MAPREDUCE-6213.patch against trunk revision a6ed489. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5100//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5100//console This message is automatically generated.
        Hide
        qwertymaniac Harsh J added a comment -

        +1 for the change. Committing shortly. Thanks Peng!

        Show
        qwertymaniac Harsh J added a comment - +1 for the change. Committing shortly. Thanks Peng!
        Hide
        qwertymaniac Harsh J added a comment -

        Committed to branch-2 and trunk.

        Show
        qwertymaniac Harsh J added a comment - Committed to branch-2 and trunk.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7394 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7394/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7394 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7394/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #140 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/140/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #140 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/140/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #874 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/874/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #874 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/874/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2072 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2072/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2072 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2072/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #131 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/131/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #131 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/131/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #140 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/140/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #140 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/140/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2090 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2090/)
        MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2090 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2090/ ) MAPREDUCE-6213 . NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. (harsh: rev e1e09052e861926112493d6041aae01ab594b547) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRWebAppUtil.java hadoop-mapreduce-project/CHANGES.txt

          People

          • Assignee:
            peng.zhang Peng Zhang
            Reporter:
            peng.zhang Peng Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development