Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3457

NPE when NodeManager.serviceInit fails and stopRecoveryStore called

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When NodeManager service init fails during stopRecoveryStore null pointer exception is thrown

       @Override
        protected void serviceInit(Configuration conf) throws Exception {
         ..
        try {
            exec.init();
          } catch (IOException e) {
            throw new YarnRuntimeException("Failed to initialize container executor", e);
          }    
      
      this.context = createNMContext(containerTokenSecretManager,
              nmTokenSecretManager, nmStore);
          
      

      context is null when service init fails

        private void stopRecoveryStore() throws IOException {
          nmStore.stop();
          if (context.getDecommissioned() && nmStore.canRecover()) {
         ..
          }
        }
      

      Null pointer exception thrown

      015-04-07 17:31:45,807 WARN org.apache.hadoop.service.AbstractService: When stopping the service NodeManager : java.lang.NullPointerException
      java.lang.NullPointerException
      at org.apache.hadoop.yarn.server.nodemanager.NodeManager.stopRecoveryStore(NodeManager.java:168)
      at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStop(NodeManager.java:280)
      at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
      at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
      at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
      at org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
      at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:484)
      at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:534)

      1. YARN-3457.001.patch
        2 kB
        Bibin A Chundatt

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12723815/YARN-3457.001.patch
          against trunk revision ab04ff9.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7252//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7252//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723815/YARN-3457.001.patch against trunk revision ab04ff9. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7252//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7252//console This message is automatically generated.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          +1, committing this shortly.

          Show
          ozawa Tsuyoshi Ozawa added a comment - +1, committing this shortly.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk and branch-2. Thanks Bibin A Chundatt for your contribution!

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Bibin A Chundatt for your contribution!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7531 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7531/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7531 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7531/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Hide
          bibinchundatt Bibin A Chundatt added a comment -

          Thank you Tsuyoshi Ozawa for checking and committing patch.

          Show
          bibinchundatt Bibin A Chundatt added a comment - Thank you Tsuyoshi Ozawa for checking and committing patch.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #157 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/157/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #157 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/157/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2089 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2089/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2089 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2089/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #148 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/148/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #148 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/148/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #891 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/891/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #891 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/891/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/158/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/158/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2107 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2107/)
          YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2107 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2107/ ) YARN-3457 . NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. (ozawa: rev dd852f5b8c8fe9e52d15987605f36b5b60f02701) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java hadoop-yarn-project/CHANGES.txt
          Hide
          jlowe Jason Lowe added a comment -

          I pulled this into branch-2.7 as well.

          Show
          jlowe Jason Lowe added a comment - I pulled this into branch-2.7 as well.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7822 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7822/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7822 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7822/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #927 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/927/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #927 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/927/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/195/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/195/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2125 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2125/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2125 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2125/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/185/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/185/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2143/)
          Update fix version for YARN-3457 and YARN-3537. (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2143 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2143/ ) Update fix version for YARN-3457 and YARN-3537 . (jlowe: rev f7de6198da030cb9cd62dcfd3c378dbbf857e017) hadoop-yarn-project/CHANGES.txt

            People

            • Assignee:
              bibinchundatt Bibin A Chundatt
              Reporter:
              bibinchundatt Bibin A Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development