Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4355

NPE while processing localizer heartbeat

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While analyzing YARN-4354 I noticed a nodemanager was getting NPEs while processing a private localizer heartbeat. I think there's a race where we can cleanup resources for an application and therefore remove the app local resource tracker just as we are trying to handle the localizer heartbeat.

      1. YARN-4355.01.patch
        12 kB
        Varun Saxena
      2. YARN-4355.02.patch
        11 kB
        Jonathan Hung
      3. YARN-4355.03.patch
        11 kB
        Varun Saxena
      4. YARN-4355.04.patch
        12 kB
        Varun Saxena
      5. YARN-4355.05.patch
        12 kB
        Varun Saxena
      6. YARN-4355-branch-2.7.001.patch
        11 kB
        Jonathan Hung
      7. YARN-4355-branch-2.7.002.patch
        11 kB
        Jonathan Hung

        Issue Links

          Activity

          Hide
          jlowe Jason Lowe added a comment -

          Stacktrace:

          java.lang.NullPointerException
                  at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.getPathForLocalization(ResourceLocalizationService.java:1089)
                  at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.processHeartbeat(ResourceLocalizationService.java:1054)
                  at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:681)
                  at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:330)
                  at org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:48)
                  at org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:63)
                  at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
                  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
                  at org.apache.hadoop.ipc.Server.call(Server.java:2297)
                  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:654)
                  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:621)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at javax.security.auth.Subject.doAs(Subject.java:422)
                  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1680)
                  at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2247)
          

          The nodemanager was in the process of tearing down, so applications where being cleaned up. Looks like localizer heartbeats can come in and we can lose the localizer tracker just as the localizer heartbeat tries to use it.

          Show
          jlowe Jason Lowe added a comment - Stacktrace: java.lang.NullPointerException at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.getPathForLocalization(ResourceLocalizationService.java:1089) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.processHeartbeat(ResourceLocalizationService.java:1054) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:681) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:330) at org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:48) at org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:63) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) at org.apache.hadoop.ipc.Server.call(Server.java:2297) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:654) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:621) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1680) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2247) The nodemanager was in the process of tearing down, so applications where being cleaned up. Looks like localizer heartbeats can come in and we can lose the localizer tracker just as the localizer heartbeat tries to use it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 39s trunk passed
          +1 compile 0m 26s trunk passed with JDK v1.8.0_66
          +1 compile 0m 28s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 12s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 55s trunk passed
          +1 javadoc 0m 17s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 23s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 21s the patch passed with JDK v1.8.0_66
          +1 javac 0m 21s the patch passed
          +1 compile 0m 25s the patch passed with JDK v1.7.0_91
          +1 javac 0m 25s the patch passed
          -1 checkstyle 0m 12s Patch generated 3 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager (total was 152, now 149).
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 59s the patch passed
          +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 20s the patch passed with JDK v1.7.0_91
          +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
          +1 unit 9m 6s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          33m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch
          JIRA Issue YARN-4355
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 363a76d8f023 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0e76f1f
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10221/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10221/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10221/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 39s trunk passed +1 compile 0m 26s trunk passed with JDK v1.8.0_66 +1 compile 0m 28s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 55s trunk passed +1 javadoc 0m 17s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 23s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 24s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_66 +1 javac 0m 21s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_91 +1 javac 0m 25s the patch passed -1 checkstyle 0m 12s Patch generated 3 new checkstyle issues in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager (total was 152, now 149). +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 59s the patch passed +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 20s the patch passed with JDK v1.7.0_91 +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 9m 6s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 33m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch JIRA Issue YARN-4355 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 363a76d8f023 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0e76f1f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10221/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10221/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10221/console This message was automatically generated.
          Hide
          varun_saxena Varun Saxena added a comment -

          Kindly review. The checkstyle issues are due to the surrounding statements within switch case. If I change the indentation of the line modified by me, I will have to change several lines before and after it, which have no direct connection with the change required for the patch. So leaving it as it is.

          Show
          varun_saxena Varun Saxena added a comment - Kindly review. The checkstyle issues are due to the surrounding statements within switch case. If I change the indentation of the line modified by me, I will have to change several lines before and after it, which have no direct connection with the change required for the patch. So leaving it as it is.
          Hide
          varun_saxena Varun Saxena added a comment -

          In the patch I have introduced a null check wherever required. On next HB, nodemanager will issue a DIE.
          Did not make the methods or code blocks synchronized to avoid race because this should happen only in rare scenarios.

          Show
          varun_saxena Varun Saxena added a comment - In the patch I have introduced a null check wherever required. On next HB, nodemanager will issue a DIE. Did not make the methods or code blocks synchronized to avoid race because this should happen only in rare scenarios.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 38s trunk passed
          +1 compile 0m 24s trunk passed with JDK v1.8.0_91
          +1 compile 0m 26s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 51s trunk passed
          +1 javadoc 0m 18s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 22s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 21s the patch passed with JDK v1.8.0_91
          +1 javac 0m 21s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.7.0_95
          +1 javac 0m 24s the patch passed
          -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: patch generated 6 new + 236 unchanged - 6 fixed = 242 total (was 242)
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 1s the patch passed
          +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 19s the patch passed with JDK v1.7.0_95
          +1 unit 11m 6s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91.
          +1 unit 11m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          37m 45s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch
          JIRA Issue YARN-4355
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1c78acbbfb7c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 47c41e7
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11375/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11375/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11375/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 38s trunk passed +1 compile 0m 24s trunk passed with JDK v1.8.0_91 +1 compile 0m 26s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 51s trunk passed +1 javadoc 0m 18s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 22s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 24s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_91 +1 javac 0m 21s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_95 +1 javac 0m 24s the patch passed -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: patch generated 6 new + 236 unchanged - 6 fixed = 242 total (was 242) +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 1s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_95 +1 unit 11m 6s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. +1 unit 11m 36s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 37m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch JIRA Issue YARN-4355 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1c78acbbfb7c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 47c41e7 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11375/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11375/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11375/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 25s trunk passed
          +1 compile 0m 38s trunk passed with JDK v1.8.0_91
          +1 compile 0m 36s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 39s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          +1 findbugs 1m 6s trunk passed
          +1 javadoc 0m 33s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 35s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.8.0_91
          +1 javac 0m 40s the patch passed
          +1 compile 0m 34s the patch passed with JDK v1.7.0_95
          +1 javac 0m 34s the patch passed
          -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: patch generated 6 new + 237 unchanged - 6 fixed = 243 total (was 243)
          +1 mvnsite 0m 35s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 21s the patch passed
          +1 javadoc 0m 27s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_95
          +1 unit 13m 18s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91.
          +1 unit 12m 54s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          47m 37s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch
          JIRA Issue YARN-4355
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux daa4b9d40852 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 47c41e7
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11376/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11376/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11376/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 25s trunk passed +1 compile 0m 38s trunk passed with JDK v1.8.0_91 +1 compile 0m 36s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 33s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 35s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_91 +1 javac 0m 40s the patch passed +1 compile 0m 34s the patch passed with JDK v1.7.0_95 +1 javac 0m 34s the patch passed -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: patch generated 6 new + 237 unchanged - 6 fixed = 243 total (was 243) +1 mvnsite 0m 35s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 21s the patch passed +1 javadoc 0m 27s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 26s the patch passed with JDK v1.7.0_95 +1 unit 13m 18s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_91. +1 unit 12m 54s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 47m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch JIRA Issue YARN-4355 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux daa4b9d40852 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 47c41e7 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11376/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11376/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11376/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Two meta-comments:

          • Should the log messages really be WARN? What will the admin do about it?
          • Can we write out heartbeat instead of HB? Even knowing what we're talking about, it took me an embarrassing amount of puzzling to figure out what HB stands for. History....???
          Show
          templedf Daniel Templeton added a comment - Two meta-comments: Should the log messages really be WARN? What will the admin do about it? Can we write out heartbeat instead of HB? Even knowing what we're talking about, it took me an embarrassing amount of puzzling to figure out what HB stands for. History....???
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          2.7.3 is under release process, changing target-version to 2.7.4.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - 2.7.3 is under release process, changing target-version to 2.7.4.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 4s YARN-4355 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch
          JIRA Issue YARN-4355
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12820/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-4355 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781441/YARN-4355.01.patch JIRA Issue YARN-4355 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12820/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Latest patch doesnt seem to compile, can you rebase the patch to make some progress in this jira

          Show
          Naganarasimha Naganarasimha G R added a comment - Latest patch doesnt seem to compile, can you rebase the patch to make some progress in this jira
          Hide
          jhung Jonathan Hung added a comment - - edited

          The initial patch looks good to me.

          Naganarasimha G R, Daniel Templeton, I rebased and uploaded a patch based on Varun Saxena's patch (conflict was due to YARN-1942) and addressed the comments, can you please take a look? Thanks!

          Also it seems if this race condition is hit, the localization service on that node will die which seems like a pretty serious issue? Let me know if I understood correctly.

          Show
          jhung Jonathan Hung added a comment - - edited The initial patch looks good to me. Naganarasimha G R , Daniel Templeton , I rebased and uploaded a patch based on Varun Saxena 's patch (conflict was due to YARN-1942 ) and addressed the comments, can you please take a look? Thanks! Also it seems if this race condition is hit, the localization service on that node will die which seems like a pretty serious issue? Let me know if I understood correctly.
          Hide
          varun_saxena Varun Saxena added a comment -

          Thanks Jonathan Hung for rebasing the patch.
          I am so sorry I missed comments made by Naga and Daniel.

          Naganarasimha G R / Daniel Templeton, can either one of you commit it if latest patch looks good ?

          Show
          varun_saxena Varun Saxena added a comment - Thanks Jonathan Hung for rebasing the patch. I am so sorry I missed comments made by Naga and Daniel. Naganarasimha G R / Daniel Templeton , can either one of you commit it if latest patch looks good ?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Varun Saxena & Jonathan Hung for working on the patch, At initial glance patch looks to be fine, let me have a final look at the patch(over the weekend) and commit it.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Varun Saxena & Jonathan Hung for working on the patch, At initial glance patch looks to be fine, let me have a final look at the patch(over the weekend) and commit it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 24s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 32s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 53s trunk passed
          +1 javadoc 0m 18s trunk passed
          +1 mvninstall 0m 28s the patch passed
          +1 compile 0m 30s the patch passed
          -1 javac 0m 30s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16)
          -0 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 7 new + 232 unchanged - 6 fixed = 239 total (was 238)
          +1 mvnsite 0m 28s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 54s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 15m 4s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          30m 6s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836988/YARN-4355.02.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5384a4798cdf 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 69dd5fa
          Default Java 1.8.0_111
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13782/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13782/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13782/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13782/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 24s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 53s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 28s the patch passed +1 compile 0m 30s the patch passed -1 javac 0m 30s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16) -0 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 7 new + 232 unchanged - 6 fixed = 239 total (was 238) +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 15m 4s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 30m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836988/YARN-4355.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5384a4798cdf 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 69dd5fa Default Java 1.8.0_111 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-YARN-Build/13782/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13782/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13782/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13782/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          varun_saxena Varun Saxena added a comment - - edited

          Fixed checkstyle issues reported in QA report above. 3 checkstyle issues cant be fixed as the surrounding statements will have to be changed for it which is unnecessary.

          Show
          varun_saxena Varun Saxena added a comment - - edited Fixed checkstyle issues reported in QA report above. 3 checkstyle issues cant be fixed as the surrounding statements will have to be changed for it which is unnecessary.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 25s trunk passed
          +1 compile 0m 27s trunk passed
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 32s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 45s trunk passed
          +1 javadoc 0m 18s trunk passed
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 26s the patch passed
          +1 javac 0m 26s the patch passed
          -0 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 233 unchanged - 5 fixed = 236 total (was 238)
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 56s the patch passed
          +1 javadoc 0m 15s the patch passed
          +1 unit 15m 9s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          31m 12s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837125/YARN-4355.03.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 80a007fdf344 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 19b3779
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13784/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13784/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13784/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 25s trunk passed +1 compile 0m 27s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 45s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 26s the patch passed +1 javac 0m 26s the patch passed -0 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 233 unchanged - 5 fixed = 236 total (was 238) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 56s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 15m 9s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 31m 12s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837125/YARN-4355.03.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 80a007fdf344 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 19b3779 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13784/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13784/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13784/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          templedf Daniel Templeton added a comment -

          Two minor comments on the test:

          • You use both assertEquals() and Assert.fail(). It would be better to be consistent about using static imports or not.
          • I always prefer having a message in the assert that explains what's actually failing and what that implies.
          Show
          templedf Daniel Templeton added a comment - Two minor comments on the test: You use both assertEquals() and Assert.fail() . It would be better to be consistent about using static imports or not. I always prefer having a message in the assert that explains what's actually failing and what that implies.
          Hide
          varun_saxena Varun Saxena added a comment -

          Attaching a patch after addressing Daniel Templeton's comments

          Show
          varun_saxena Varun Saxena added a comment - Attaching a patch after addressing Daniel Templeton 's comments
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 8s trunk passed
          +1 compile 0m 27s trunk passed
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 26s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 42s trunk passed
          +1 javadoc 0m 17s trunk passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          -0 checkstyle 0m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 233 unchanged - 6 fixed = 236 total (was 239)
          +1 mvnsite 0m 25s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 45s the patch passed
          +1 javadoc 0m 14s the patch passed
          -1 unit 15m 42s hadoop-yarn-server-nodemanager in the patch failed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          29m 41s



          Reason Tests
          Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837630/YARN-4355.04.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 9513ecc567e4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d8bab3d
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13797/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13797/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13797/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13797/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 8s trunk passed +1 compile 0m 27s trunk passed +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 42s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed -0 checkstyle 0m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 233 unchanged - 6 fixed = 236 total (was 239) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 45s the patch passed +1 javadoc 0m 14s the patch passed -1 unit 15m 42s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 29m 41s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837630/YARN-4355.04.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9513ecc567e4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d8bab3d Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13797/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13797/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13797/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13797/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Varun Saxena for the patch,
          I have some doubts for avoiding race @ ResourceLocalizationService, ln no 1119.

          1. If cleanup has happened then are there chances of having pending LocalizerResourceRequestEvent in LocalizerRunner ?
          2. Also cant we pass the same tracker instance to getPathForLocalization from LocalizerRunner.processHeartbeat ?

          Few trivial nits:

          1. TestResourceLocalizationService, ln no 1543 : no need to create a new array list when we are using Arrays.asList(req1, req2)
          2. TestResourceLocalizationService, ln no 1581 : may be we can join the line below line as it doesn't exceed 80 chars.
          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Varun Saxena for the patch, I have some doubts for avoiding race @ ResourceLocalizationService, ln no 1119. If cleanup has happened then are there chances of having pending LocalizerResourceRequestEvent in LocalizerRunner ? Also cant we pass the same tracker instance to getPathForLocalization from LocalizerRunner.processHeartbeat ? Few trivial nits: TestResourceLocalizationService, ln no 1543 : no need to create a new array list when we are using Arrays.asList(req1, req2) TestResourceLocalizationService, ln no 1581 : may be we can join the line below line as it doesn't exceed 80 chars.
          Hide
          varun_saxena Varun Saxena added a comment -

          Also cant we pass the same tracker instance to getPathForLocalization from LocalizerRunner.processHeartbeat ?

          That is what is being done. Previously tracker was fetched inside getPathForLocalization. Do you mean something else ?

          1119	          LocalResourcesTracker tracker = getLocalResourcesTracker(
          1120	              next.getVisibility(), user, applicationId);
          1121	          if (tracker != null) {
          1122	            ResourceLocalizationSpec resource =
          1123	                NodeManagerBuilderUtils.newResourceLocalizationSpec(next,
          1124	                getPathForLocalization(next, tracker));
          1125	            rsrcs.add(resource);
          1126	          }
          

          If cleanup has happened then are there chances of having pending LocalizerResourceRequestEvent in LocalizerRunner ?

          It can even if rarely. This section of code isn't really synchronized. So event processing for cleaning up container resources and destroying application resources can happen before localizer HB is fully processed. Localizer will only DIE if it cannot find the localizer in list of localizers which is removed when container is cleaned up. But it is possible that HB carries on for processing if it finds the localizer but later does not find the tracker as application resources are later destroyed before HB is fully processed due to the corresponding sections are not guarded by lock. Evidence of this is the NPE reported in JIRA which came on a real cluster. This NPE came when NM was shutting down so all the apps on the NM were being cleaned up as well. So yes there can be chances of having pending events. And anyways having a null check is not a bad thing to do.
          Why did I not synchronize this section of code as a solution. Well the reason was that the possibility of this race happening is very rare.

          Show
          varun_saxena Varun Saxena added a comment - Also cant we pass the same tracker instance to getPathForLocalization from LocalizerRunner.processHeartbeat ? That is what is being done. Previously tracker was fetched inside getPathForLocalization. Do you mean something else ? 1119 LocalResourcesTracker tracker = getLocalResourcesTracker( 1120 next.getVisibility(), user, applicationId); 1121 if (tracker != null ) { 1122 ResourceLocalizationSpec resource = 1123 NodeManagerBuilderUtils.newResourceLocalizationSpec(next, 1124 getPathForLocalization(next, tracker)); 1125 rsrcs.add(resource); 1126 } If cleanup has happened then are there chances of having pending LocalizerResourceRequestEvent in LocalizerRunner ? It can even if rarely. This section of code isn't really synchronized. So event processing for cleaning up container resources and destroying application resources can happen before localizer HB is fully processed. Localizer will only DIE if it cannot find the localizer in list of localizers which is removed when container is cleaned up. But it is possible that HB carries on for processing if it finds the localizer but later does not find the tracker as application resources are later destroyed before HB is fully processed due to the corresponding sections are not guarded by lock. Evidence of this is the NPE reported in JIRA which came on a real cluster. This NPE came when NM was shutting down so all the apps on the NM were being cleaned up as well. So yes there can be chances of having pending events. And anyways having a null check is not a bad thing to do. Why did I not synchronize this section of code as a solution. Well the reason was that the possibility of this race happening is very rare.
          Hide
          varun_saxena Varun Saxena added a comment -

          Attaching a patch fixing 2 nits pointed out by Naga above.

          Show
          varun_saxena Varun Saxena added a comment - Attaching a patch fixing 2 nits pointed out by Naga above.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 49s trunk passed
          +1 compile 0m 27s trunk passed
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 26s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 40s trunk passed
          +1 javadoc 0m 17s trunk passed
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 27s the patch passed
          +1 javac 0m 27s the patch passed
          -0 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 232 unchanged - 6 fixed = 235 total (was 238)
          +1 mvnsite 0m 29s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 55s the patch passed
          +1 javadoc 0m 18s the patch passed
          -1 unit 14m 51s hadoop-yarn-server-nodemanager in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          29m 0s



          Reason Tests
          Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e809691
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838034/YARN-4355.05.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7dd455c2d9bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dbb133c
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13829/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13829/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13829/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13829/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 49s trunk passed +1 compile 0m 27s trunk passed +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 40s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 27s the patch passed +1 javac 0m 27s the patch passed -0 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 232 unchanged - 6 fixed = 235 total (was 238) +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 55s the patch passed +1 javadoc 0m 18s the patch passed -1 unit 14m 51s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 29m 0s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838034/YARN-4355.05.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7dd455c2d9bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dbb133c Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13829/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13829/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13829/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13829/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          +1,
          Thanks for the explanation Varun Saxena, went through the flow again and yes the fixes are proper, IMO its good enough to be committed, will wait for a day more so that Jason Lowe or others has a chance to review.

          Show
          Naganarasimha Naganarasimha G R added a comment - +1, Thanks for the explanation Varun Saxena , went through the flow again and yes the fixes are proper, IMO its good enough to be committed, will wait for a day more so that Jason Lowe or others has a chance to review.
          Hide
          jhung Jonathan Hung added a comment -

          Hi Naganarasimha G R, I also attached a branch-2.7 patch, if you could take a look and commit this one as well that would be great, thanks! (It is basically the same thing as the trunk patch, except stat.getLocalPath().toPath() becomes ConverterUtils.getPathFromYarnURL(stat.getLocalPath()).)

          Show
          jhung Jonathan Hung added a comment - Hi Naganarasimha G R , I also attached a branch-2.7 patch, if you could take a look and commit this one as well that would be great, thanks! (It is basically the same thing as the trunk patch, except stat.getLocalPath().toPath() becomes ConverterUtils.getPathFromYarnURL(stat.getLocalPath()) .)
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 8m 58s Docker failed to build yetus/hadoop:c420dfe.



          Subsystem Report/Notes
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838816/YARN-4355-branch-2.7.001.patch
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13900/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 8m 58s Docker failed to build yetus/hadoop:c420dfe. Subsystem Report/Notes JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838816/YARN-4355-branch-2.7.001.patch Console output https://builds.apache.org/job/PreCommit-YARN-Build/13900/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jhung Jonathan Hung added a comment -

          Attaching branch-2.7 patch with diff from root directory

          Show
          jhung Jonathan Hung added a comment - Attaching branch-2.7 patch with diff from root directory
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 3m 57s Docker failed to build yetus/hadoop:c420dfe.



          Subsystem Report/Notes
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838826/YARN-4355-branch-2.7.002.patch
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13903/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 3m 57s Docker failed to build yetus/hadoop:c420dfe. Subsystem Report/Notes JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838826/YARN-4355-branch-2.7.002.patch Console output https://builds.apache.org/job/PreCommit-YARN-Build/13903/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 25s branch-2.7 passed
          +1 compile 0m 22s branch-2.7 passed with JDK v1.8.0_111
          +1 compile 0m 27s branch-2.7 passed with JDK v1.7.0_111
          +1 checkstyle 0m 18s branch-2.7 passed
          +1 mvnsite 0m 28s branch-2.7 passed
          +1 mvneclipse 0m 13s branch-2.7 passed
          +1 findbugs 0m 51s branch-2.7 passed
          +1 javadoc 0m 15s branch-2.7 passed with JDK v1.8.0_111
          +1 javadoc 0m 19s branch-2.7 passed with JDK v1.7.0_111
          +1 mvninstall 0m 22s the patch passed
          +1 compile 0m 20s the patch passed with JDK v1.8.0_111
          +1 javac 0m 20s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.7.0_111
          +1 javac 0m 24s the patch passed
          -0 checkstyle 0m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 332 unchanged - 2 fixed = 335 total (was 334)
          +1 mvnsite 0m 25s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          -1 whitespace 0m 0s The patch has 3336 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 1m 17s The patch 102 line(s) with tabs.
          +1 findbugs 0m 58s the patch passed
          +1 javadoc 0m 13s the patch passed with JDK v1.8.0_111
          +1 javadoc 0m 17s the patch passed with JDK v1.7.0_111
          +1 unit 5m 54s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_111.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          40m 29s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue YARN-4355
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838833/YARN-4355-branch-2.7.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7fc877b5f519 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 81bf6f2
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/whitespace-tabs.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13906/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13906/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 25s branch-2.7 passed +1 compile 0m 22s branch-2.7 passed with JDK v1.8.0_111 +1 compile 0m 27s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 18s branch-2.7 passed +1 mvnsite 0m 28s branch-2.7 passed +1 mvneclipse 0m 13s branch-2.7 passed +1 findbugs 0m 51s branch-2.7 passed +1 javadoc 0m 15s branch-2.7 passed with JDK v1.8.0_111 +1 javadoc 0m 19s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 22s the patch passed +1 compile 0m 20s the patch passed with JDK v1.8.0_111 +1 javac 0m 20s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_111 +1 javac 0m 24s the patch passed -0 checkstyle 0m 16s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 3 new + 332 unchanged - 2 fixed = 335 total (was 334) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 3336 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 17s The patch 102 line(s) with tabs. +1 findbugs 0m 58s the patch passed +1 javadoc 0m 13s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 17s the patch passed with JDK v1.7.0_111 +1 unit 5m 54s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 40m 29s Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue YARN-4355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838833/YARN-4355-branch-2.7.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7fc877b5f519 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 81bf6f2 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13906/artifact/patchprocess/whitespace-tabs.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13906/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13906/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1, latest trunk and 2.7 patches look good to me.

          Show
          jlowe Jason Lowe added a comment - +1, latest trunk and 2.7 patches look good to me.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Jonathan Hung for 2.7 patch & Jason Lowe for the review, committing it shortly !

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Jonathan Hung for 2.7 patch & Jason Lowe for the review, committing it shortly !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10837 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10837/)
          YARN-4355. NPE while processing localizer heartbeat. Contributed by (naganarasimha_gr: rev 7ffb9943b8838a3bb56684e0722db40d800743a2)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10837 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10837/ ) YARN-4355 . NPE while processing localizer heartbeat. Contributed by (naganarasimha_gr: rev 7ffb9943b8838a3bb56684e0722db40d800743a2) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the contribution Varun Saxena and additionally supporting in rebase by Jonathan Hung. Thanks for review from Jason Lowe & Daniel Templeton.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the contribution Varun Saxena and additionally supporting in rebase by Jonathan Hung . Thanks for review from Jason Lowe & Daniel Templeton .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Looks to be changes.txt updation is missed in branch-2.7, can you please update.?

          Show
          brahmareddy Brahma Reddy Battula added a comment - Looks to be changes.txt updation is missed in branch-2.7, can you please update.?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Brahma, have done addendum commit with modifications for changes.txt

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Brahma , have done addendum commit with modifications for changes.txt
          Hide
          varun_saxena Varun Saxena added a comment -

          Thanks Naganarasimha G R for the commit.
          Thanks Jonathan Hung for the rebase. And thanks Daniel Templeton and Jason Lowe for reviews.

          Show
          varun_saxena Varun Saxena added a comment - Thanks Naganarasimha G R for the commit. Thanks Jonathan Hung for the rebase. And thanks Daniel Templeton and Jason Lowe for reviews.
          Hide
          jhung Jonathan Hung added a comment -

          Thanks everyone! Also I missed the CHANGES.txt addition, thanks Brahma Reddy Battula for catching that.

          Show
          jhung Jonathan Hung added a comment - Thanks everyone! Also I missed the CHANGES.txt addition, thanks Brahma Reddy Battula for catching that.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Its not exactly a contributor's responsibility actually while committing i missed it for 2.7 and in other upper versions just specifying in Fix version takes care of release notes.

          Show
          Naganarasimha Naganarasimha G R added a comment - Its not exactly a contributor's responsibility actually while committing i missed it for 2.7 and in other upper versions just specifying in Fix version takes care of release notes.

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development