Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      YARN-5620 and YARN-5637 allows an AM to explicitly upgrade a container with a new launch context and subsequently rollback / commit the change on the Container. This can also be used to simply restart the Container as well.

      This JIRA proposes to extend the ContainerManagementProtocol with the following API:

      • reInitializeContainer
      • rollbackLastUpgrade
      • commitLastUpgrade
      • restartContainer
      1. YARN-5609.001.patch
        79 kB
        Arun Suresh
      2. YARN-5609.002.patch
        82 kB
        Arun Suresh
      3. YARN-5609.003.patch
        91 kB
        Arun Suresh
      4. YARN-5609.004.patch
        95 kB
        Arun Suresh
      5. YARN-5609.005.patch
        98 kB
        Arun Suresh
      6. YARN-5609.006.patch
        98 kB
        Arun Suresh
      7. YARN-5609.007.patch
        107 kB
        Arun Suresh
      8. YARN-5609.008.patch
        100 kB
        Arun Suresh
      9. YARN-5609.009.patch
        102 kB
        Arun Suresh
      10. YARN-5609.010.patch
        100 kB
        Arun Suresh

        Activity

        Hide
        jianhe Jian He added a comment -

        one question, what is the difference between restartContainer and upgradeContainer ? is restartContainer the same as upgradeContainer without new launchcontext.

        Show
        jianhe Jian He added a comment - one question, what is the difference between restartContainer and upgradeContainer ? is restartContainer the same as upgradeContainer without new launchcontext.
        Hide
        asuresh Arun Suresh added a comment -

        Yup... thats the way I plan to implement it...

        Show
        asuresh Arun Suresh added a comment - Yup... thats the way I plan to implement it...
        Hide
        jianhe Jian He added a comment - - edited

        Also, I plan to use HADOOP-11552 for the relocalize API in NMClient so that AM does not need to poll for the completion of localization. Basically, the API will block until the localization is asynchronously done. Does this make sense for upgrade too ?

        Show
        jianhe Jian He added a comment - - edited Also, I plan to use HADOOP-11552 for the relocalize API in NMClient so that AM does not need to poll for the completion of localization. Basically, the API will block until the localization is asynchronously done. Does this make sense for upgrade too ?
        Hide
        asuresh Arun Suresh added a comment -

        Yup.. that definitely makes sense.. happy to help with the reviews there..
        Also, isn't HADOOP-11552 a server side change ? I am guessing there won't be any significant changes on the client side NMClient and the all the major changes to be isolated to the ContainerManagerProtocolPBServiceImpl

        Show
        asuresh Arun Suresh added a comment - Yup.. that definitely makes sense.. happy to help with the reviews there.. Also, isn't HADOOP-11552 a server side change ? I am guessing there won't be any significant changes on the client side NMClient and the all the major changes to be isolated to the ContainerManagerProtocolPBServiceImpl
        Hide
        asuresh Arun Suresh added a comment - - edited

        Uploading initial patch:

        • This includes all the protocol, PBImpl changes and any changes resulting in modifying the ContainerManagerProtocol.
        • This does not contain the changes to NMClient.
        • Decided to go with reInitializeContainer instead of upgradeContainer, since the API can very well be used to move back to any old version as long as a Launch context is provided.
        Show
        asuresh Arun Suresh added a comment - - edited Uploading initial patch: This includes all the protocol, PBImpl changes and any changes resulting in modifying the ContainerManagerProtocol. This does not contain the changes to NMClient . Decided to go with reInitializeContainer instead of upgradeContainer , since the API can very well be used to move back to any old version as long as a Launch context is provided.
        Hide
        jianhe Jian He added a comment -

        Yes, it'll be sever-side changes and a bit of client.

        Show
        jianhe Jian He added a comment - Yes, it'll be sever-side changes and a bit of client.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
        0 mvndep 1m 36s Maven dependency ordering for branch
        +1 mvninstall 6m 58s trunk passed
        +1 compile 6m 53s trunk passed
        +1 checkstyle 1m 32s trunk passed
        +1 mvnsite 2m 51s trunk passed
        +1 mvneclipse 1m 23s trunk passed
        +1 findbugs 4m 44s trunk passed
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 2m 20s the patch passed
        +1 compile 7m 20s the patch passed
        +1 cc 7m 20s the patch passed
        +1 javac 7m 20s the patch passed
        -1 checkstyle 1m 32s root: The patch generated 24 new + 443 unchanged - 1 fixed = 467 total (was 444)
        +1 mvnsite 3m 21s the patch passed
        +1 mvneclipse 1m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 6m 41s the patch passed
        -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123)
        -1 javadoc 0m 27s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 0m 33s hadoop-yarn-api in the patch passed.
        +1 unit 2m 39s hadoop-yarn-common in the patch passed.
        +1 unit 0m 33s hadoop-yarn-server-common in the patch passed.
        -1 unit 18m 28s hadoop-yarn-server-nodemanager in the patch failed.
        +1 unit 34m 32s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 25s The patch does not generate ASF License warnings.
        120m 57s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRegression
          hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          hadoop.yarn.server.nodemanager.containermanager.TestContainerManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829164/YARN-5609.001.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux d5a517cb5440 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b8a30f2
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13145/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13145/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 1m 36s Maven dependency ordering for branch +1 mvninstall 6m 58s trunk passed +1 compile 6m 53s trunk passed +1 checkstyle 1m 32s trunk passed +1 mvnsite 2m 51s trunk passed +1 mvneclipse 1m 23s trunk passed +1 findbugs 4m 44s trunk passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 2m 20s the patch passed +1 compile 7m 20s the patch passed +1 cc 7m 20s the patch passed +1 javac 7m 20s the patch passed -1 checkstyle 1m 32s root: The patch generated 24 new + 443 unchanged - 1 fixed = 467 total (was 444) +1 mvnsite 3m 21s the patch passed +1 mvneclipse 1m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 6m 41s the patch passed -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123) -1 javadoc 0m 27s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 0m 33s hadoop-yarn-api in the patch passed. +1 unit 2m 39s hadoop-yarn-common in the patch passed. +1 unit 0m 33s hadoop-yarn-server-common in the patch passed. -1 unit 18m 28s hadoop-yarn-server-nodemanager in the patch failed. +1 unit 34m 32s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 120m 57s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRegression   hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.nodemanager.containermanager.TestContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829164/YARN-5609.001.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux d5a517cb5440 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b8a30f2 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13145/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13145/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13145/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -
        • Can you add comments in commitLastReInitialization that once committed, user will not be able to rollback.
        • shouldn't set it to null?
                ContainerLaunchContext launchContext =
                    reInitEvent.getReInitLaunchContext() == null ?
                        container.launchContext : null;
          
        • how about RestartResponse-> RestartContainerResponse, ReInitializationRequest-> ReInitializeContainerRequest ?
        Show
        jianhe Jian He added a comment - Can you add comments in commitLastReInitialization that once committed, user will not be able to rollback. shouldn't set it to null? ContainerLaunchContext launchContext = reInitEvent.getReInitLaunchContext() == null ? container.launchContext : null ; how about RestartResponse-> RestartContainerResponse, ReInitializationRequest-> ReInitializeContainerRequest ?
        Hide
        asuresh Arun Suresh added a comment -

        Uploading patch based on Jian He's suggestions. Thanks of the review !!

        • Also fixing testcase errors (which was due to the fact that the launch context was null.. good catch Jian)
        • Added some basic authorization check to see if the remoteUgi and NMTokenIdentifier presented by the caller is correct. Wondering if we need to also ensure that only the application that started the container can reinitialize it. Do we need this for the localize api as well ? Thoughts Jian He ?
        Show
        asuresh Arun Suresh added a comment - Uploading patch based on Jian He 's suggestions. Thanks of the review !! Also fixing testcase errors (which was due to the fact that the launch context was null.. good catch Jian) Added some basic authorization check to see if the remoteUgi and NMTokenIdentifier presented by the caller is correct. Wondering if we need to also ensure that only the application that started the container can reinitialize it. Do we need this for the localize api as well ? Thoughts Jian He ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 31s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 10 new or modified test files.
        0 mvndep 1m 39s Maven dependency ordering for branch
        +1 mvninstall 8m 59s trunk passed
        +1 compile 9m 24s trunk passed
        +1 checkstyle 1m 44s trunk passed
        +1 mvnsite 2m 56s trunk passed
        +1 mvneclipse 1m 25s trunk passed
        +1 findbugs 4m 46s trunk passed
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 2m 16s the patch passed
        +1 compile 7m 5s the patch passed
        +1 cc 7m 5s the patch passed
        +1 javac 7m 5s the patch passed
        -1 checkstyle 1m 38s root: The patch generated 36 new + 471 unchanged - 1 fixed = 507 total (was 472)
        +1 mvnsite 2m 57s the patch passed
        +1 mvneclipse 1m 31s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 7m 18s the patch passed
        -1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123)
        -1 javadoc 0m 25s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 0m 32s hadoop-yarn-api in the patch passed.
        +1 unit 2m 40s hadoop-yarn-common in the patch passed.
        +1 unit 0m 34s hadoop-yarn-server-common in the patch passed.
        +1 unit 15m 24s hadoop-yarn-server-nodemanager in the patch passed.
        -1 unit 35m 14s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 8m 48s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        135m 28s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMAdminService



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829236/YARN-5609.002.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux bf36a673d2c3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b8a30f2
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13150/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13150/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 10 new or modified test files. 0 mvndep 1m 39s Maven dependency ordering for branch +1 mvninstall 8m 59s trunk passed +1 compile 9m 24s trunk passed +1 checkstyle 1m 44s trunk passed +1 mvnsite 2m 56s trunk passed +1 mvneclipse 1m 25s trunk passed +1 findbugs 4m 46s trunk passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 2m 16s the patch passed +1 compile 7m 5s the patch passed +1 cc 7m 5s the patch passed +1 javac 7m 5s the patch passed -1 checkstyle 1m 38s root: The patch generated 36 new + 471 unchanged - 1 fixed = 507 total (was 472) +1 mvnsite 2m 57s the patch passed +1 mvneclipse 1m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 7m 18s the patch passed -1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123) -1 javadoc 0m 25s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 0m 32s hadoop-yarn-api in the patch passed. +1 unit 2m 40s hadoop-yarn-common in the patch passed. +1 unit 0m 34s hadoop-yarn-server-common in the patch passed. +1 unit 15m 24s hadoop-yarn-server-nodemanager in the patch passed. -1 unit 35m 14s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 48s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 135m 28s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMAdminService Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829236/YARN-5609.002.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux bf36a673d2c3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b8a30f2 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13150/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13150/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13150/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        Thanks Arun, some more comments:

        • IIUC, when restarting the container, the reInitEvent.getResourceSet() is empty
                ContainerLaunchContext launchContext =
                    reInitEvent.getReInitLaunchContext() == null ?
                        container.launchContext : reInitEvent.getReInitLaunchContext();
                return new ReInitializationContext(
                    launchContext, reInitEvent.getResourceSet(),
          

          and later on, here it will return empty newResourceSet because oldLaunchContext is null? and this caused the container restarted with incorrect symlinks. do you mind adding a UT for restart container too ?

              private ResourceSet mergedResourceSet() {
                if (oldLaunchContext == null) {
                  return newResourceSet;
                }
          
        • should we add some success/failure audit log to the API ?

          Wondering if we need to also ensure that only the application that started the container can reinitialize it.

          yeah, I agree.

        Show
        jianhe Jian He added a comment - Thanks Arun, some more comments: IIUC, when restarting the container, the reInitEvent.getResourceSet() is empty ContainerLaunchContext launchContext = reInitEvent.getReInitLaunchContext() == null ? container.launchContext : reInitEvent.getReInitLaunchContext(); return new ReInitializationContext( launchContext, reInitEvent.getResourceSet(), and later on, here it will return empty newResourceSet because oldLaunchContext is null? and this caused the container restarted with incorrect symlinks. do you mind adding a UT for restart container too ? private ResourceSet mergedResourceSet() { if (oldLaunchContext == null ) { return newResourceSet; } should we add some success/failure audit log to the API ? Wondering if we need to also ensure that only the application that started the container can reinitialize it. yeah, I agree.
        Hide
        asuresh Arun Suresh added a comment - - edited

        Uploading updated patch. Thanks Jian He..

        • Added a new testcase for restart
        • Added an authorization check to ensure only applications that started the container can reinit/restart/rollback etc.
        • Added some metrics to monitor the number of reinitialized and auto-rolledback containers.

        Will fix the checkstyles (and add the Audit logs) once we are fine with the API / class names etc.

        Show
        asuresh Arun Suresh added a comment - - edited Uploading updated patch. Thanks Jian He .. Added a new testcase for restart Added an authorization check to ensure only applications that started the container can reinit/restart/rollback etc. Added some metrics to monitor the number of reinitialized and auto-rolledback containers. Will fix the checkstyles (and add the Audit logs) once we are fine with the API / class names etc.
        Hide
        jianhe Jian He added a comment -
        • I think this will cause the resources to be re-requested on restart. Even though the effect might still be the same, because the resources are already localized and the requests will be ignored, but I think we can still try to avoid sending these unnecessary events in case the resource set is large ?
                  // This is a Restart...
                  return new ReInitializationContext(
                      container.launchContext, container.resourceSet, null, null);
          

          Also, suppose this is a restart after upgrade, then the old contexts are wiped out by this call, and user won't be able to rollback, after the restart.

        • Can we add some comments about what ReInitializationContext#newResourceSet contains, on upgrade, it contains pendingResources only, while on rollback it contains full copy of original resources.
        • while looking at previous code: is it possible for this call to return null ? If it's possible , then later code will throw NPE.
              private ReInitializationContext createContextForRollback() {
                if (oldLaunchContext == null) {
                  return null;
                } else {
          
        Show
        jianhe Jian He added a comment - I think this will cause the resources to be re-requested on restart. Even though the effect might still be the same, because the resources are already localized and the requests will be ignored, but I think we can still try to avoid sending these unnecessary events in case the resource set is large ? // This is a Restart... return new ReInitializationContext( container.launchContext, container.resourceSet, null , null ); Also, suppose this is a restart after upgrade, then the old contexts are wiped out by this call, and user won't be able to rollback, after the restart. Can we add some comments about what ReInitializationContext#newResourceSet contains, on upgrade, it contains pendingResources only, while on rollback it contains full copy of original resources. while looking at previous code: is it possible for this call to return null ? If it's possible , then later code will throw NPE. private ReInitializationContext createContextForRollback() { if (oldLaunchContext == null ) { return null ; } else {
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 57s trunk passed
        +1 compile 6m 59s trunk passed
        +1 checkstyle 1m 33s trunk passed
        +1 mvnsite 2m 52s trunk passed
        +1 mvneclipse 1m 24s trunk passed
        +1 findbugs 4m 52s trunk passed
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 21s the patch passed
        +1 compile 7m 13s the patch passed
        +1 cc 7m 13s the patch passed
        +1 javac 7m 13s the patch passed
        -1 checkstyle 1m 35s root: The patch generated 38 new + 494 unchanged - 1 fixed = 532 total (was 495)
        +1 mvnsite 2m 59s the patch passed
        +1 mvneclipse 1m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 49s the patch passed
        -1 javadoc 0m 17s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123)
        -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 0m 26s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        -1 unit 15m 19s hadoop-yarn-server-nodemanager in the patch failed.
        -1 unit 34m 2s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 8m 47s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        113m 34s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager
          hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829340/YARN-5609.003.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 245b238268e2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e52d6e7
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13160/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13160/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 57s trunk passed +1 compile 6m 59s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 2m 52s trunk passed +1 mvneclipse 1m 24s trunk passed +1 findbugs 4m 52s trunk passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 21s the patch passed +1 compile 7m 13s the patch passed +1 cc 7m 13s the patch passed +1 javac 7m 13s the patch passed -1 checkstyle 1m 35s root: The patch generated 38 new + 494 unchanged - 1 fixed = 532 total (was 495) +1 mvnsite 2m 59s the patch passed +1 mvneclipse 1m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 49s the patch passed -1 javadoc 0m 17s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123) -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 0m 26s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. -1 unit 15m 19s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 34m 2s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 47s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 113m 34s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager   hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829340/YARN-5609.003.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 245b238268e2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e52d6e7 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13160/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13160/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13160/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment - - edited

        Updating patch, Thanks Jian He..

        • Addressed most of your concerns.
        • Added some more comments to clarify some assumptions.
        • Added testcases to verify that Explicit rollback is still possible AFTER upgraded container has been restarted.

        I think this will cause the resources to be re-requested on restart. Even though the effect might still be the same, because the resources are already localized and the requests will be ignored, but I think we can still try to avoid sending these unnecessary events in case the resource set is large ?

        I had intentionally kept it that way (my thinking was that the Tracker will then verify that the resources.. directories etc. are still good)...
        But I agree with you.. It is inefficient. I've updated patch to make sure that in case of rollback and restart, this wont happen.. I've also put some comments there.. do take a look and let me know if its fine.

        With regard to this:

         
            private ReInitializationContext createContextForRollback() {
              if (oldLaunchContext == null) {
                return null;
              } else {
        

        There should not be a NPE, since it is always called in conjunctions with a container.canRollback() which returns true only if oldLaunchContext is non null.

        Show
        asuresh Arun Suresh added a comment - - edited Updating patch, Thanks Jian He .. Addressed most of your concerns. Added some more comments to clarify some assumptions. Added testcases to verify that Explicit rollback is still possible AFTER upgraded container has been restarted. I think this will cause the resources to be re-requested on restart. Even though the effect might still be the same, because the resources are already localized and the requests will be ignored, but I think we can still try to avoid sending these unnecessary events in case the resource set is large ? I had intentionally kept it that way (my thinking was that the Tracker will then verify that the resources.. directories etc. are still good)... But I agree with you.. It is inefficient. I've updated patch to make sure that in case of rollback and restart, this wont happen.. I've also put some comments there.. do take a look and let me know if its fine. With regard to this: private ReInitializationContext createContextForRollback() { if (oldLaunchContext == null) { return null; } else { There should not be a NPE, since it is always called in conjunctions with a container.canRollback() which returns true only if oldLaunchContext is non null.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 53s trunk passed
        +1 compile 7m 13s trunk passed
        +1 checkstyle 1m 33s trunk passed
        +1 mvnsite 2m 58s trunk passed
        +1 mvneclipse 1m 25s trunk passed
        +1 findbugs 4m 50s trunk passed
        -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 21s the patch passed
        +1 compile 6m 59s the patch passed
        +1 cc 6m 59s the patch passed
        +1 javac 6m 59s the patch passed
        -1 checkstyle 1m 34s root: The patch generated 28 new + 494 unchanged - 1 fixed = 522 total (was 495)
        +1 mvnsite 2m 56s the patch passed
        +1 mvneclipse 1m 24s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 37s the patch passed
        -1 javadoc 0m 18s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123)
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 0m 26s hadoop-yarn-api in the patch passed.
        +1 unit 2m 20s hadoop-yarn-common in the patch passed.
        +1 unit 0m 28s hadoop-yarn-server-common in the patch passed.
        -1 unit 14m 42s hadoop-yarn-server-nodemanager in the patch failed.
        +1 unit 34m 8s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 45s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        112m 47s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829415/YARN-5609.004.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 6cdd0b97d032 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c6d1d74
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13164/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13164/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 53s trunk passed +1 compile 7m 13s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 2m 58s trunk passed +1 mvneclipse 1m 25s trunk passed +1 findbugs 4m 50s trunk passed -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 21s the patch passed +1 compile 6m 59s the patch passed +1 cc 6m 59s the patch passed +1 javac 6m 59s the patch passed -1 checkstyle 1m 34s root: The patch generated 28 new + 494 unchanged - 1 fixed = 522 total (was 495) +1 mvnsite 2m 56s the patch passed +1 mvneclipse 1m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 37s the patch passed -1 javadoc 0m 18s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 3 new + 123 unchanged - 0 fixed = 126 total (was 123) -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 0m 26s hadoop-yarn-api in the patch passed. +1 unit 2m 20s hadoop-yarn-common in the patch passed. +1 unit 0m 28s hadoop-yarn-server-common in the patch passed. -1 unit 14m 42s hadoop-yarn-server-nodemanager in the patch failed. +1 unit 34m 8s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 45s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 112m 47s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829415/YARN-5609.004.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 6cdd0b97d032 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c6d1d74 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13164/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13164/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13164/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment -

        Uploading patch:

        • Fixing Test failures, checkstyles and javadocs.
        • Added NMAudit logging messages when reinitialization starts and completes.
        Show
        asuresh Arun Suresh added a comment - Uploading patch: Fixing Test failures, checkstyles and javadocs. Added NMAudit logging messages when reinitialization starts and completes.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 7m 15s trunk passed
        +1 compile 7m 26s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 3m 0s trunk passed
        +1 mvneclipse 1m 25s trunk passed
        +1 findbugs 5m 6s trunk passed
        +1 javadoc 2m 18s trunk passed
        0 mvndep 0m 26s Maven dependency ordering for patch
        +1 mvninstall 2m 37s the patch passed
        +1 compile 7m 17s the patch passed
        +1 cc 7m 17s the patch passed
        +1 javac 7m 17s the patch passed
        +1 checkstyle 1m 35s root: The patch generated 0 new + 492 unchanged - 17 fixed = 492 total (was 509)
        +1 mvnsite 3m 1s the patch passed
        +1 mvneclipse 1m 23s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 55s the patch passed
        +1 javadoc 1m 57s the patch passed
        +1 unit 0m 26s hadoop-yarn-api in the patch passed.
        +1 unit 2m 20s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        -1 unit 14m 40s hadoop-yarn-server-nodemanager in the patch failed.
        -1 unit 34m 43s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 8m 47s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        115m 48s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRegression
          hadoop.yarn.server.nodemanager.containermanager.TestContainerManager
          hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829463/YARN-5609.005.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 323780bc91bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e80386d
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13169/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13169/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 7m 15s trunk passed +1 compile 7m 26s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 3m 0s trunk passed +1 mvneclipse 1m 25s trunk passed +1 findbugs 5m 6s trunk passed +1 javadoc 2m 18s trunk passed 0 mvndep 0m 26s Maven dependency ordering for patch +1 mvninstall 2m 37s the patch passed +1 compile 7m 17s the patch passed +1 cc 7m 17s the patch passed +1 javac 7m 17s the patch passed +1 checkstyle 1m 35s root: The patch generated 0 new + 492 unchanged - 17 fixed = 492 total (was 509) +1 mvnsite 3m 1s the patch passed +1 mvneclipse 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 55s the patch passed +1 javadoc 1m 57s the patch passed +1 unit 0m 26s hadoop-yarn-api in the patch passed. +1 unit 2m 20s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. -1 unit 14m 40s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 34m 43s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 47s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 115m 48s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.TestContainerManagerRegression   hadoop.yarn.server.nodemanager.containermanager.TestContainerManager   hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829463/YARN-5609.005.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 323780bc91bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e80386d Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13169/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13169/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment - - edited

        Updating patch to make testcase less fragile..

        Show
        asuresh Arun Suresh added a comment - - edited Updating patch to make testcase less fragile..
        Hide
        jianhe Jian He added a comment - - edited

        I had intentionally kept it that way (my thinking was that the Tracker will then verify that the resources.. directories etc. are still good)

        Yep, I also had that in mind for rollback. And I think it is indeed needed for rollback, because the old resource may have a chance to get purged...
        But for restart, as the same resources are re-used, we don't need to re-check. So, I guess we need to retain the behavior for rollback ?

        Show
        jianhe Jian He added a comment - - edited I had intentionally kept it that way (my thinking was that the Tracker will then verify that the resources.. directories etc. are still good) Yep, I also had that in mind for rollback. And I think it is indeed needed for rollback, because the old resource may have a chance to get purged... But for restart, as the same resources are re-used, we don't need to re-check. So, I guess we need to retain the behavior for rollback ?
        Hide
        jianhe Jian He added a comment -

        since it is always called in conjunctions with a container.canRollback() which returns true only if oldLaunchContext is non null.

        I see, should we remove the if null condition ? as it'll never happens.

        Show
        jianhe Jian He added a comment - since it is always called in conjunctions with a container.canRollback() which returns true only if oldLaunchContext is non null. I see, should we remove the if null condition ? as it'll never happens.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 47s trunk passed
        +1 compile 7m 54s trunk passed
        +1 checkstyle 1m 41s trunk passed
        +1 mvnsite 3m 17s trunk passed
        +1 mvneclipse 1m 27s trunk passed
        +1 findbugs 6m 8s trunk passed
        +1 javadoc 2m 0s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 25s the patch passed
        +1 compile 7m 4s the patch passed
        +1 cc 7m 4s the patch passed
        +1 javac 7m 4s the patch passed
        +1 checkstyle 1m 34s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 2m 52s the patch passed
        +1 mvneclipse 1m 24s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 30s the patch passed
        +1 javadoc 1m 53s the patch passed
        +1 unit 0m 24s hadoop-yarn-api in the patch passed.
        +1 unit 2m 17s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        +1 unit 14m 42s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 34m 0s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 46s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        115m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829491/YARN-5609.006.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 5c171a87473e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 964e546
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13172/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13172/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 47s trunk passed +1 compile 7m 54s trunk passed +1 checkstyle 1m 41s trunk passed +1 mvnsite 3m 17s trunk passed +1 mvneclipse 1m 27s trunk passed +1 findbugs 6m 8s trunk passed +1 javadoc 2m 0s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 25s the patch passed +1 compile 7m 4s the patch passed +1 cc 7m 4s the patch passed +1 javac 7m 4s the patch passed +1 checkstyle 1m 34s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 2m 52s the patch passed +1 mvneclipse 1m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 30s the patch passed +1 javadoc 1m 53s the patch passed +1 unit 0m 24s hadoop-yarn-api in the patch passed. +1 unit 2m 17s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 unit 14m 42s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 34m 0s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 46s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 115m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829491/YARN-5609.006.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 5c171a87473e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 964e546 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13172/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13172/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment -

        Updating Patch based on your suggestions..

        But for restart, as the same resources are re-used, we don't need to re-check. So, I guess we need to retain the behavior for rollback ?

        Agreed... In the latest patch, the rollback ResourceSet is now a clone of the prev resourceSet, with all the required LocalResources moved back into the pending list. Had to change a few things to get this to work though.. do take a look..

        Show
        asuresh Arun Suresh added a comment - Updating Patch based on your suggestions.. But for restart, as the same resources are re-used, we don't need to re-check. So, I guess we need to retain the behavior for rollback ? Agreed... In the latest patch, the rollback ResourceSet is now a clone of the prev resourceSet, with all the required LocalResources moved back into the pending list. Had to change a few things to get this to work though.. do take a look..
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 7m 25s trunk passed
        +1 compile 8m 5s trunk passed
        +1 checkstyle 1m 39s trunk passed
        +1 mvnsite 3m 17s trunk passed
        +1 mvneclipse 1m 34s trunk passed
        +1 findbugs 6m 10s trunk passed
        +1 javadoc 2m 16s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 2m 24s the patch passed
        +1 compile 7m 13s the patch passed
        +1 cc 7m 13s the patch passed
        +1 javac 7m 13s the patch passed
        +1 checkstyle 1m 33s root: The patch generated 0 new + 493 unchanged - 17 fixed = 493 total (was 510)
        +1 mvnsite 2m 50s the patch passed
        +1 mvneclipse 1m 23s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 30s the patch passed
        +1 javadoc 1m 53s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 19s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        +1 unit 14m 47s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 33m 49s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 44s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        116m 17s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829523/YARN-5609.007.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 39ae929a922e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 964e546
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13176/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13176/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 25s trunk passed +1 compile 8m 5s trunk passed +1 checkstyle 1m 39s trunk passed +1 mvnsite 3m 17s trunk passed +1 mvneclipse 1m 34s trunk passed +1 findbugs 6m 10s trunk passed +1 javadoc 2m 16s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 24s the patch passed +1 compile 7m 13s the patch passed +1 cc 7m 13s the patch passed +1 javac 7m 13s the patch passed +1 checkstyle 1m 33s root: The patch generated 0 new + 493 unchanged - 17 fixed = 493 total (was 510) +1 mvnsite 2m 50s the patch passed +1 mvneclipse 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 30s the patch passed +1 javadoc 1m 53s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 unit 14m 47s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 33m 49s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 44s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 116m 17s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829523/YARN-5609.007.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 39ae929a922e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 964e546 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13176/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13176/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        I had re-checked the code... actually, even the old resources will also not be purged because in the container reInit state-machine we are never sending the cleanup event, and the background cleanup thread also cannot purge it, because the resource is still considered being used by the container (refCount > 0).
        I think it's an ok assumption to keep two versions of resources current and previous. Code is also simpler.. We probably need a followup patch to let the background cleanup thread clean the resourceSet older than the last. User anyway has to provide the full set of resources via the reInit API if they want to rollback to older than last. what do you think...

        Show
        jianhe Jian He added a comment - I had re-checked the code... actually, even the old resources will also not be purged because in the container reInit state-machine we are never sending the cleanup event, and the background cleanup thread also cannot purge it, because the resource is still considered being used by the container (refCount > 0). I think it's an ok assumption to keep two versions of resources current and previous. Code is also simpler.. We probably need a followup patch to let the background cleanup thread clean the resourceSet older than the last. User anyway has to provide the full set of resources via the reInit API if they want to rollback to older than last. what do you think...
        Hide
        asuresh Arun Suresh added a comment -

        User anyway has to provide the full set of resources via the reInit API if they want to rollback to older than last. what do you think...

        You mean rollback to a version before previous ?... Agreed

        So just to clarify, the ReInit behavior as per the latest patch (v007) is:

        1. Container C1 currently has resources (a and b) symlinked to (s1 and s2)
        2. The reInitialization req. comes in with autocommit = false and which asks for resource c symlinked to s1
          • The ReInitContext created will have newResourceSet = (c) and oldResourceSet = Clone of current resourceset (which contains a and b, but it is moved to the pending list)
        3. Container moves into REINITIALIZING state while c is being localized.
        4. Once c is localized, container gets a RESOURCE_LOCALIZED event. Since there are no more pending resources, in the ResourceLocalizedWhileReInitTransition, a CLEANUP_CONTAINER_FOR_REINIT is sent to the launcher.
        5. The CLEANUP_CONTAINER_FOR_REINIT is handled by the ContainerLaunch which kills the current process and sends a CONTAINER_KILLED_ON_REQUEST to the container.
        6. This moves the Container to LOCALIZED state and invokes the KilledForReInitializationTransition which does a container.sendLaunchEvent() which restarts the process with the new launch context and resourceSet.

        AutoRollback (in case of container failure after re-init):

        1. In the RetryFailureTransition, we decide the container has exhausted its retries, and we decide to rollback. Unlike normal Reinit, the container is NOT running now..
        2. Since the oldResourceSet is a clone of the previous state, with a and b still in pending, C1 sends LOCALIZE_CONTAINER_RESOURCES event to localizer and moves to REINITIALIZING state to wait for response.
        3. Once a and b has been verified (Currently its is not re-localized.. as you pointed out... but the Tracker still checks if the resource exits) RESOURCE_LOCALIZED is sent to the Container.
        4. Unlike the normal reInit case, since the process is already dead, I cannot send a CLEANUP_CONTAINER_FOR_REINIT to the ContainerLaunch (since it does not exit) so I send the CONTAINER_KILLED_ON_REQUEST to the container.. which is handled by container and proceeds as steps 5 and 6 above.

        Explicit Rollback (AM invoked) is Exactly like the normal ReInit case, but autoCommit is true, so no rollback state is saved (since you should not be able to rollback a rollback)

        Restart is also similar to Reinit except that we first check if a rollback state exists.. if so, it just copies the reference of the oldResourceSet and oldLaunchcontext from the existing reinitContext to the new reInitContext. No resourceSet cloning takes place here (since the oldResourceSet is already a clone)

        Let me know if the above is fine...

        Show
        asuresh Arun Suresh added a comment - User anyway has to provide the full set of resources via the reInit API if they want to rollback to older than last. what do you think... You mean rollback to a version before previous ?... Agreed So just to clarify, the ReInit behavior as per the latest patch (v007) is: Container C1 currently has resources ( a and b ) symlinked to ( s1 and s2 ) The reInitialization req. comes in with autocommit = false and which asks for resource c symlinked to s1 The ReInitContext created will have newResourceSet = (c) and oldResourceSet = Clone of current resourceset (which contains a and b, but it is moved to the pending list) Container moves into REINITIALIZING state while c is being localized. Once c is localized, container gets a RESOURCE_LOCALIZED event. Since there are no more pending resources, in the ResourceLocalizedWhileReInitTransition , a CLEANUP_CONTAINER_FOR_REINIT is sent to the launcher. The CLEANUP_CONTAINER_FOR_REINIT is handled by the ContainerLaunch which kills the current process and sends a CONTAINER_KILLED_ON_REQUEST to the container. This moves the Container to LOCALIZED state and invokes the KilledForReInitializationTransition which does a container.sendLaunchEvent() which restarts the process with the new launch context and resourceSet. AutoRollback (in case of container failure after re-init): In the RetryFailureTransition , we decide the container has exhausted its retries, and we decide to rollback. Unlike normal Reinit, the container is NOT running now.. Since the oldResourceSet is a clone of the previous state, with a and b still in pending, C1 sends LOCALIZE_CONTAINER_RESOURCES event to localizer and moves to REINITIALIZING state to wait for response. Once a and b has been verified (Currently its is not re-localized.. as you pointed out... but the Tracker still checks if the resource exits) RESOURCE_LOCALIZED is sent to the Container. Unlike the normal reInit case, since the process is already dead, I cannot send a CLEANUP_CONTAINER_FOR_REINIT to the ContainerLaunch (since it does not exit) so I send the CONTAINER_KILLED_ON_REQUEST to the container.. which is handled by container and proceeds as steps 5 and 6 above. Explicit Rollback (AM invoked) is Exactly like the normal ReInit case, but autoCommit is true, so no rollback state is saved (since you should not be able to rollback a rollback) Restart is also similar to Reinit except that we first check if a rollback state exists.. if so, it just copies the reference of the oldResourceSet and oldLaunchcontext from the existing reinitContext to the new reInitContext. No resourceSet cloning takes place here (since the oldResourceSet is already a clone) Let me know if the above is fine...
        Hide
        jianhe Jian He added a comment -

        Yep, what I meant is the second last patch is fine, as the issue I raised earlier is invalid. So do you think we can go with the second last patch ?

        Show
        jianhe Jian He added a comment - Yep, what I meant is the second last patch is fine, as the issue I raised earlier is invalid. So do you think we can go with the second last patch ?
        Hide
        asuresh Arun Suresh added a comment -

        Ah ok (you mean v006).. sure.. I had found a minor bug in that patch.. I'll fix and repost it as v008...

        Show
        asuresh Arun Suresh added a comment - Ah ok (you mean v006).. sure.. I had found a minor bug in that patch.. I'll fix and repost it as v008...
        Hide
        asuresh Arun Suresh added a comment -

        Uploading new patch, which is basically v006 patch with a minor bug fix (during rollback, we should not merge the current resourceSet with new one.. it should just be replaced).

        Show
        asuresh Arun Suresh added a comment - Uploading new patch, which is basically v006 patch with a minor bug fix (during rollback, we should not merge the current resourceSet with new one.. it should just be replaced).
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 19s Maven dependency ordering for branch
        +1 mvninstall 7m 12s trunk passed
        +1 compile 7m 22s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 3m 0s trunk passed
        +1 mvneclipse 1m 27s trunk passed
        +1 findbugs 4m 56s trunk passed
        +1 javadoc 1m 59s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 27s the patch passed
        +1 compile 7m 9s the patch passed
        +1 cc 7m 9s the patch passed
        +1 javac 7m 9s the patch passed
        +1 checkstyle 1m 31s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 2m 56s the patch passed
        +1 mvneclipse 1m 25s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 42s the patch passed
        +1 javadoc 1m 52s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        +1 unit 15m 6s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 35m 28s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 9m 20s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        116m 18s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829621/YARN-5609.008.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux c5e48e2f03bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 964e546
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13177/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13177/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 19s Maven dependency ordering for branch +1 mvninstall 7m 12s trunk passed +1 compile 7m 22s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 3m 0s trunk passed +1 mvneclipse 1m 27s trunk passed +1 findbugs 4m 56s trunk passed +1 javadoc 1m 59s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 27s the patch passed +1 compile 7m 9s the patch passed +1 cc 7m 9s the patch passed +1 javac 7m 9s the patch passed +1 checkstyle 1m 31s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 2m 56s the patch passed +1 mvneclipse 1m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 42s the patch passed +1 javadoc 1m 52s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 unit 15m 6s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 35m 28s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 9m 20s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 116m 18s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829621/YARN-5609.008.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux c5e48e2f03bb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 964e546 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13177/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13177/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        during rollback, we should not merge the current resourceSet with new one.. it should just be replaced

        The original 006 patch, it seems not merging,
        so when it creates the rollback context, it sets the oldLaunchContext to null;

                return new ReInitializationContext(
                    oldLaunchContext, oldResourceSet, null, null);
              }
        

        And in the mergedResourceSet
        It returns newResourceSet if oldLaunchContext is null

             if (oldLaunchContext == null) {
                return newResourceSet;
              }
        

        Am I right?

        Show
        jianhe Jian He added a comment - during rollback, we should not merge the current resourceSet with new one.. it should just be replaced The original 006 patch, it seems not merging, so when it creates the rollback context, it sets the oldLaunchContext to null; return new ReInitializationContext( oldLaunchContext, oldResourceSet, null , null ); } And in the mergedResourceSet It returns newResourceSet if oldLaunchContext is null if (oldLaunchContext == null ) { return newResourceSet; } Am I right?
        Hide
        asuresh Arun Suresh added a comment -

        during rollback, we should not merge the current resourceSet with new one.. it should just be replaced

        Oops.. I meant restart instead of rollback. in the case of a restart (if there exists a rollback state when the restart is called), both old and new Launch contexts are non-null, so in v006, it used to merge it.. now it doesnt. I know the isRollback boolean is not really needed... but I thought it makes the code clearer..

        Show
        asuresh Arun Suresh added a comment - during rollback, we should not merge the current resourceSet with new one.. it should just be replaced Oops.. I meant restart instead of rollback . in the case of a restart (if there exists a rollback state when the restart is called), both old and new Launch contexts are non-null, so in v006, it used to merge it.. now it doesnt. I know the isRollback boolean is not really needed... but I thought it makes the code clearer..
        Hide
        jianhe Jian He added a comment -

        IIUC, even if it's a merged operation, the current will be a superset of previous, so the end result is the same ? probably both patches are fine..

        Show
        jianhe Jian He added a comment - IIUC, even if it's a merged operation, the current will be a superset of previous, so the end result is the same ? probably both patches are fine..
        Hide
        asuresh Arun Suresh added a comment -

        true... which is why the tests were passing in both cases... but I think 008 makes things explicit...
        Let me know if you are ok with it.. I will commit it tomorrow... if nobody else has any objections (cc: Varun Vasudev)

        Show
        asuresh Arun Suresh added a comment - true... which is why the tests were passing in both cases... but I think 008 makes things explicit... Let me know if you are ok with it.. I will commit it tomorrow... if nobody else has any objections (cc: Varun Vasudev )
        Hide
        jianhe Jian He added a comment - - edited

        looks good to me overall,
        I think wherever setIsReInitializing(false) is called endReInitingContainer should be called. Otherwise it's possible endReInitingContainer is not invoked.
        Maybe have a common method for both

        Show
        jianhe Jian He added a comment - - edited looks good to me overall, I think wherever setIsReInitializing(false) is called endReInitingContainer should be called. Otherwise it's possible endReInitingContainer is not invoked. Maybe have a common method for both
        Hide
        asuresh Arun Suresh added a comment -

        Uploading patch

        I think wherever setIsReInitializing(false) is called endReInitingContainer should be called. Otherwise it's possible endReInitingContainer is not invoked.

        Agreed... fixed this in v009

        Show
        asuresh Arun Suresh added a comment - Uploading patch I think wherever setIsReInitializing(false) is called endReInitingContainer should be called. Otherwise it's possible endReInitingContainer is not invoked. Agreed... fixed this in v009
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 7m 52s trunk passed
        +1 compile 8m 8s trunk passed
        +1 checkstyle 1m 37s trunk passed
        +1 mvnsite 3m 12s trunk passed
        +1 mvneclipse 1m 28s trunk passed
        +1 findbugs 5m 43s trunk passed
        +1 javadoc 1m 59s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 33s the patch passed
        +1 compile 7m 22s the patch passed
        +1 cc 7m 22s the patch passed
        +1 javac 7m 22s the patch passed
        +1 checkstyle 1m 33s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 3m 0s the patch passed
        +1 mvneclipse 1m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 7m 44s the patch passed
        +1 javadoc 2m 13s the patch passed
        +1 unit 0m 34s hadoop-yarn-api in the patch passed.
        +1 unit 2m 35s hadoop-yarn-common in the patch passed.
        +1 unit 0m 32s hadoop-yarn-server-common in the patch passed.
        -1 unit 15m 29s hadoop-yarn-server-nodemanager in the patch failed.
        -1 unit 36m 44s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 8m 42s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        123m 28s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830018/YARN-5609.009.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux fc38b7f04b8e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e5ef51e
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13199/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13199/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 52s trunk passed +1 compile 8m 8s trunk passed +1 checkstyle 1m 37s trunk passed +1 mvnsite 3m 12s trunk passed +1 mvneclipse 1m 28s trunk passed +1 findbugs 5m 43s trunk passed +1 javadoc 1m 59s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 33s the patch passed +1 compile 7m 22s the patch passed +1 cc 7m 22s the patch passed +1 javac 7m 22s the patch passed +1 checkstyle 1m 33s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 3m 0s the patch passed +1 mvneclipse 1m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 7m 44s the patch passed +1 javadoc 2m 13s the patch passed +1 unit 0m 34s hadoop-yarn-api in the patch passed. +1 unit 2m 35s hadoop-yarn-common in the patch passed. +1 unit 0m 32s hadoop-yarn-server-common in the patch passed. -1 unit 15m 29s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 36m 44s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 42s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 123m 28s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830018/YARN-5609.009.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux fc38b7f04b8e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e5ef51e Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13199/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13199/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13199/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        asuresh Arun Suresh added a comment -

        The test case failures are not related to the patch..

        Show
        asuresh Arun Suresh added a comment - The test case failures are not related to the patch..
        Hide
        jianhe Jian He added a comment -

        I saw that the latest patch removed a couple occurrence of container.setIsReInitializing(false)... Shouldn't it be the other way around that endReInitingContainer need to be added where container.setIsReInitializing(false) was called ? Otherwise, the reInitingContainer counter is not reset back to zero if container failed to reInit or happen to succeed before reInit.

        Basically, if we add both metrics.reInitingContainer and metrics.endReInitingContainer into the setIsReInitializing method on top of original, that may work...

        Show
        jianhe Jian He added a comment - I saw that the latest patch removed a couple occurrence of container.setIsReInitializing(false)... Shouldn't it be the other way around that endReInitingContainer need to be added where container.setIsReInitializing(false) was called ? Otherwise, the reInitingContainer counter is not reset back to zero if container failed to reInit or happen to succeed before reInit. Basically, if we add both metrics.reInitingContainer and metrics.endReInitingContainer into the setIsReInitializing method on top of original, that may work...
        Hide
        asuresh Arun Suresh added a comment - - edited

        So, the container.setIsReInitializing(false) statements were remanants of when we did not have the REINITIALIZING state.. We don't really need it anymore.

        if we add both metrics.reInitingContainer and metrics.endReInitingContainer into the setIsReInitializing method on top of original, that may work..

        The setReinitializing(true) was meant to deal with the race condition between the ContainerManager API and the Container. It technically does not signify the start of a reinitialization.. that is actually triggered either in the ReInitializationTransition or in the RetryFailureTransition before rollback...
        That said, im not very particular about it... I can update the patch coincide the metric with the setReInit call from the API... but to be honest, I prefer it the way it is in the last patch.

        Show
        asuresh Arun Suresh added a comment - - edited So, the container.setIsReInitializing(false) statements were remanants of when we did not have the REINITIALIZING state.. We don't really need it anymore. if we add both metrics.reInitingContainer and metrics.endReInitingContainer into the setIsReInitializing method on top of original, that may work.. The setReinitializing(true) was meant to deal with the race condition between the ContainerManager API and the Container. It technically does not signify the start of a reinitialization.. that is actually triggered either in the ReInitializationTransition or in the RetryFailureTransition before rollback... That said, im not very particular about it... I can update the patch coincide the metric with the setReInit call from the API... but to be honest, I prefer it the way it is in the last patch.
        Hide
        jianhe Jian He added a comment -

        I see, I'm ok to not use the same method. The main thing I'm referring is that the containersReIniting counter may not be decremented if the container failed to reInit or succeed on reInit ? It only decrements when the container successfully reInited. This will cause the containersReIniting stay at a positive value forever, right ?

        Show
        jianhe Jian He added a comment - I see, I'm ok to not use the same method. The main thing I'm referring is that the containersReIniting counter may not be decremented if the container failed to reInit or succeed on reInit ? It only decrements when the container successfully reInited. This will cause the containersReIniting stay at a positive value forever, right ?
        Hide
        asuresh Arun Suresh added a comment -

        Hmm.. so how about I put back the old setIsReinitializing(false) but then decrement the metric only if the state has changed from true to false ?

        Show
        asuresh Arun Suresh added a comment - Hmm.. so how about I put back the old setIsReinitializing(false) but then decrement the metric only if the state has changed from true to false ?
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 54s trunk passed
        +1 compile 7m 10s trunk passed
        +1 checkstyle 1m 37s trunk passed
        +1 mvnsite 3m 0s trunk passed
        +1 mvneclipse 1m 26s trunk passed
        +1 findbugs 4m 55s trunk passed
        +1 javadoc 1m 53s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 28s the patch passed
        +1 compile 7m 9s the patch passed
        +1 cc 7m 9s the patch passed
        +1 javac 7m 9s the patch passed
        +1 checkstyle 1m 35s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 3m 2s the patch passed
        +1 mvneclipse 1m 30s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 50s the patch passed
        +1 javadoc 1m 57s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 19s hadoop-yarn-common in the patch passed.
        +1 unit 0m 27s hadoop-yarn-server-common in the patch passed.
        +1 unit 14m 44s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 35m 18s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 45s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        114m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830018/YARN-5609.009.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux b14f3448bff1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6eb700e
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13204/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13204/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 54s trunk passed +1 compile 7m 10s trunk passed +1 checkstyle 1m 37s trunk passed +1 mvnsite 3m 0s trunk passed +1 mvneclipse 1m 26s trunk passed +1 findbugs 4m 55s trunk passed +1 javadoc 1m 53s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 28s the patch passed +1 compile 7m 9s the patch passed +1 cc 7m 9s the patch passed +1 javac 7m 9s the patch passed +1 checkstyle 1m 35s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 3m 2s the patch passed +1 mvneclipse 1m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 50s the patch passed +1 javadoc 1m 57s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 unit 0m 27s hadoop-yarn-server-common in the patch passed. +1 unit 14m 44s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 35m 18s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 45s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 114m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830018/YARN-5609.009.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux b14f3448bff1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13204/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13204/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        yep, sounds good to me.

        Show
        jianhe Jian He added a comment - yep, sounds good to me.
        Hide
        asuresh Arun Suresh added a comment -

        Uploading patch (v010), which is basically (v008) but slight modification based on Jian's request :

        ..so how about I put back the old setIsReinitializing(false) but then decrement the metric only if the state has changed from true to false ?

        Show
        asuresh Arun Suresh added a comment - Uploading patch (v010), which is basically (v008) but slight modification based on Jian's request : ..so how about I put back the old setIsReinitializing(false) but then decrement the metric only if the state has changed from true to false ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 43s trunk passed
        +1 compile 6m 57s trunk passed
        +1 checkstyle 1m 34s trunk passed
        +1 mvnsite 2m 54s trunk passed
        +1 mvneclipse 1m 26s trunk passed
        +1 findbugs 4m 46s trunk passed
        +1 javadoc 1m 54s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 2m 19s the patch passed
        +1 compile 6m 53s the patch passed
        +1 cc 6m 53s the patch passed
        +1 javac 6m 53s the patch passed
        +1 checkstyle 1m 35s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 2m 52s the patch passed
        +1 mvneclipse 1m 23s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 35s the patch passed
        +1 javadoc 1m 52s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        +1 unit 0m 28s hadoop-yarn-server-common in the patch passed.
        -1 unit 14m 53s hadoop-yarn-server-nodemanager in the patch failed.
        -1 unit 33m 42s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 8m 42s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        111m 38s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher
          hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830254/YARN-5609.010.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux b732326773d5 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 14a696f
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13210/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13210/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 43s trunk passed +1 compile 6m 57s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 2m 54s trunk passed +1 mvneclipse 1m 26s trunk passed +1 findbugs 4m 46s trunk passed +1 javadoc 1m 54s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 19s the patch passed +1 compile 6m 53s the patch passed +1 cc 6m 53s the patch passed +1 javac 6m 53s the patch passed +1 checkstyle 1m 35s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 2m 52s the patch passed +1 mvneclipse 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 35s the patch passed +1 javadoc 1m 52s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 0m 28s hadoop-yarn-server-common in the patch passed. -1 unit 14m 53s hadoop-yarn-server-nodemanager in the patch failed. -1 unit 33m 42s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 8m 42s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 111m 38s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher   hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830254/YARN-5609.010.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux b732326773d5 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 14a696f Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt https://builds.apache.org/job/PreCommit-YARN-Build/13210/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13210/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13210/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 59s trunk passed
        +1 compile 7m 40s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 2m 53s trunk passed
        +1 mvneclipse 1m 25s trunk passed
        +1 findbugs 5m 4s trunk passed
        +1 javadoc 1m 59s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 2m 35s the patch passed
        +1 compile 7m 28s the patch passed
        +1 cc 7m 28s the patch passed
        +1 javac 7m 28s the patch passed
        +1 checkstyle 1m 39s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508)
        +1 mvnsite 3m 8s the patch passed
        +1 mvneclipse 1m 27s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 6m 15s the patch passed
        +1 javadoc 1m 53s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        +1 unit 0m 26s hadoop-yarn-server-common in the patch passed.
        +1 unit 14m 43s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 34m 2s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        115m 15s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830254/YARN-5609.010.patch
        JIRA Issue YARN-5609
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 623b78a6a662 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 14a696f
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13212/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13212/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 59s trunk passed +1 compile 7m 40s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 2m 53s trunk passed +1 mvneclipse 1m 25s trunk passed +1 findbugs 5m 4s trunk passed +1 javadoc 1m 59s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 35s the patch passed +1 compile 7m 28s the patch passed +1 cc 7m 28s the patch passed +1 javac 7m 28s the patch passed +1 checkstyle 1m 39s root: The patch generated 0 new + 491 unchanged - 17 fixed = 491 total (was 508) +1 mvnsite 3m 8s the patch passed +1 mvneclipse 1m 27s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 6m 15s the patch passed +1 javadoc 1m 53s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 0m 26s hadoop-yarn-server-common in the patch passed. +1 unit 14m 43s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 34m 2s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 8m 43s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 115m 15s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830254/YARN-5609.010.patch JIRA Issue YARN-5609 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 623b78a6a662 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 14a696f Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13212/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13212/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        +1,
        Committed to trunk and branch-2, thanks Arun Suresh !

        Show
        jianhe Jian He added a comment - +1, Committed to trunk and branch-2, thanks Arun Suresh !
        Hide
        asuresh Arun Suresh added a comment -

        Thanks for the reviews and the commit Jian He

        Show
        asuresh Arun Suresh added a comment - Thanks for the reviews and the commit Jian He
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #10487 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10487/)
        YARN-5609. Expose upgrade and restart API in (jianhe: rev fe644bafe7b4fb5b07f7cf08a7d7044abbf55027)

        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #10487 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10487/ ) YARN-5609 . Expose upgrade and restart API in (jianhe: rev fe644bafe7b4fb5b07f7cf08a7d7044abbf55027) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10488 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10488/)
        Revert "YARN-5609. Expose upgrade and restart API in (arun suresh: rev 2f163cd5cfaf8308f50b6a92c21498b78ada6953)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
          YARN-5609. Expose upgrade and restart API in (arun suresh: rev 4815d024c59cb029e2053d94c7aed33eb8053d3e)
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/CommitResponsePBImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RestartContainerResponsePBImpl.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReInitializeContainerRequest.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReInitializeContainerResponse.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RollbackResponse.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RollbackResponsePBImpl.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReInitializeContainerResponsePBImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/CommitResponse.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RestartContainerResponse.java
        • (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReInitializeContainerRequestPBImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10488 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10488/ ) Revert " YARN-5609 . Expose upgrade and restart API in (arun suresh: rev 2f163cd5cfaf8308f50b6a92c21498b78ada6953) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto YARN-5609 . Expose upgrade and restart API in (arun suresh: rev 4815d024c59cb029e2053d94c7aed33eb8053d3e) (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/CommitResponsePBImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ContainerManagementProtocolPBClientImpl.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RestartContainerResponsePBImpl.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReInitializeContainerRequest.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManagementProtocol.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ReInitializeContainerResponse.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ContainerManagementProtocolPBServiceImpl.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/containermanagement_protocol.proto (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncherImpl.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RollbackResponse.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RollbackResponsePBImpl.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReInitializeContainerResponsePBImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerResourceIncreaseRPC.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAMAuthorization.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/metrics/NodeManagerMetrics.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/CommitResponse.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RestartContainerResponse.java (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/ReInitializeContainerRequestPBImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/NodeManager.java
        Hide
        andrew.wang Andrew Wang added a comment -

        As a reminder, please set a 3.x fix version when committing too. Thanks!

        Show
        andrew.wang Andrew Wang added a comment - As a reminder, please set a 3.x fix version when committing too. Thanks!

          People

          • Assignee:
            asuresh Arun Suresh
            Reporter:
            asuresh Arun Suresh
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development