Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5382

RM does not audit log kill request for active applications

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ClientRMService will audit a kill request but only if it either fails to issue the kill or if the kill is sent to an already finished application. It does not create a log entry when the application is active which is arguably the most important case to audit.

      1. YARN-5382.06.patch
        22 kB
        Vrushali C
      2. YARN-5382.07.patch
        25 kB
        Vrushali C
      3. YARN-5382.08.patch
        25 kB
        Vrushali C
      4. YARN-5382.09.patch
        24 kB
        Vrushali C
      5. YARN-5382.10.patch
        25 kB
        Vrushali C
      6. YARN-5382.11.patch
        24 kB
        Vrushali C
      7. YARN-5382.12.patch
        95 kB
        Vrushali C
      8. YARN-5382.13.patch
        24 kB
        Vrushali C
      9. YARN-5382.14.patch
        24 kB
        Vrushali C
      10. YARN-5382.15.patch
        24 kB
        Vrushali C
      11. YARN-5382-branch-2.7.01.patch
        1 kB
        Vrushali C
      12. YARN-5382-branch-2.7.02.patch
        1 kB
        Vrushali C
      13. YARN-5382-branch-2.7.03.patch
        16 kB
        Vrushali C
      14. YARN-5382-branch-2.7.04.patch
        23 kB
        Vrushali C
      15. YARN-5382-branch-2.7.05.patch
        17 kB
        Vrushali C
      16. YARN-5382-branch-2.7.09.patch
        23 kB
        Vrushali C
      17. YARN-5382-branch-2.7.10.patch
        23 kB
        Vrushali C
      18. YARN-5382-branch-2.7.11.patch
        25 kB
        Vrushali C
      19. YARN-5382-branch-2.7.12.patch
        25 kB
        Vrushali C
      20. YARN-5382-branch-2.7.15.patch
        23 kB
        Vrushali C

        Activity

        Hide
        vrushalic Vrushali C added a comment -

        I think I can add this, will upload a patch shortly

        Show
        vrushalic Vrushali C added a comment - I think I can add this, will upload a patch shortly
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        NGarla_Unused I think Target version 2.7.4 should be more suitable. As per discussion in common-dev for the release 2.7.3, Vinod might have already created RC or he will be creating it today. Basically 2.7.3 checkins are locked unless issue is very critical/blocker.

        Show
        rohithsharma Rohith Sharma K S added a comment - NGarla_Unused I think Target version 2.7.4 should be more suitable. As per discussion in common-dev for the release 2.7.3, Vinod might have already created RC or he will be creating it today. Basically 2.7.3 checkins are locked unless issue is very critical/blocker.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v1.

        Jason Lowe : Is this along the lines of what you had in mind?

        I used a simpler LOG.info and did not use the RMAuditLogger.logSuccess call since I figured we don't know at that point if it is a SUCCESS or not.

        Show
        vrushalic Vrushali C added a comment - Uploading patch v1. Jason Lowe : Is this along the lines of what you had in mind? I used a simpler LOG.info and did not use the RMAuditLogger.logSuccess call since I figured we don't know at that point if it is a SUCCESS or not.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the info missed the mail for 2.7.3 by Vinod ! Have updated back

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the info missed the mail for 2.7.3 by Vinod ! Have updated back
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 58s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 13s branch-2.7 passed
        +1 compile 0m 27s branch-2.7 passed with JDK v1.8.0_91
        +1 compile 0m 31s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 25s branch-2.7 passed
        +1 mvnsite 0m 40s branch-2.7 passed
        +1 mvneclipse 0m 23s branch-2.7 passed
        -1 findbugs 1m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 20s branch-2.7 passed with JDK v1.8.0_91
        +1 javadoc 0m 23s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 28s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_91
        +1 javac 0m 23s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_101
        +1 javac 0m 26s the patch passed
        +1 checkstyle 0m 18s the patch passed
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 1271 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 36s The patch 70 line(s) with tabs.
        +1 findbugs 1m 13s the patch passed
        +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91
        +1 javadoc 0m 21s the patch passed with JDK v1.7.0_101
        -1 unit 54m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
        -1 unit 54m 42s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        129m 54s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818109/YARN-5382-branch-2.7.01.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9d52686af217 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / c35c1e6
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12338/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12338/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 58s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 13s branch-2.7 passed +1 compile 0m 27s branch-2.7 passed with JDK v1.8.0_91 +1 compile 0m 31s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 25s branch-2.7 passed +1 mvnsite 0m 40s branch-2.7 passed +1 mvneclipse 0m 23s branch-2.7 passed -1 findbugs 1m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 20s branch-2.7 passed with JDK v1.8.0_91 +1 javadoc 0m 23s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 28s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_91 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_101 +1 javac 0m 26s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1271 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 36s The patch 70 line(s) with tabs. +1 findbugs 1m 13s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 21s the patch passed with JDK v1.7.0_101 -1 unit 54m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 54m 42s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 129m 54s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818109/YARN-5382-branch-2.7.01.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9d52686af217 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / c35c1e6 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12338/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12338/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12338/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for the patch, Vrushali C!

        I don't think we should do a vanilla log here. It won't appear in the audit log stream, and that's where users are going to look if they have configured the audit log to a separate file (as we have). We should simply audit log a success here, as for all practical purposes it is a success. We verified the app ID is valid, active, and the user has permissions to do so. At this point the only race is whether the app completes before the kill arrives, but we already log success when the kill arrives after the app completes, so why not here as well?

        Show
        jlowe Jason Lowe added a comment - Thanks for the patch, Vrushali C ! I don't think we should do a vanilla log here. It won't appear in the audit log stream, and that's where users are going to look if they have configured the audit log to a separate file (as we have). We should simply audit log a success here, as for all practical purposes it is a success. We verified the app ID is valid, active, and the user has permissions to do so. At this point the only race is whether the app completes before the kill arrives, but we already log success when the kill arrives after the app completes, so why not here as well?
        Hide
        vrushalic Vrushali C added a comment - - edited

        Thanks Jason Lowe! Uploading patch v2 which uses RMAuditLogger.

        But with this, I now see two exactly same log messages in the RM log since the forceKillApplication function is entered twice.

        I was wondering if the message can be made slightly different? But that would mean adding something to RMAuditLogger itself to log a different style of message.

        From my pseudo-distributed setup:

        
        [hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni ClientRMService logs/yarn-vchannapattan-resourcemanager-cchannapattan.log | grep "Kill App" |  grep  application_1468608647317_0001
        
        logs/yarn-vchannapattan-resourcemanager-channapattan.log:348:2016-07-15 11:51:10,773 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan	IP=127.0.0.1	OPERATION=Kill Application Request	TARGET=ClientRMService	RESULT=SUCCESS	APPID=application_1468608647317_0001
        
        logs/yarn-vchannapattan-resourcemanager-tw-mbp13-channapattan.log:387:2016-07-15 11:51:10,987 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan	IP=127.0.0.1	OPERATION=Kill Application Request	TARGET=ClientRMService	RESULT=SUCCESS	APPID=application_1468608647317_0001
        
        [ hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
        
        Show
        vrushalic Vrushali C added a comment - - edited Thanks Jason Lowe ! Uploading patch v2 which uses RMAuditLogger. But with this, I now see two exactly same log messages in the RM log since the forceKillApplication function is entered twice. I was wondering if the message can be made slightly different? But that would mean adding something to RMAuditLogger itself to log a different style of message. From my pseudo-distributed setup: [hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni ClientRMService logs/yarn-vchannapattan-resourcemanager-cchannapattan.log | grep "Kill App" | grep application_1468608647317_0001 logs/yarn-vchannapattan-resourcemanager-channapattan.log:348:2016-07-15 11:51:10,773 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan IP=127.0.0.1 OPERATION=Kill Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1468608647317_0001 logs/yarn-vchannapattan-resourcemanager-tw-mbp13-channapattan.log:387:2016-07-15 11:51:10,987 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan IP=127.0.0.1 OPERATION=Kill Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1468608647317_0001 [ hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 26s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 8s branch-2.7 passed
        +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_91
        +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 21s branch-2.7 passed
        +1 mvnsite 0m 33s branch-2.7 passed
        +1 mvneclipse 0m 17s branch-2.7 passed
        -1 findbugs 1m 3s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_91
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 24s the patch passed with JDK v1.8.0_91
        +1 javac 0m 24s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_101
        +1 javac 0m 26s the patch passed
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1620 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 39s The patch 74 line(s) with tabs.
        +1 findbugs 1m 12s the patch passed
        +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91
        +1 javadoc 0m 21s the patch passed with JDK v1.7.0_101
        -1 unit 49m 23s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
        -1 unit 50m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        116m 47s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818226/YARN-5382-branch-2.7.02.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c1724d50a8de 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / c35c1e6
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12343/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12343/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 8s branch-2.7 passed +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_91 +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 21s branch-2.7 passed +1 mvnsite 0m 33s branch-2.7 passed +1 mvneclipse 0m 17s branch-2.7 passed -1 findbugs 1m 3s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_91 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 27s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_91 +1 javac 0m 24s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_101 +1 javac 0m 26s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1620 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 39s The patch 74 line(s) with tabs. +1 findbugs 1m 12s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 21s the patch passed with JDK v1.7.0_101 -1 unit 49m 23s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 50m 34s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 116m 47s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818226/YARN-5382-branch-2.7.02.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c1724d50a8de 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / c35c1e6 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12343/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12343/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12343/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Ah, I see. The client is continuing to issue the kill request until the application is reported as killed. Unfortunately if the client does not linger until the kill is reported completely successful then we can end up in a situation where the RM does end up killing the app based on the initial request but it is never logged anywhere.

        One thing we could do is no longer audit killing of completed applications (it's not that interesting anyway) and only audit the killing of active apps. We could check the app state and avoid logging for any app that is completed or in the process of completing (i.e.: state is FINAL_SAVING, KILLING, etc.). Then for most cases we should only get a single audit log unless the RM is slow at dispatching events. Similarly we could set a flag on the application as being killed and only log if it hasn't been flagged.

        Looks like this was introduced in YARN-1446. Pinging Vinod Kumar Vavilapalli and Jian He in case they have opinions on how this should be fixed.

        Show
        jlowe Jason Lowe added a comment - Ah, I see. The client is continuing to issue the kill request until the application is reported as killed. Unfortunately if the client does not linger until the kill is reported completely successful then we can end up in a situation where the RM does end up killing the app based on the initial request but it is never logged anywhere. One thing we could do is no longer audit killing of completed applications (it's not that interesting anyway) and only audit the killing of active apps. We could check the app state and avoid logging for any app that is completed or in the process of completing (i.e.: state is FINAL_SAVING, KILLING, etc.). Then for most cases we should only get a single audit log unless the RM is slow at dispatching events. Similarly we could set a flag on the application as being killed and only log if it hasn't been flagged. Looks like this was introduced in YARN-1446 . Pinging Vinod Kumar Vavilapalli and Jian He in case they have opinions on how this should be fixed.
        Hide
        jianhe Jian He added a comment -

        How about move the audit log in RMApp#AppKilledTransition to avoid explicitly check the state and an additional flag?

        Show
        jianhe Jian He added a comment - How about move the audit log in RMApp#AppKilledTransition to avoid explicitly check the state and an additional flag?
        Hide
        jlowe Jason Lowe added a comment -

        I like the general idea, but I'm not sure a literal move of the audit log to the transition will work. The audit log will try to log the remote IP of the caller, but at the AppKilledTransition we're no longer in an RPC context so there's no remote caller. The basic information is actually in the kill message after YARN-5053, but not in a way that we can pull apart and pass as individual pieces of information to the audit logger (e.g.: user, remote IP, etc.). We could extend the kill event to optionally contain those bits then extend the audit logger API so we can manually specify the user and remote IP rather than have the audit logger always assume it can get them on its own.

        Show
        jlowe Jason Lowe added a comment - I like the general idea, but I'm not sure a literal move of the audit log to the transition will work. The audit log will try to log the remote IP of the caller, but at the AppKilledTransition we're no longer in an RPC context so there's no remote caller. The basic information is actually in the kill message after YARN-5053 , but not in a way that we can pull apart and pass as individual pieces of information to the audit logger (e.g.: user, remote IP, etc.). We could extend the kill event to optionally contain those bits then extend the audit logger API so we can manually specify the user and remote IP rather than have the audit logger always assume it can get them on its own.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jason Lowe and Jian He! Will update the patch to include auditing of killing of active apps only.

        Show
        vrushalic Vrushali C added a comment - Thanks Jason Lowe and Jian He ! Will update the patch to include auditing of killing of active apps only.
        Hide
        jlowe Jason Lowe added a comment -

        Will update the patch to include auditing of killing of active apps only.

        Actually I think we should go with Jian's suggestion. Auditing active apps could still generate duplicate events if the event dispatch is delayed, and Jian's suggestion means we'll only log it once when the app transitions from active to starting the kill processing. We will need to enhance the kill event to include the requesting user and remote IP address so it can be audit logged properly within the RMAppImpl transition.

        Show
        jlowe Jason Lowe added a comment - Will update the patch to include auditing of killing of active apps only. Actually I think we should go with Jian's suggestion. Auditing active apps could still generate duplicate events if the event dispatch is delayed, and Jian's suggestion means we'll only log it once when the app transitions from active to starting the kill processing. We will need to enhance the kill event to include the requesting user and remote IP address so it can be audit logged properly within the RMAppImpl transition.
        Hide
        vrushalic Vrushali C added a comment -

        I see, thanks, sounds good. Will do that.

        Show
        vrushalic Vrushali C added a comment - I see, thanks, sounds good. Will do that.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v3.

        Changes made:

        • Added a RMAppKillEvent that contains the callerUGI and the remote IP
        • Added a method (log success) to print the ip (I am wondering if we need a corresponding log failure method?)
        • Updated the diagnostic message be similar to YARN-5053
        • Tested on local machine in a pseudo distributed setup as well through unit test TestRMAppTransitions.

        I see the following in the RM log when I ran and killed a sleep job:

        
        vrushali[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni auditlogger logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log  | grep -i "kill app"
        logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log:204:2016-07-21 00:53:07,493 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vvrushali    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087565504_0001
        [hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
        

        I see the following in the unit test log:

        
        [hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$ grep  "Kill Application " target/surefire-reports/org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions-output.txt
        2016-07-21 00:54:02,761 INFO  [Thread-5] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0001
        2016-07-21 00:54:03,086 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0007
        2016-07-21 00:54:03,155 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0009
        2016-07-21 00:54:03,258 INFO  [Thread-53] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0012
        2016-07-21 00:54:03,328 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0014
        2016-07-21 00:54:03,587 INFO  [Thread-102] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0026
        2016-07-21 00:54:03,754 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0032
        2016-07-21 00:54:03,807 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0034
        2016-07-21 00:54:03,882 INFO  [Thread-150] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0037
        2016-07-21 00:54:03,926 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0039
        [hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$
        
        
        Show
        vrushalic Vrushali C added a comment - Uploading patch v3. Changes made: Added a RMAppKillEvent that contains the callerUGI and the remote IP Added a method (log success) to print the ip (I am wondering if we need a corresponding log failure method?) Updated the diagnostic message be similar to YARN-5053 Tested on local machine in a pseudo distributed setup as well through unit test TestRMAppTransitions. I see the following in the RM log when I ran and killed a sleep job: vrushali[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni auditlogger logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log  | grep -i "kill app" logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log:204:2016-07-21 00:53:07,493 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vvrushali    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087565504_0001 [hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ I see the following in the unit test log: [hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$ grep  "Kill Application " target/surefire-reports/org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions-output.txt 2016-07-21 00:54:02,761 INFO  [ Thread -5] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0001 2016-07-21 00:54:03,086 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0007 2016-07-21 00:54:03,155 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0009 2016-07-21 00:54:03,258 INFO  [ Thread -53] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0012 2016-07-21 00:54:03,328 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0014 2016-07-21 00:54:03,587 INFO  [ Thread -102] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0026 2016-07-21 00:54:03,754 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0032 2016-07-21 00:54:03,807 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0034 2016-07-21 00:54:03,882 INFO  [ Thread -150] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0037 2016-07-21 00:54:03,926 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0039 [hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 15m 53s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 7s branch-2.7 passed
        +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_91
        +1 compile 0m 31s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 28s branch-2.7 passed
        +1 mvnsite 0m 38s branch-2.7 passed
        +1 mvneclipse 0m 20s branch-2.7 passed
        -1 findbugs 1m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 22s branch-2.7 passed with JDK v1.8.0_91
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 29s the patch passed with JDK v1.8.0_91
        +1 javac 0m 28s the patch passed
        +1 compile 0m 28s the patch passed with JDK v1.7.0_101
        +1 javac 0m 28s the patch passed
        -1 checkstyle 0m 25s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 39 new + 690 unchanged - 0 fixed = 729 total (was 690)
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 1977 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 48s The patch 82 line(s) with tabs.
        +1 findbugs 1m 17s the patch passed
        -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
        -1 javadoc 0m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
        -1 unit 54m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
        -1 unit 53m 58s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        145m 18s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819282/YARN-5382-branch-2.7.03.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 46f7ae067a56 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 1b8641a
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12437/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12437/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 53s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 7s branch-2.7 passed +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_91 +1 compile 0m 31s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 28s branch-2.7 passed +1 mvnsite 0m 38s branch-2.7 passed +1 mvneclipse 0m 20s branch-2.7 passed -1 findbugs 1m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 22s branch-2.7 passed with JDK v1.8.0_91 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 27s the patch passed +1 compile 0m 29s the patch passed with JDK v1.8.0_91 +1 javac 0m 28s the patch passed +1 compile 0m 28s the patch passed with JDK v1.7.0_101 +1 javac 0m 28s the patch passed -1 checkstyle 0m 25s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 39 new + 690 unchanged - 0 fixed = 729 total (was 690) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1977 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 48s The patch 82 line(s) with tabs. +1 findbugs 1m 17s the patch passed -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 javadoc 0m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) -1 unit 54m 51s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 53m 58s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 145m 18s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819282/YARN-5382-branch-2.7.03.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 46f7ae067a56 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 1b8641a Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/12437/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12437/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12437/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        Hi Vrushali C
        Thanks for sharing the patch. Few comments

        1. RMAppKillEvent is now used as a special child event when client invokes kill command on an app for extra audit logging. For other app kill scenarios within scheduler, we still use RMAppEvent only. So could we add more clarity to the event name to indicate it does only an extra logging. Else we can use this new RMAppKillEvent for all internal callers and pass EMPTY message.
        2. RMAppKillEvent may not need to take RMAppEventType.KILL as an argument. Since its a kill event, we can consider RMAppEventType.KILL as event while invoking super from RMAppKillEvent constructor.

        Show
        sunilg Sunil G added a comment - Hi Vrushali C Thanks for sharing the patch. Few comments 1. RMAppKillEvent is now used as a special child event when client invokes kill command on an app for extra audit logging. For other app kill scenarios within scheduler, we still use RMAppEvent only. So could we add more clarity to the event name to indicate it does only an extra logging. Else we can use this new RMAppKillEvent for all internal callers and pass EMPTY message. 2. RMAppKillEvent may not need to take RMAppEventType.KILL as an argument. Since its a kill event, we can consider RMAppEventType.KILL as event while invoking super from RMAppKillEvent constructor.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Sunil G! Yes, let me update the code on both of the points mentioned. Thanks..

        Show
        vrushalic Vrushali C added a comment - Thanks Sunil G ! Yes, let me update the code on both of the points mentioned. Thanks..
        Hide
        vrushalic Vrushali C added a comment -

        Attaching patch v4 that addresses review suggestions by Sunil G

        • updated the scheduler app kill calls to create RMAppKillEvent
        • updated the RMAppKillEvent class to set the event type to KILL in constructor
        Show
        vrushalic Vrushali C added a comment - Attaching patch v4 that addresses review suggestions by Sunil G updated the scheduler app kill calls to create RMAppKillEvent updated the RMAppKillEvent class to set the event type to KILL in constructor
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 16m 35s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 36s branch-2.7 passed
        +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 34s branch-2.7 passed
        +1 mvnsite 0m 36s branch-2.7 passed
        +1 mvneclipse 0m 16s branch-2.7 passed
        -1 findbugs 1m 4s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 19s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 25s the patch passed with JDK v1.8.0_101
        +1 javac 0m 25s the patch passed
        +1 compile 0m 27s the patch passed with JDK v1.7.0_101
        +1 javac 0m 27s the patch passed
        -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 45 new + 1278 unchanged - 2 fixed = 1323 total (was 1280)
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 2355 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 57s The patch 86 line(s) with tabs.
        +1 findbugs 1m 13s the patch passed
        -1 javadoc 0m 16s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 javadoc 0m 21s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
        -1 unit 56m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 unit 53m 33s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        143m 57s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819479/YARN-5382-branch-2.7.04.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e3258f6d0436 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 2a5b8e5
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12448/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12448/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 35s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 36s branch-2.7 passed +1 compile 0m 26s branch-2.7 passed with JDK v1.8.0_101 +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 34s branch-2.7 passed +1 mvnsite 0m 36s branch-2.7 passed +1 mvneclipse 0m 16s branch-2.7 passed -1 findbugs 1m 4s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 19s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 27s the patch passed +1 compile 0m 25s the patch passed with JDK v1.8.0_101 +1 javac 0m 25s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_101 +1 javac 0m 27s the patch passed -1 checkstyle 0m 32s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 45 new + 1278 unchanged - 2 fixed = 1323 total (was 1280) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 2355 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 57s The patch 86 line(s) with tabs. +1 findbugs 1m 13s the patch passed -1 javadoc 0m 16s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 javadoc 0m 21s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) -1 unit 56m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 unit 53m 33s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 143m 57s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819479/YARN-5382-branch-2.7.04.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e3258f6d0436 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 2a5b8e5 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12448/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12448/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12448/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        I see one javadoc warning and a few checkstyle warnings in the patch - will fix that along with any other review comments that might come up.
        The other failures in the build (the findbugs, javadoc, whitespace) are not due to this patch. The tests that have failed I believe are unrelated to the changes in patch.

        Show
        vrushalic Vrushali C added a comment - I see one javadoc warning and a few checkstyle warnings in the patch - will fix that along with any other review comments that might come up. The other failures in the build (the findbugs, javadoc, whitespace) are not due to this patch. The tests that have failed I believe are unrelated to the changes in patch.
        Hide
        sunilg Sunil G added a comment -

        Hi Vrushali C
        Thanks for updating the patch. I think there is a slight misunderstanding, I guess my comment was not very clear. What I was trying convey is that If we choose to use RMAppKillEvent for all cases, I think for internal kill events (within scheduler such as AbstractYarnScheduler and CapacityScheduler) we do not need to do audit logging. So we can either pass a flag to log or can pass username/ip as empty to take the logging decision.

        I think for internal scheduler kill events, we will not be able to derive username or remote ip correctly as its not user driven. Thoughts?

        Show
        sunilg Sunil G added a comment - Hi Vrushali C Thanks for updating the patch. I think there is a slight misunderstanding, I guess my comment was not very clear. What I was trying convey is that If we choose to use RMAppKillEvent for all cases, I think for internal kill events (within scheduler such as AbstractYarnScheduler and CapacityScheduler ) we do not need to do audit logging. So we can either pass a flag to log or can pass username/ip as empty to take the logging decision. I think for internal scheduler kill events, we will not be able to derive username or remote ip correctly as its not user driven. Thoughts?
        Hide
        vrushalic Vrushali C added a comment -

        Oh I see. I will update the patch. Thanks Sunil G

        Show
        vrushalic Vrushali C added a comment - Oh I see. I will update the patch. Thanks Sunil G
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v5.

        I modified the patch to add more clarity to the event name to indicate it is to be used for extra logging only. Accordingly all internal invocations for app kill will still use AppEvent. I figured this was better instead of assigning nulls to callerUGI or remoteIP in the new class RMAppKillEvent in the previous patch.

        Thanks Sunil G , Jason Lowe for the discussion so far. Please let me know what you think of this patch.

        Show
        vrushalic Vrushali C added a comment - Uploading patch v5. I modified the patch to add more clarity to the event name to indicate it is to be used for extra logging only. Accordingly all internal invocations for app kill will still use AppEvent. I figured this was better instead of assigning nulls to callerUGI or remoteIP in the new class RMAppKillEvent in the previous patch. Thanks Sunil G , Jason Lowe for the discussion so far. Please let me know what you think of this patch.
        Hide
        sunilg Sunil G added a comment -

        Yes Vrushali C. Thanks for updating the patch.
        I think the current approach taken in this patch is fine. Patch generally looks fine for me.

        Show
        sunilg Sunil G added a comment - Yes Vrushali C . Thanks for updating the patch. I think the current approach taken in this patch is fine. Patch generally looks fine for me.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Sunil G!

        Jason Lowe Would appreciate your feedback as well.. Thanks!

        Show
        vrushalic Vrushali C added a comment - Thanks Sunil G ! Jason Lowe Would appreciate your feedback as well.. Thanks!
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for the update, Vrushali!

        I should have said this earlier: normally we review patches against trunk first, since that's where the change must go before it goes anywhere else. We can only put this into 2.7 once it's also in trunk, branch-2, and branch-2.8, otherwise we risk releasing a fix/feature that disappears in later releases.

        createSuccessLog should build on other versions rather than replicate the code. For example, now that we have an overload that takes the IP address as a string, the one that doesn't take an IP address should get the IP address as a string and call the other version.

        Nit: ClientRMService#forceKillApplication is calling Server.getRemoteAddress three times which is wasteful, we should just cache this in a local.

        Would RMAppKillByClientEvent be a more meaningful name than RMAppKillLogEvent?

        Should we be passing around an InetAddress in the event and audit log overloads instead of a String for improved type safety? Not a must-fix, just wondering if it would be clearer.

        I'm not sure we should log empty strings for values that aren't provided. It would probably be better to log "null" which is what it will already do for user names, for example, if we just let the null pass through. Not sure if audit log parsers will properly parse if there isn't some kind of corresponding token listed for a key.

        Existing audit log code will not log a key for an IP address if it can't obtain it. This code will log an IP key with no value. May be a reason to pass the InetAddress through and let the audit logger decide whether to add the key if the value is non-null and it can obtain the address.

        In the tests, why do we need a fooUser and a testUGI? I think we only need one of these. fooUser is created and then only used to get information to create testUGI, and I don't see why we can't just create testUGI directly. Also that UGI could be a static final "constant" in the test class rather than having each test method replicate the code.

        Show
        jlowe Jason Lowe added a comment - Thanks for the update, Vrushali! I should have said this earlier: normally we review patches against trunk first, since that's where the change must go before it goes anywhere else. We can only put this into 2.7 once it's also in trunk, branch-2, and branch-2.8, otherwise we risk releasing a fix/feature that disappears in later releases. createSuccessLog should build on other versions rather than replicate the code. For example, now that we have an overload that takes the IP address as a string, the one that doesn't take an IP address should get the IP address as a string and call the other version. Nit: ClientRMService#forceKillApplication is calling Server.getRemoteAddress three times which is wasteful, we should just cache this in a local. Would RMAppKillByClientEvent be a more meaningful name than RMAppKillLogEvent? Should we be passing around an InetAddress in the event and audit log overloads instead of a String for improved type safety? Not a must-fix, just wondering if it would be clearer. I'm not sure we should log empty strings for values that aren't provided. It would probably be better to log "null" which is what it will already do for user names, for example, if we just let the null pass through. Not sure if audit log parsers will properly parse if there isn't some kind of corresponding token listed for a key. Existing audit log code will not log a key for an IP address if it can't obtain it. This code will log an IP key with no value. May be a reason to pass the InetAddress through and let the audit logger decide whether to add the key if the value is non-null and it can obtain the address. In the tests, why do we need a fooUser and a testUGI? I think we only need one of these. fooUser is created and then only used to get information to create testUGI, and I don't see why we can't just create testUGI directly. Also that UGI could be a static final "constant" in the test class rather than having each test method replicate the code.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jason Lowe! I will make the changes and upload an updated patch for trunk as well as branch-2.

        Show
        vrushalic Vrushali C added a comment - Thanks Jason Lowe ! I will make the changes and upload an updated patch for trunk as well as branch-2.
        Hide
        jlowe Jason Lowe added a comment -

        Note that if the trunk patch applies as-is to branch-2 (as I suspect it will) then there's no need for a separate patch.

        Show
        jlowe Jason Lowe added a comment - Note that if the trunk patch applies as-is to branch-2 (as I suspect it will) then there's no need for a separate patch.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v6 (against trunk).

        • Modified the name of the new Event class to be RMAppKillByClientLogEvent
        • Changed the ip address parameter to be of type InetAddress instead of String
        • Updated the unit tests in TestRMAppTransitions and added more to the tests in TestRMAuditLogger to include these changes
        Show
        vrushalic Vrushali C added a comment - Uploading patch v6 (against trunk). Modified the name of the new Event class to be RMAppKillByClientLogEvent Changed the ip address parameter to be of type InetAddress instead of String Updated the unit tests in TestRMAppTransitions and added more to the tests in TestRMAuditLogger to include these changes
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 5s trunk passed
        +1 compile 0m 33s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 39s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 58s trunk passed
        +1 javadoc 0m 21s trunk passed
        -1 mvninstall 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 compile 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 javac 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 9 new + 214 unchanged - 1 fixed = 223 total (was 215)
        -1 mvnsite 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 0m 18s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 javadoc 0m 19s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 1 new + 963 unchanged - 0 fixed = 964 total (was 963)
        -1 unit 0m 18s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 14s The patch does not generate ASF License warnings.
        13m 40s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820819/YARN-5382.06.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 59ee83361456 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ce3d68e
        Default Java 1.8.0_101
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        compile https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javac https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12550/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12550/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 5s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 58s trunk passed +1 javadoc 0m 21s trunk passed -1 mvninstall 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. -1 javac 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 9 new + 214 unchanged - 1 fixed = 223 total (was 215) -1 mvnsite 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 18s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 19s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 1 new + 963 unchanged - 0 fixed = 964 total (was 963) -1 unit 0m 18s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 13m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820819/YARN-5382.06.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 59ee83361456 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ce3d68e Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12550/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12550/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12550/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        The previous patch v06 was incomplete. Uploading new patch v7 which includes the missing class

        Show
        vrushalic Vrushali C added a comment - The previous patch v06 was incomplete. Uploading new patch v7 which includes the missing class
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 7s trunk passed
        +1 compile 0m 39s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 48s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 1m 6s trunk passed
        +1 javadoc 0m 24s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 0m 38s the patch passed
        +1 javac 0m 38s the patch passed
        -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 10 new + 214 unchanged - 1 fixed = 224 total (was 215)
        +1 mvnsite 0m 43s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 14s the patch passed
        -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 36m 46s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        53m 59s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820851/YARN-5382.07.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e00ada0eb0f3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8ebf2e9
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12555/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12555/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12555/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12555/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 7s trunk passed +1 compile 0m 39s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 48s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 24s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 0m 38s the patch passed +1 javac 0m 38s the patch passed -1 checkstyle 0m 22s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 10 new + 214 unchanged - 1 fixed = 224 total (was 215) +1 mvnsite 0m 43s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 14s the patch passed -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 36m 46s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 53m 59s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820851/YARN-5382.07.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e00ada0eb0f3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8ebf2e9 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12555/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12555/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12555/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12555/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Will fix the checkstyle issues shortly

        Show
        vrushalic Vrushali C added a comment - Will fix the checkstyle issues shortly
        Hide
        vrushalic Vrushali C added a comment -

        Uploading v8 that addresses previous checkstyle warnings

        Show
        vrushalic Vrushali C added a comment - Uploading v8 that addresses previous checkstyle warnings
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 23s trunk passed
        +1 compile 0m 39s trunk passed
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 0m 44s trunk passed
        +1 mvneclipse 0m 20s trunk passed
        +1 findbugs 1m 7s trunk passed
        +1 javadoc 0m 23s trunk passed
        +1 mvninstall 0m 35s the patch passed
        +1 compile 0m 31s the patch passed
        +1 javac 0m 31s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215)
        +1 mvnsite 0m 38s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 2s the patch passed
        -1 javadoc 0m 18s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 36m 35s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        53m 24s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821106/YARN-5382.08.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux abab02a9aa80 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 95f2b98
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12569/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12569/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12569/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12569/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 23s trunk passed +1 compile 0m 39s trunk passed +1 checkstyle 0m 24s trunk passed +1 mvnsite 0m 44s trunk passed +1 mvneclipse 0m 20s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 23s trunk passed +1 mvninstall 0m 35s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) +1 mvnsite 0m 38s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 2s the patch passed -1 javadoc 0m 18s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 36m 35s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 53m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821106/YARN-5382.08.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux abab02a9aa80 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 95f2b98 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12569/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12569/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12569/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12569/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Wondering what I can do to fix the checkstyle warning for

        ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java:91:  static String createSuccessLog(String user, String operation, String target,:17: More than 7 parameters (found 9).
        

        There were earlier 8 parameters, this patch adds one more (ip).

        Show
        vrushalic Vrushali C added a comment - Wondering what I can do to fix the checkstyle warning for ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java:91: static String createSuccessLog( String user, String operation, String target,:17: More than 7 parameters (found 9). There were earlier 8 parameters, this patch adds one more (ip).
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for updating the patch!

        Nit: RMAppKillByClientLogEvent should just be RMAppKillByClientEvent. The event represents an application kill request coming from a client. The logging is an implementation detail of how the event will be processed. Event classes should be named after what event they represent rather than how the event will be processed, otherwise when the processing details change then the event name becomes misleading/confusing.

        The createSuccessLog version without an IP parameter should just call the createSuccessLog version with the parameter. Instead of passing null, it can call getRemoteIp. Then createSuccessLog with an IP parameter can be simplified to just logging an IP key if the ip is not null. That makes it consistent with how it treats the other, possibly-null parameters as well.

        Otherwise patch looks good. Could you also provide patches for 2.8 and 2.7?

        Show
        jlowe Jason Lowe added a comment - Thanks for updating the patch! Nit: RMAppKillByClientLogEvent should just be RMAppKillByClientEvent. The event represents an application kill request coming from a client. The logging is an implementation detail of how the event will be processed. Event classes should be named after what event they represent rather than how the event will be processed, otherwise when the processing details change then the event name becomes misleading/confusing. The createSuccessLog version without an IP parameter should just call the createSuccessLog version with the parameter. Instead of passing null, it can call getRemoteIp. Then createSuccessLog with an IP parameter can be simplified to just logging an IP key if the ip is not null. That makes it consistent with how it treats the other, possibly-null parameters as well. Otherwise patch looks good. Could you also provide patches for 2.8 and 2.7?
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jason Lowe , yes will update the trunk patch with your suggestions and provide the patches for 2.7 and 2.8 as well in my next upload.

        Show
        vrushalic Vrushali C added a comment - Thanks Jason Lowe , yes will update the trunk patch with your suggestions and provide the patches for 2.7 and 2.8 as well in my next upload.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v9 for trunk and 2.7

        Show
        vrushalic Vrushali C added a comment - Uploading patch v9 for trunk and 2.7
        Hide
        jianhe Jian He added a comment -

        With this patch, looks like killing a normal running app will cause RM audit three times.
        1. In KillAttemptTransition
        2. AppKilledTransition
        3. In ClientRMService after the app is successfully killed.

        Vrushali C is this what you see in your testing?
        Jason Lowe, what is the user's usual expectation. Does user expect audit logging both before killing and after killing successfully ?

        Show
        jianhe Jian He added a comment - With this patch, looks like killing a normal running app will cause RM audit three times. 1. In KillAttemptTransition 2. AppKilledTransition 3. In ClientRMService after the app is successfully killed. Vrushali C is this what you see in your testing? Jason Lowe , what is the user's usual expectation. Does user expect audit logging both before killing and after killing successfully ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 33s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 54s trunk passed
        +1 compile 0m 35s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 39s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 1m 2s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 31s the patch passed
        +1 javac 0m 31s the patch passed
        -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215)
        +1 mvnsite 0m 38s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 6s the patch passed
        -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 33m 9s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        49m 32s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821499/YARN-5382.09.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux cb1d6d097cd2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9f473cf
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12608/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12608/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12608/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12608/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 33s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 54s trunk passed +1 compile 0m 35s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 1m 2s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) +1 mvnsite 0m 38s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 6s the patch passed -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 33m 9s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 49m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821499/YARN-5382.09.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cb1d6d097cd2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9f473cf Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12608/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12608/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12608/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12608/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        Does user expect audit logging both before killing and after killing successfully ?

        Ideally from the ClientRMService perspective it should be logged when the request comes in, just like web servers audit log requests they serve. Unfortunately the polling-for-killed logic makes this messy to implement cleanly, so logging once when the app is killed would be the next best option, IMHO.

        Sorry I missed the fact that the success logging in ClientRMService was still there. At some point it was removed, but I missed it didn't stay that way. I also missed that AttemptKilledTransition and AppKilledTransition can both be triggered for an app being killed.

        Show
        jlowe Jason Lowe added a comment - Does user expect audit logging both before killing and after killing successfully ? Ideally from the ClientRMService perspective it should be logged when the request comes in, just like web servers audit log requests they serve. Unfortunately the polling-for-killed logic makes this messy to implement cleanly, so logging once when the app is killed would be the next best option, IMHO. Sorry I missed the fact that the success logging in ClientRMService was still there. At some point it was removed, but I missed it didn't stay that way. I also missed that AttemptKilledTransition and AppKilledTransition can both be triggered for an app being killed.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jian He and Jason Lowe.
        Apologies, I somehow missed that the success logging in ClientRMService on the isAppFinalStateStored snuck back in. I think that happened when I rebased to latest during one of the patches. Will remove it now.

        Jian He, Would it be then okay to keep the logging in RMAppImpl#AppKilledTransition as well as ClientRMService? Will remove the one in RMAppImpl#KillAttemptTransition.

        Show
        vrushalic Vrushali C added a comment - Thanks Jian He and Jason Lowe . Apologies, I somehow missed that the success logging in ClientRMService on the isAppFinalStateStored snuck back in. I think that happened when I rebased to latest during one of the patches. Will remove it now. Jian He , Would it be then okay to keep the logging in RMAppImpl#AppKilledTransition as well as ClientRMService? Will remove the one in RMAppImpl#KillAttemptTransition.
        Hide
        jlowe Jason Lowe added a comment -

        If we keep the kill success logging in both a transition and in ClientRMService then we'll get two audit logs instead of one.

        I also don't think it's as simple as removing the one from KillAttemptTransition since then we won't get a log if the RM fails over just as it saved the killed state of an app but before it executed the AppKilledTransition. IMHO we need to log it once, before we enter the FINAL_SAVING state to record the killed transition. Then we might get two audit logs during a failover (one on each RM instance) but that's far preferable to none.

        Show
        jlowe Jason Lowe added a comment - If we keep the kill success logging in both a transition and in ClientRMService then we'll get two audit logs instead of one. I also don't think it's as simple as removing the one from KillAttemptTransition since then we won't get a log if the RM fails over just as it saved the killed state of an app but before it executed the AppKilledTransition. IMHO we need to log it once, before we enter the FINAL_SAVING state to record the killed transition. Then we might get two audit logs during a failover (one on each RM instance) but that's far preferable to none.
        Hide
        vrushalic Vrushali C added a comment - - edited

        So with the last uploaded patch v9 on branch-2.7 (https://issues.apache.org/jira/secure/attachment/12821498/YARN-5382-branch-2.7.09.patch) . The 2.7 patch does not have logging in ClientRMService.

        I see only one audit log message when I ran a sleep job and killed it on pseudo-distributed setup on my laptop.

        [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -i Rmauditlogg logs/yarn-vchannapattan-resourcemanager-machine13-channapattan.log  | grep -i Kill
        2016-08-02 14:00:19,186 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan	IP=127.0.0.1	OPERATION=Kill Application Request	TARGET=RMAppImpl	RESULT=SUCCESS	APPID=application_1470171585834_0001
        2016-08-02 14:00:19,195 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan	OPERATION=Application Finished - Killed	TARGET=RMAppManager	RESULT=SUCCES	APPID=application_1470171585834_0001
        [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
        

        On another window:

        [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4-SNAPSHOT.jar sleep -m 100 -r 1000 -mt 300 -rt 300
        16/08/02 14:00:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        16/08/02 14:00:03 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
        16/08/02 14:00:05 INFO mapreduce.JobSubmitter: number of splits:100
        16/08/02 14:00:05 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470171585834_0001
        16/08/02 14:00:05 INFO impl.YarnClientImpl: Submitted application application_1470171585834_0001
        16/08/02 14:00:05 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1470171585834_0001/
        16/08/02 14:00:05 INFO mapreduce.Job: Running job: job_1470171585834_0001
        ^C
        [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/yarn application -kill  application_1470171585834_0001
        16/08/02 14:00:16 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
        16/08/02 14:00:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        Killing application application_1470171585834_0001
        16/08/02 14:00:19 INFO impl.YarnClientImpl: Killed application application_1470171585834_0001
        [t-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
        

        I need to update the patch for trunk to include removal of the audit logging upon isAppFinalStateStored check.

        Show
        vrushalic Vrushali C added a comment - - edited So with the last uploaded patch v9 on branch-2.7 ( https://issues.apache.org/jira/secure/attachment/12821498/YARN-5382-branch-2.7.09.patch ) . The 2.7 patch does not have logging in ClientRMService. I see only one audit log message when I ran a sleep job and killed it on pseudo-distributed setup on my laptop. [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -i Rmauditlogg logs/yarn-vchannapattan-resourcemanager-machine13-channapattan.log | grep -i Kill 2016-08-02 14:00:19,186 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl RESULT=SUCCESS APPID=application_1470171585834_0001 2016-08-02 14:00:19,195 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan OPERATION=Application Finished - Killed TARGET=RMAppManager RESULT=SUCCES APPID=application_1470171585834_0001 [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ On another window: [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4-SNAPSHOT.jar sleep -m 100 -r 1000 -mt 300 -rt 300 16/08/02 14:00:03 WARN util.NativeCodeLoader: Unable to load native -hadoop library for your platform... using builtin-java classes where applicable 16/08/02 14:00:03 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 16/08/02 14:00:05 INFO mapreduce.JobSubmitter: number of splits:100 16/08/02 14:00:05 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1470171585834_0001 16/08/02 14:00:05 INFO impl.YarnClientImpl: Submitted application application_1470171585834_0001 16/08/02 14:00:05 INFO mapreduce.Job: The url to track the job: http: //localhost:8088/proxy/application_1470171585834_0001/ 16/08/02 14:00:05 INFO mapreduce.Job: Running job: job_1470171585834_0001 ^C [machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/yarn application -kill application_1470171585834_0001 16/08/02 14:00:16 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 16/08/02 14:00:17 WARN util.NativeCodeLoader: Unable to load native -hadoop library for your platform... using builtin-java classes where applicable Killing application application_1470171585834_0001 16/08/02 14:00:19 INFO impl.YarnClientImpl: Killed application application_1470171585834_0001 [t-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ I need to update the patch for trunk to include removal of the audit logging upon isAppFinalStateStored check.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading v10 patch for trunk (as well as branch 2.7 for simplicity of keeping track of patch numbers).

        The only change from v9 is that the trunk patch removes the existing audit logging in ClientRMService#forceKillApplication at the application.isAppFinalStateStored() check.

        I tried running several test jobs on my pseudo cluster and tried invoking kill at different times, but I only see one audit log message in the RM logs.

        Show
        vrushalic Vrushali C added a comment - Uploading v10 patch for trunk (as well as branch 2.7 for simplicity of keeping track of patch numbers). The only change from v9 is that the trunk patch removes the existing audit logging in ClientRMService#forceKillApplication at the application.isAppFinalStateStored() check. I tried running several test jobs on my pseudo cluster and tried invoking kill at different times, but I only see one audit log message in the RM logs.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 36s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 21s branch-2.7 passed
        +1 compile 0m 25s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 28s branch-2.7 passed
        +1 mvnsite 0m 37s branch-2.7 passed
        +1 mvneclipse 0m 18s branch-2.7 passed
        -1 findbugs 1m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 20s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 28s the patch passed
        +1 compile 0m 22s the patch passed with JDK v1.8.0_101
        +1 javac 0m 22s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_101
        +1 javac 0m 26s the patch passed
        -1 checkstyle 0m 24s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 34 new + 684 unchanged - 7 fixed = 718 total (was 691)
        +1 mvnsite 0m 31s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 3946 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 1m 41s The patch 96 line(s) with tabs.
        +1 findbugs 1m 12s the patch passed
        -1 javadoc 0m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2)
        -1 unit 49m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 unit 50m 23s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        132m 53s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821691/YARN-5382-branch-2.7.10.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c105263af71e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / a8886dc
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12617/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12617/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 36s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 21s branch-2.7 passed +1 compile 0m 25s branch-2.7 passed with JDK v1.8.0_101 +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 28s branch-2.7 passed +1 mvnsite 0m 37s branch-2.7 passed +1 mvneclipse 0m 18s branch-2.7 passed -1 findbugs 1m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 20s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 28s the patch passed +1 compile 0m 22s the patch passed with JDK v1.8.0_101 +1 javac 0m 22s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_101 +1 javac 0m 26s the patch passed -1 checkstyle 0m 24s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 34 new + 684 unchanged - 7 fixed = 718 total (was 691) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 3946 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 1m 41s The patch 96 line(s) with tabs. +1 findbugs 1m 12s the patch passed -1 javadoc 0m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101 with JDK v1.7.0_101 generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) -1 unit 49m 48s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 unit 50m 23s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 132m 53s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821691/YARN-5382-branch-2.7.10.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c105263af71e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / a8886dc Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12617/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12617/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12617/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        I see only one audit log message when I ran a sleep job and killed it on pseudo-distributed setup on my laptop

        I checked the code more, that's because AppKilledTransition will not get the RMAppKillByClientEvent any more if there exists an attempt - AppKilledTransition is processing the event sent from RMAppAttempt if there exists the attempt. Anyway, this actually makes things better, because we won't have two audit logs.

        • This code is exactly the same in two places, would you make a common method for it ?
                if (event instanceof RMAppKillByClientEvent) {
                  RMAppKillByClientEvent killEvent = (RMAppKillByClientEvent) event;
                  UserGroupInformation callerUGI = killEvent.getCallerUGI();
                  String userName = null;
                  if (callerUGI != null) {
                    userName = callerUGI.getShortUserName();
                  }
                  InetAddress remoteIP = killEvent.getIp();
                  RMAuditLogger.logSuccess(userName, AuditConstants.KILL_APP_REQUEST,
                      "RMAppImpl", event.getApplicationId(), remoteIP);
                }
          
        • Isn't "greater than" the correct wording ?
          -    Assert.assertTrue("application start time is not greater than 0",
          +    Assert.assertTrue("application start time is not greater then 0",
          
        • several parameters are not used in this method testSuccessLogFormatHelperWithIP, remove them ?
        • nit highlighted by IDE: "returns the {@link CallerUGI}

          " the CallerUGI is actually not a link.

        Show
        jianhe Jian He added a comment - I see only one audit log message when I ran a sleep job and killed it on pseudo-distributed setup on my laptop I checked the code more, that's because AppKilledTransition will not get the RMAppKillByClientEvent any more if there exists an attempt - AppKilledTransition is processing the event sent from RMAppAttempt if there exists the attempt. Anyway, this actually makes things better, because we won't have two audit logs. This code is exactly the same in two places, would you make a common method for it ? if (event instanceof RMAppKillByClientEvent) { RMAppKillByClientEvent killEvent = (RMAppKillByClientEvent) event; UserGroupInformation callerUGI = killEvent.getCallerUGI(); String userName = null ; if (callerUGI != null ) { userName = callerUGI.getShortUserName(); } InetAddress remoteIP = killEvent.getIp(); RMAuditLogger.logSuccess(userName, AuditConstants.KILL_APP_REQUEST, "RMAppImpl" , event.getApplicationId(), remoteIP); } Isn't "greater than" the correct wording ? - Assert.assertTrue( "application start time is not greater than 0" , + Assert.assertTrue( "application start time is not greater then 0" , several parameters are not used in this method testSuccessLogFormatHelperWithIP, remove them ? nit highlighted by IDE: "returns the {@link CallerUGI} " the CallerUGI is actually not a link.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jian He, appreciate the feedback!
        Uploading v11 with suggested changes.

        Also, the "greater then" wording in the patch on trunk came in because the existing 2.7 codebase has those. But I have fixed that for both trunk and 2.7 in the v11 patch.

        Show
        vrushalic Vrushali C added a comment - Thanks Jian He , appreciate the feedback! Uploading v11 with suggested changes. Also, the "greater then" wording in the patch on trunk came in because the existing 2.7 codebase has those. But I have fixed that for both trunk and 2.7 in the v11 patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 42s trunk passed
        +1 compile 0m 35s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 41s trunk passed
        +1 mvneclipse 0m 19s trunk passed
        +1 findbugs 0m 56s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 214 unchanged - 1 fixed = 216 total (was 215)
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 1s the patch passed
        -1 javadoc 0m 19s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 4 new + 963 unchanged - 0 fixed = 967 total (was 963)
        +1 unit 37m 2s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        53m 41s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821911/YARN-5382.11.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9df726fe0192 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / db4a61d
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12632/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12632/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12632/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12632/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 42s trunk passed +1 compile 0m 35s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 41s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 214 unchanged - 1 fixed = 216 total (was 215) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 1s the patch passed -1 javadoc 0m 19s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 4 new + 963 unchanged - 0 fixed = 967 total (was 963) +1 unit 37m 2s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 53m 41s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821911/YARN-5382.11.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9df726fe0192 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / db4a61d Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12632/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12632/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12632/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12632/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Will quickly fix the javadoc warnings and one of the findbugs issue. I am not sure I can fix the other findbugs (the one about 9 params instead of 7, there are already 8 params in that function).

        Show
        vrushalic Vrushali C added a comment - Will quickly fix the javadoc warnings and one of the findbugs issue. I am not sure I can fix the other findbugs (the one about 9 params instead of 7, there are already 8 params in that function).
        Hide
        vrushalic Vrushali C added a comment -

        Uploading v12 that addresses javadoc warnings.

        Show
        vrushalic Vrushali C added a comment - Uploading v12 that addresses javadoc warnings.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 15m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 32s branch-2.7 passed
        +1 compile 0m 27s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 0m 29s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 29s branch-2.7 passed
        +1 mvnsite 0m 37s branch-2.7 passed
        +1 mvneclipse 0m 18s branch-2.7 passed
        -1 findbugs 1m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 22s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 0m 25s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 24s the patch passed with JDK v1.8.0_101
        +1 javac 0m 24s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_101
        +1 javac 0m 26s the patch passed
        -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 31 new + 683 unchanged - 7 fixed = 714 total (was 690)
        +1 mvnsite 0m 32s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 4582 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 1m 45s The patch 104 line(s) with tabs.
        +1 findbugs 1m 14s the patch passed
        -1 javadoc 0m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        +1 javadoc 0m 23s the patch passed with JDK v1.7.0_101
        -1 unit 54m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 unit 53m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        144m 10s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821924/YARN-5382-branch-2.7.12.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a43523cbd7a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 9669779
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12633/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12633/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 32s branch-2.7 passed +1 compile 0m 27s branch-2.7 passed with JDK v1.8.0_101 +1 compile 0m 29s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 29s branch-2.7 passed +1 mvnsite 0m 37s branch-2.7 passed +1 mvneclipse 0m 18s branch-2.7 passed -1 findbugs 1m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 22s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 0m 25s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 27s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_101 +1 javac 0m 24s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_101 +1 javac 0m 26s the patch passed -1 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 31 new + 683 unchanged - 7 fixed = 714 total (was 690) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 4582 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 1m 45s The patch 104 line(s) with tabs. +1 findbugs 1m 14s the patch passed -1 javadoc 0m 17s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. +1 javadoc 0m 23s the patch passed with JDK v1.7.0_101 -1 unit 54m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 unit 53m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_101. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 144m 10s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821924/YARN-5382-branch-2.7.12.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a43523cbd7a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 9669779 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/12633/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12633/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12633/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        Vrushali C, looks like you reformatted the whole RMAppImpl java file, and introduced a lot of format changes, would you revert those ?

        Show
        jianhe Jian He added a comment - Vrushali C , looks like you reformatted the whole RMAppImpl java file, and introduced a lot of format changes, would you revert those ?
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jian He, appreciate your time and review on this.
        Uploading patch 13 for trunk.

        Show
        vrushalic Vrushali C added a comment - Thanks Jian He , appreciate your time and review on this. Uploading patch 13 for trunk.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 6m 54s trunk passed
        +1 compile 0m 33s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 39s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 58s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 31s the patch passed
        +1 javac 0m 31s the patch passed
        -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215)
        +1 mvnsite 0m 36s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 5s the patch passed
        -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 unit 36m 50s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        51m 42s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822329/YARN-5382.13.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 39a6bb10d1ec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d9a354c
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12658/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12658/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 54s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 58s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 5s the patch passed -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 36m 50s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 51m 42s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822329/YARN-5382.13.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 39a6bb10d1ec 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d9a354c Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12658/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12658/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12658/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        lgtm,
        Jason Lowe, would you like to check?

        Show
        jianhe Jian He added a comment - lgtm, Jason Lowe , would you like to check?
        Hide
        jlowe Jason Lowe added a comment -

        The auditLogKillEvent method is now being called from the AppRejectedTransition instead of the KillAttemptTransition which I assume is unintentional. I think we need the audit log event in the KillAttemptTransition to make sure apps that are killed before they start running still get logged.

        Show
        jlowe Jason Lowe added a comment - The auditLogKillEvent method is now being called from the AppRejectedTransition instead of the KillAttemptTransition which I assume is unintentional. I think we need the audit log event in the KillAttemptTransition to make sure apps that are killed before they start running still get logged.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks Jason Lowe, yes updating the patch for trunk.

        Show
        vrushalic Vrushali C added a comment - Thanks Jason Lowe , yes updating the patch for trunk.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 52s trunk passed
        +1 compile 0m 32s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 38s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 1m 0s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215)
        +1 mvnsite 0m 35s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 0s the patch passed
        -1 javadoc 0m 36s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 unit 34m 14s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        50m 9s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822694/YARN-5382.14.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 168893af76d9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0705489
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12683/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12683/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12683/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12683/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 52s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 38s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 1m 0s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) +1 mvnsite 0m 35s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 0s the patch passed -1 javadoc 0m 36s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 34m 14s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 50m 9s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822694/YARN-5382.14.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 168893af76d9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0705489 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12683/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12683/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12683/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12683/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jianhe Jian He added a comment -

        ah, I missed it.. thanks for catching this !

        Show
        jianhe Jian He added a comment - ah, I missed it.. thanks for catching this !
        Hide
        jlowe Jason Lowe added a comment -

        +1 for the latest patch. Committing this.

        Show
        jlowe Jason Lowe added a comment - +1 for the latest patch. Committing this.
        Hide
        jlowe Jason Lowe added a comment -

        Scratch that commit, the javadoc error flagged above is related to the patch. Please investigate:

        [ERROR] /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppKillByClientEvent.java:42: error: bad HTML entity
        [ERROR] * @param callerUGI caller's user & group information
        [ERROR] ^
        

        Also the branch-2.7 patch no longer applies and needs to be rebased.

        Show
        jlowe Jason Lowe added a comment - Scratch that commit, the javadoc error flagged above is related to the patch. Please investigate: [ERROR] /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppKillByClientEvent.java:42: error: bad HTML entity [ERROR] * @param callerUGI caller's user & group information [ERROR] ^ Also the branch-2.7 patch no longer applies and needs to be rebased.
        Hide
        vrushalic Vrushali C added a comment - - edited

        Uploading patches for trunk as well as branch 2.7 after rebasing to latest.
        Appreciate everyone's time, effort and patience on reviewing these patches.

        Show
        vrushalic Vrushali C added a comment - - edited Uploading patches for trunk as well as branch 2.7 after rebasing to latest. Appreciate everyone's time, effort and patience on reviewing these patches.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 6m 41s trunk passed
        +1 compile 0m 31s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 54s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 33s the patch passed
        +1 compile 0m 33s the patch passed
        +1 javac 0m 33s the patch passed
        -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215)
        +1 mvnsite 0m 39s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 1m 10s the patch passed
        +1 javadoc 0m 20s the patch passed
        -1 unit 37m 40s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        52m 25s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822861/YARN-5382.15.patch
        JIRA Issue YARN-5382
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0f8e4996b6dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c4b77ae
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12700/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12700/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 41s trunk passed +1 compile 0m 31s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 54s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 33s the patch passed +1 compile 0m 33s the patch passed +1 javac 0m 33s the patch passed -1 checkstyle 0m 20s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 214 unchanged - 1 fixed = 215 total (was 215) +1 mvnsite 0m 39s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 1m 10s the patch passed +1 javadoc 0m 20s the patch passed -1 unit 37m 40s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 52m 25s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822861/YARN-5382.15.patch JIRA Issue YARN-5382 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0f8e4996b6dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c4b77ae Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12700/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12700/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12700/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jlowe Jason Lowe added a comment -

        +1 for the latest trunk and branch-2.7 patches. I'll commit this tomorrow if there are no objections.

        Show
        jlowe Jason Lowe added a comment - +1 for the latest trunk and branch-2.7 patches. I'll commit this tomorrow if there are no objections.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10254 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10254/)
        YARN-5382. RM does not audit log kill request for active applications. (jlowe: rev 5c95bb315ba605b3bed77966a99a63854234e2c5)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppKillByClientEvent.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10254 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10254/ ) YARN-5382 . RM does not audit log kill request for active applications. (jlowe: rev 5c95bb315ba605b3bed77966a99a63854234e2c5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppKillByClientEvent.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
        Hide
        jlowe Jason Lowe added a comment -

        Thanks to Vrushali C for the contribution and to Jian He for addiional review! I committed this to trunk, branch-2, branch-2.8, and branch-2.7.

        Show
        jlowe Jason Lowe added a comment - Thanks to Vrushali C for the contribution and to Jian He for addiional review! I committed this to trunk, branch-2, branch-2.8, and branch-2.7.

          People

          • Assignee:
            vrushalic Vrushali C
            Reporter:
            jlowe Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development