Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3692

Allow REST API to set a user generated message when killing an application

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently YARN's REST API supports killing an application without setting a diagnostic message. It would be good to provide that support.

      Use Case
      Usually this helps in workflow management in a multi-tenant environment when the workflow scheduler (or the hadoop admin) wants to kill a job - and let the user know the reason why the job was killed. Killing the job by setting a diagnostic message is a very good solution for that. Ideally, we can set the diagnostic message on all such interface:
      yarn kill -applicationId ... -diagnosticMessage "some message added by admin/workflow"
      REST API

      { 'state': 'KILLED', 'diagnosticMessage': 'some message added by admin/workflow'}
      1. 0001-YARN-3692.patch
        10 kB
        Rohith Sharma K S
      2. 0002-YARN-3692.patch
        14 kB
        Rohith Sharma K S
      3. 0003-YARN-3692.patch
        10 kB
        Rohith Sharma K S
      4. 0004-YARN-3692.patch
        15 kB
        Rohith Sharma K S
      5. 0005-YARN-3692.1.patch
        17 kB
        Rohith Sharma K S
      6. 0005-YARN-3692.patch
        17 kB
        Rohith Sharma K S
      7. 0006-YARN-3692.patch
        17 kB
        Rohith Sharma K S
      8. 0007-YARN-3692.1.patch
        18 kB
        Rohith Sharma K S
      9. 0007-YARN-3692.patch
        18 kB
        Rohith Sharma K S

        Activity

        Hide
        Naganarasimha Naganarasimha G R added a comment -

        As it was a small fix corrected it my self and thus finished committing to trunk, branch-2 & branch-2.8.
        Thanks for the contribution Rohith Sharma K S and reviews from Sunil G,Allen Wittenauer, & Steve Loughran

        Show
        Naganarasimha Naganarasimha G R added a comment - As it was a small fix corrected it my self and thus finished committing to trunk, branch-2 & branch-2.8. Thanks for the contribution Rohith Sharma K S and reviews from Sunil G , Allen Wittenauer , & Steve Loughran
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10476 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10476/)
        YARN-3692. Allow REST API to set a user generated message when killing (naganarasimha_gr: rev d0372dc613136910160e9d42bd5eaa0d4bde2356)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppsModification.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppState.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10476 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10476/ ) YARN-3692 . Allow REST API to set a user generated message when killing (naganarasimha_gr: rev d0372dc613136910160e9d42bd5eaa0d4bde2356) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesAppsModification.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppState.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/KillApplicationRequest.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/KillApplicationRequestPBImpl.java
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Rohith Sharma K S,
        Seems like patch doesnt seem to apply on branch-2

        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hadoop-yarn-server-resourcemanager: Compilation failure: Compilation failure:
        [ERROR] /opt/git/commit/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java:[1162,23] local variable diagnostic is accessed from within inner class; needs to be declared final
        [ERROR] /opt/git/commit/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java:[1163,40] local variable diagnostic is accessed from within inner class; needs to be declared final
        

        also shall i commit for 2.8?

        Show
        Naganarasimha Naganarasimha G R added a comment - Rohith Sharma K S , Seems like patch doesnt seem to apply on branch-2 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile ( default -compile) on project hadoop-yarn-server-resourcemanager: Compilation failure: Compilation failure: [ERROR] /opt/git/commit/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java:[1162,23] local variable diagnostic is accessed from within inner class; needs to be declared final [ERROR] /opt/git/commit/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java:[1163,40] local variable diagnostic is accessed from within inner class; needs to be declared final also shall i commit for 2.8?
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the latest patch Rohith Sharma K S,
        +1 LGTM, if no further comments will commit it today.

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the latest patch Rohith Sharma K S , +1 LGTM, if no further comments will commit it today.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 1m 45s Maven dependency ordering for branch
        +1 mvninstall 7m 7s trunk passed
        +1 compile 7m 17s trunk passed
        +1 checkstyle 1m 28s trunk passed
        +1 mvnsite 2m 28s trunk passed
        +1 mvneclipse 1m 14s trunk passed
        +1 findbugs 3m 46s trunk passed
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 59s the patch passed
        +1 compile 8m 14s the patch passed
        +1 cc 8m 14s the patch passed
        +1 javac 8m 14s the patch passed
        +1 checkstyle 1m 36s root: The patch generated 0 new + 232 unchanged - 1 fixed = 232 total (was 233)
        +1 mvnsite 2m 49s the patch passed
        +1 mvneclipse 1m 18s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 33s the patch passed
        -1 javadoc 0m 20s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 javadoc 0m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157)
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 17s hadoop-yarn-common in the patch passed.
        +1 unit 34m 4s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 5s hadoop-yarn-client in the patch passed.
        +1 unit 118m 20s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 33s The patch does not generate ASF License warnings.
        222m 5s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829380/0007-YARN-3692.1.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux e2eb8116fd6f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2b66d9e
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13161/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13161/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 45s Maven dependency ordering for branch +1 mvninstall 7m 7s trunk passed +1 compile 7m 17s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 2m 28s trunk passed +1 mvneclipse 1m 14s trunk passed +1 findbugs 3m 46s trunk passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 59s the patch passed +1 compile 8m 14s the patch passed +1 cc 8m 14s the patch passed +1 javac 8m 14s the patch passed +1 checkstyle 1m 36s root: The patch generated 0 new + 232 unchanged - 1 fixed = 232 total (was 233) +1 mvnsite 2m 49s the patch passed +1 mvneclipse 1m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 33s the patch passed -1 javadoc 0m 20s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157) +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 17s hadoop-yarn-common in the patch passed. +1 unit 34m 4s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 5s hadoop-yarn-client in the patch passed. +1 unit 118m 20s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 222m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829380/0007-YARN-3692.1.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux e2eb8116fd6f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2b66d9e Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13161/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13161/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13161/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated the patch

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated the patch
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        oops, let me upload new patch. thanks

        Show
        rohithsharma Rohith Sharma K S added a comment - oops, let me upload new patch. thanks
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        hi Rohith Sharma K S,
        Seems like patch is erroneously created, can you check and recreate the patch ?
        Looks like overall things are fine, can get it committed if no other issues from others.

        Show
        Naganarasimha Naganarasimha G R added a comment - hi Rohith Sharma K S , Seems like patch is erroneously created, can you check and recreate the patch ? Looks like overall things are fine, can get it committed if no other issues from others.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 6s YARN-3692 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829167/0007-YARN-3692.patch
        JIRA Issue YARN-3692
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13146/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 6s YARN-3692 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829167/0007-YARN-3692.patch JIRA Issue YARN-3692 Console output https://builds.apache.org/job/PreCommit-YARN-Build/13146/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 1m 38s Maven dependency ordering for branch
        +1 mvninstall 7m 0s trunk passed
        +1 compile 6m 49s trunk passed
        +1 checkstyle 1m 27s trunk passed
        +1 mvnsite 2m 28s trunk passed
        +1 mvneclipse 1m 13s trunk passed
        +1 findbugs 3m 45s trunk passed
        +1 javadoc 1m 35s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 11s the patch passed
        +1 compile 8m 21s the patch passed
        +1 cc 8m 21s the patch passed
        +1 javac 8m 21s the patch passed
        +1 checkstyle 1m 40s root: The patch generated 0 new + 232 unchanged - 1 fixed = 232 total (was 233)
        +1 mvnsite 2m 56s the patch passed
        +1 mvneclipse 1m 23s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 58s the patch passed
        -1 javadoc 0m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157)
        +1 unit 0m 29s hadoop-yarn-api in the patch passed.
        +1 unit 2m 32s hadoop-yarn-common in the patch passed.
        +1 unit 35m 22s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 9s hadoop-yarn-client in the patch passed.
        +1 unit 122m 17s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        228m 19s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829142/0006-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 2d4ac19ce4d0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 3552c2b
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13142/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13142/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13142/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 38s Maven dependency ordering for branch +1 mvninstall 7m 0s trunk passed +1 compile 6m 49s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 2m 28s trunk passed +1 mvneclipse 1m 13s trunk passed +1 findbugs 3m 45s trunk passed +1 javadoc 1m 35s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 11s the patch passed +1 compile 8m 21s the patch passed +1 cc 8m 21s the patch passed +1 javac 8m 21s the patch passed +1 checkstyle 1m 40s root: The patch generated 0 new + 232 unchanged - 1 fixed = 232 total (was 233) +1 mvnsite 2m 56s the patch passed +1 mvneclipse 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 58s the patch passed -1 javadoc 0m 15s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157) +1 unit 0m 29s hadoop-yarn-api in the patch passed. +1 unit 2m 32s hadoop-yarn-common in the patch passed. +1 unit 35m 22s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 9s hadoop-yarn-client in the patch passed. +1 unit 122m 17s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 228m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829142/0006-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 2d4ac19ce4d0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3552c2b Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13142/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13142/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13142/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated the patch

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated the patch
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks Rohith Sharma K S, patch is almost fine except for 1 minor nit :

        • if we print UGI then we get the log statement like dr.who (auth:SIMPLE) instead dr.who should be better by using callerUGI.getShortUserName().
        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks Rohith Sharma K S , patch is almost fine except for 1 minor nit : if we print UGI then we get the log statement like dr.who (auth:SIMPLE) instead dr.who should be better by using callerUGI.getShortUserName() .
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated patch fixing first 2 comments. RMAdminCli and Web UI support can be done in separate JIRA.

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated patch fixing first 2 comments. RMAdminCli and Web UI support can be done in separate JIRA.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Hi Rohith Sharma K S, Thanks for the update, overall patch looks fine, i got to test it too this time.
        few nits :

        1. Kill application application_1474052822592_0004 received from dr.who (auth:SIMPLE)Test Kill 1 is the diagnostic message which was received, which does not seem to be a proper sentence. How about Application "application_1474052822592_0004" was killed by dr.who with diagnostic message: Test Kill 1
        2. check style issue seems to be valid
        3. CLI support also needs to be given, i am fine with handling in a seperate jira
        Show
        Naganarasimha Naganarasimha G R added a comment - Hi Rohith Sharma K S , Thanks for the update, overall patch looks fine, i got to test it too this time. few nits : Kill application application_1474052822592_0004 received from dr.who (auth:SIMPLE)Test Kill 1 is the diagnostic message which was received, which does not seem to be a proper sentence. How about Application "application_1474052822592_0004" was killed by dr.who with diagnostic message: Test Kill 1 check style issue seems to be valid CLI support also needs to be given, i am fine with handling in a seperate jira
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 44s trunk passed
        +1 compile 7m 58s trunk passed
        +1 checkstyle 1m 35s trunk passed
        +1 mvnsite 2m 48s trunk passed
        +1 mvneclipse 1m 22s trunk passed
        +1 findbugs 4m 28s trunk passed
        +1 javadoc 1m 39s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 0s the patch passed
        +1 compile 7m 2s the patch passed
        +1 cc 7m 2s the patch passed
        +1 javac 7m 2s the patch passed
        -1 checkstyle 1m 27s root: The patch generated 1 new + 233 unchanged - 1 fixed = 234 total (was 234)
        +1 mvnsite 2m 35s the patch passed
        +1 mvneclipse 1m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 22s the patch passed
        -1 javadoc 0m 16s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157)
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 19s hadoop-yarn-common in the patch passed.
        +1 unit 37m 29s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 0s hadoop-yarn-client in the patch passed.
        +1 unit 115m 17s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 29s The patch does not generate ASF License warnings.
        220m 50s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828683/0005-YARN-3692.1.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 1355118415f6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7cad7b7
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13114/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13114/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13114/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13114/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 7m 58s trunk passed +1 checkstyle 1m 35s trunk passed +1 mvnsite 2m 48s trunk passed +1 mvneclipse 1m 22s trunk passed +1 findbugs 4m 28s trunk passed +1 javadoc 1m 39s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 0s the patch passed +1 compile 7m 2s the patch passed +1 cc 7m 2s the patch passed +1 javac 7m 2s the patch passed -1 checkstyle 1m 27s root: The patch generated 1 new + 233 unchanged - 1 fixed = 234 total (was 234) +1 mvnsite 2m 35s the patch passed +1 mvneclipse 1m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 22s the patch passed -1 javadoc 0m 16s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157) +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 unit 37m 29s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 0s hadoop-yarn-client in the patch passed. +1 unit 115m 17s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 220m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828683/0005-YARN-3692.1.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 1355118415f6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7cad7b7 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13114/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13114/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13114/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13114/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Removed unused import and updated the patch

        Show
        rohithsharma Rohith Sharma K S added a comment - Removed unused import and updated the patch
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 51s trunk passed
        +1 compile 7m 9s trunk passed
        +1 checkstyle 1m 28s trunk passed
        +1 mvnsite 2m 31s trunk passed
        +1 mvneclipse 1m 14s trunk passed
        +1 findbugs 3m 45s trunk passed
        +1 javadoc 1m 33s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 0s the patch passed
        +1 compile 6m 56s the patch passed
        +1 cc 6m 56s the patch passed
        +1 javac 6m 56s the patch passed
        -1 checkstyle 1m 28s root: The patch generated 3 new + 233 unchanged - 1 fixed = 236 total (was 234)
        +1 mvnsite 2m 31s the patch passed
        +1 mvneclipse 1m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 25s the patch passed
        -1 javadoc 0m 14s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157)
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 19s hadoop-yarn-common in the patch passed.
        +1 unit 37m 28s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 5s hadoop-yarn-client in the patch passed.
        +1 unit 114m 21s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        217m 47s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828632/0005-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux fa021b97284e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2a8f55a
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13111/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13111/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13111/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13111/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 51s trunk passed +1 compile 7m 9s trunk passed +1 checkstyle 1m 28s trunk passed +1 mvnsite 2m 31s trunk passed +1 mvneclipse 1m 14s trunk passed +1 findbugs 3m 45s trunk passed +1 javadoc 1m 33s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 0s the patch passed +1 compile 6m 56s the patch passed +1 cc 6m 56s the patch passed +1 javac 6m 56s the patch passed -1 checkstyle 1m 28s root: The patch generated 3 new + 233 unchanged - 1 fixed = 236 total (was 234) +1 mvnsite 2m 31s the patch passed +1 mvneclipse 1m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 25s the patch passed -1 javadoc 0m 14s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client generated 1 new + 157 unchanged - 0 fixed = 158 total (was 157) +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 unit 37m 28s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 5s hadoop-yarn-client in the patch passed. +1 unit 114m 21s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 217m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12828632/0005-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux fa021b97284e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a8f55a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13111/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13111/artifact/patchprocess/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13111/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13111/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated the patch addressing review comments.

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated the patch addressing review comments.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Hi Rohith Sharma K S,
        Overall the approach is fine but to add few nits more

        1. YarnClient. l no 186..187, String diagnosis i used better to sync it everywhere to the same lingo
        2. Similarly in YarnClientImpl ln no 412.
        3. ClientRMService ln no 783 & 787, better to use append instead of "+"
        Show
        Naganarasimha Naganarasimha G R added a comment - Hi Rohith Sharma K S , Overall the approach is fine but to add few nits more YarnClient. l no 186..187, String diagnosis i used better to sync it everywhere to the same lingo Similarly in YarnClientImpl ln no 412. ClientRMService ln no 783 & 787, better to use append instead of "+"
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Naganarasimha Garla Would you give your review comments please so that I will rebase the patch all-together?

        Show
        rohithsharma Rohith Sharma K S added a comment - Naganarasimha Garla Would you give your review comments please so that I will rebase the patch all-together?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 6m 46s trunk passed
        +1 compile 6m 54s trunk passed
        +1 checkstyle 1m 27s trunk passed
        +1 mvnsite 2m 29s trunk passed
        +1 mvneclipse 1m 16s trunk passed
        +1 findbugs 3m 42s trunk passed
        +1 javadoc 1m 36s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 1m 59s the patch passed
        +1 compile 6m 52s the patch passed
        +1 cc 6m 52s the patch passed
        +1 javac 6m 52s the patch passed
        -1 checkstyle 1m 27s root: The patch generated 3 new + 190 unchanged - 1 fixed = 193 total (was 191)
        +1 mvnsite 2m 29s the patch passed
        +1 mvneclipse 1m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 24s the patch passed
        -1 javadoc 0m 14s hadoop-yarn-client in the patch failed.
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed.
        +1 unit 37m 31s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 16m 26s hadoop-yarn-client in the patch passed.
        +1 unit 118m 12s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        221m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827958/0004-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 41200cc542b3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / cc01ed70
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13078/artifact/patchprocess/diff-checkstyle-root.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13078/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13078/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13078/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 6m 54s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 2m 29s trunk passed +1 mvneclipse 1m 16s trunk passed +1 findbugs 3m 42s trunk passed +1 javadoc 1m 36s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 59s the patch passed +1 compile 6m 52s the patch passed +1 cc 6m 52s the patch passed +1 javac 6m 52s the patch passed -1 checkstyle 1m 27s root: The patch generated 3 new + 190 unchanged - 1 fixed = 193 total (was 191) +1 mvnsite 2m 29s the patch passed +1 mvneclipse 1m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 24s the patch passed -1 javadoc 0m 14s hadoop-yarn-client in the patch failed. +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 unit 37m 31s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 16m 26s hadoop-yarn-client in the patch passed. +1 unit 118m 12s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 221m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827958/0004-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 41200cc542b3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cc01ed70 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13078/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13078/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13078/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/13078/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        I think for trunk, patch looks fine. Few minor nits.

        1. KillApplicationRequestPBImpl.java
        getDiagnostics might not need the below null check.

        if (!p.hasDiagnostics()) {
        	      return null;
        }
        

        2. RMWebServices#updateAppState doesn't seem to have test coverage.Hence new change is not validated in this. I think a separate test jira could be added to handle the same.
        3. One nit
        May be in ResourceMgrDelegate#killApplication, diagnosis ==> diagnostics

        As mentioned by you, more compatible approach is needed for branch-2.

        Show
        sunilg Sunil G added a comment - I think for trunk, patch looks fine. Few minor nits. 1. KillApplicationRequestPBImpl.java getDiagnostics might not need the below null check. if (!p.hasDiagnostics()) { return null ; } 2. RMWebServices#updateAppState doesn't seem to have test coverage.Hence new change is not validated in this. I think a separate test jira could be added to handle the same. 3. One nit May be in ResourceMgrDelegate#killApplication, diagnosis ==> diagnostics As mentioned by you, more compatible approach is needed for branch-2.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Updated the patch fixing compilation errors. Request for review

        Show
        rohithsharma Rohith Sharma K S added a comment - Updated the patch fixing compilation errors. Request for review
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Rohith Sharma K S,
        As discussed offline, IMHO i think we should not much worry about other projects extending the interface class, for the sake of compatibility i think it should be sufficient that we do not remove any methods in the interface or modify the structure in a incompatible manner. Hence i presume approach "0002-YARN-3692.patch" is better with modification for fixing the MR issue. thoughts?

        Show
        Naganarasimha Naganarasimha G R added a comment - Rohith Sharma K S , As discussed offline, IMHO i think we should not much worry about other projects extending the interface class, for the sake of compatibility i think it should be sufficient that we do not remove any methods in the interface or modify the structure in a incompatible manner. Hence i presume approach "0002- YARN-3692 .patch" is better with modification for fixing the MR issue. thoughts?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Adding proto does not cause compatible issue. The compatible issue is adding an API in YARNClient.java. This class is extended by ResourceMgrDelegate or any other components. Currently we can do change in ResourceMgrDelegate but question is what about other components which are extending YARNClient.java? So, we can commit client API into trunk and REST API's in branch.

        Show
        rohithsharma Rohith Sharma K S added a comment - Adding proto does not cause compatible issue. The compatible issue is adding an API in YARNClient.java. This class is extended by ResourceMgrDelegate or any other components. Currently we can do change in ResourceMgrDelegate but question is what about other components which are extending YARNClient.java? So, we can commit client API into trunk and REST API's in branch.
        Hide
        aw Allen Wittenauer added a comment -

        As I said above, precommit is configured to only do the minimal amount of work. That includes avoiding recompiling the entire source tree. So no, it won't detect a compilation error during the patch phase if that dependent module isn't also getting compiled. This is why the daily qbt output is important and why it's disheartening to see committers ignore it's output. (e.g., the libhdfs tests that have been failing for over a week now that were almost certainly caused by a change in hadoop-common.)

        Show
        aw Allen Wittenauer added a comment - As I said above, precommit is configured to only do the minimal amount of work. That includes avoiding recompiling the entire source tree. So no, it won't detect a compilation error during the patch phase if that dependent module isn't also getting compiled. This is why the daily qbt output is important and why it's disheartening to see committers ignore it's output. (e.g., the libhdfs tests that have been failing for over a week now that were almost certainly caused by a change in hadoop-common.)
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the reply Steve Loughran and Rohith Sharma K S, but adding a overloaded method to the interface should not amount to compatability failures right? Also even in the proto we are adding it as optional so older clients should work with the upgraded server and i see no impact with downgrades too...

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the reply Steve Loughran and Rohith Sharma K S , but adding a overloaded method to the interface should not amount to compatability failures right? Also even in the proto we are adding it as optional so older clients should work with the upgraded server and i see no impact with downgrades too...
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks Allen Wittenauer for the reply
        Just one confirmation, i can understand if jenkins is not able to catch the test case failure outside the project where patch modifications are done, but here i felt compilation error was not getting detected for other projects. so it would not count as compatability but compilation failure. Correct me if i am wrong or am i missing something here ?

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks Allen Wittenauer for the reply Just one confirmation, i can understand if jenkins is not able to catch the test case failure outside the project where patch modifications are done, but here i felt compilation error was not getting detected for other projects. so it would not count as compatability but compilation failure. Correct me if i am wrong or am i missing something here ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 7m 33s trunk passed
        +1 compile 2m 37s trunk passed
        +1 checkstyle 0m 49s trunk passed
        +1 mvnsite 2m 3s trunk passed
        +1 mvneclipse 0m 47s trunk passed
        +1 findbugs 3m 28s trunk passed
        +1 javadoc 1m 12s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 31s the patch passed
        +1 compile 3m 13s the patch passed
        +1 cc 3m 13s the patch passed
        +1 javac 3m 13s the patch passed
        -1 checkstyle 0m 45s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 129 unchanged - 0 fixed = 131 total (was 129)
        +1 mvnsite 1m 55s the patch passed
        +1 mvneclipse 0m 43s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 44s the patch passed
        +1 javadoc 1m 3s the patch passed
        +1 unit 0m 24s hadoop-yarn-api in the patch passed.
        +1 unit 2m 22s hadoop-yarn-common in the patch passed.
        -1 unit 38m 57s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        75m 5s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827140/0003-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux c71bb3298f7d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 07650bc
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13015/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13015/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 7m 33s trunk passed +1 compile 2m 37s trunk passed +1 checkstyle 0m 49s trunk passed +1 mvnsite 2m 3s trunk passed +1 mvneclipse 0m 47s trunk passed +1 findbugs 3m 28s trunk passed +1 javadoc 1m 12s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 31s the patch passed +1 compile 3m 13s the patch passed +1 cc 3m 13s the patch passed +1 javac 3m 13s the patch passed -1 checkstyle 0m 45s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 129 unchanged - 0 fixed = 131 total (was 129) +1 mvnsite 1m 55s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 44s the patch passed +1 javadoc 1m 3s the patch passed +1 unit 0m 24s hadoop-yarn-api in the patch passed. +1 unit 2m 22s hadoop-yarn-common in the patch passed. -1 unit 38m 57s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 75m 5s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827140/0003-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux c71bb3298f7d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 07650bc Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13015/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13015/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13015/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Update the patch compatible with branch-2.
        The patch adds up another parameter to pass diagnosis message in KillApplicationRequest. And provides channel to pass diagnosis via web service.

        Show
        rohithsharma Rohith Sharma K S added a comment - Update the patch compatible with branch-2. The patch adds up another parameter to pass diagnosis message in KillApplicationRequest. And provides channel to pass diagnosis via web service.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        ahh.. compilation failure.. pretty bad on myself Thanks Naga for finding it.

        For 2.9, I do changes only with REST service so that it can be compatible with branch-2. For client support let me create a separate JIRA where in compatibility can be discussed and handle only for trunk.

        Show
        rohithsharma Rohith Sharma K S added a comment - ahh.. compilation failure.. pretty bad on myself Thanks Naga for finding it. For 2.9, I do changes only with REST service so that it can be compatible with branch-2. For client support let me create a separate JIRA where in compatibility can be discussed and handle only for trunk.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        I concur with the incompat issue. This needs to be something which downgrades

        Show
        stevel@apache.org Steve Loughran added a comment - I concur with the incompat issue. This needs to be something which downgrades
        Hide
        aw Allen Wittenauer added a comment -

        I think, we need to run mvn install in all projects..? Allen Wittenauer am I right..?

        I'm not really sure what you are suggesting, but "all" is almost always not the best approach. precommit is configured to only do the minimal amount of work. If it needs to do more work in other modules, then it needs to be told do that and only that. The typical way is upload a fake patch that also touches those modules. Just be aware that it will run through the entire process for all modules touched.

        Some other thoughts:

        • the YARN tests may not be providing enough adequate API coverage if the unit tests aren't failing but MR is
        • if MR can't run without changes, then the patch in it's current form is likely an incompatible change not suited for branch-2
        Show
        aw Allen Wittenauer added a comment - I think, we need to run mvn install in all projects..? Allen Wittenauer am I right..? I'm not really sure what you are suggesting, but "all" is almost always not the best approach. precommit is configured to only do the minimal amount of work. If it needs to do more work in other modules, then it needs to be told do that and only that. The typical way is upload a fake patch that also touches those modules. Just be aware that it will run through the entire process for all modules touched. Some other thoughts: the YARN tests may not be providing enough adequate API coverage if the unit tests aren't failing but MR is if MR can't run without changes, then the patch in it's current form is likely an incompatible change not suited for branch-2
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 52s Maven dependency ordering for branch
        +1 mvninstall 6m 56s trunk passed
        +1 compile 2m 16s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 2m 0s trunk passed
        +1 mvneclipse 0m 57s trunk passed
        +1 findbugs 3m 19s trunk passed
        +1 javadoc 1m 21s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 36s the patch passed
        +1 compile 2m 15s the patch passed
        +1 cc 2m 15s the patch passed
        +1 javac 2m 15s the patch passed
        -1 checkstyle 0m 38s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 182 unchanged - 1 fixed = 184 total (was 183)
        +1 mvnsite 1m 51s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 43s the patch passed
        -1 javadoc 0m 13s hadoop-yarn-client in the patch failed.
        +1 unit 0m 22s hadoop-yarn-api in the patch passed.
        +1 unit 2m 16s hadoop-yarn-common in the patch passed.
        +1 unit 37m 29s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 unit 15m 59s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        88m 21s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826571/0002-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 85bf33fe190a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 07650bc
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13012/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13012/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13012/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13012/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 52s Maven dependency ordering for branch +1 mvninstall 6m 56s trunk passed +1 compile 2m 16s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 2m 0s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 3m 19s trunk passed +1 javadoc 1m 21s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 36s the patch passed +1 compile 2m 15s the patch passed +1 cc 2m 15s the patch passed +1 javac 2m 15s the patch passed -1 checkstyle 0m 38s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 182 unchanged - 1 fixed = 184 total (was 183) +1 mvnsite 1m 51s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 43s the patch passed -1 javadoc 0m 13s hadoop-yarn-client in the patch failed. +1 unit 0m 22s hadoop-yarn-api in the patch passed. +1 unit 2m 16s hadoop-yarn-common in the patch passed. +1 unit 37m 29s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 15m 59s hadoop-yarn-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 88m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826571/0002-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 85bf33fe190a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 07650bc Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13012/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13012/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13012/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13012/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        ResourceMgrDelegate.java in hadoop-mapreduce-client-jobclient project and unfortunately jenkins did not run this project, hence it's missed.

        relaunching jenkins again.

        so if relanuch also jenkins will not catch..

        I think, we need to run mvn install in all projects..? Allen Wittenauer am I right..?

         maven install: patch
        ============================================================================
        ============================================================================
        
        
        cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
        mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt 2>&1
        Elapsed:   0m 24s
        cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
        mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt 2>&1
        Elapsed:   0m 29s
        cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt 2>&1
        Elapsed:   0m 40s
        cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt 2>&1
        Elapsed:   0m 20s
        
        Show
        brahmareddy Brahma Reddy Battula added a comment - ResourceMgrDelegate.java in hadoop-mapreduce-client-jobclient project and unfortunately jenkins did not run this project, hence it's missed. relaunching jenkins again. so if relanuch also jenkins will not catch.. I think, we need to run mvn install in all projects..? Allen Wittenauer am I right..? maven install: patch ============================================================================ ============================================================================ cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt 2>&1 Elapsed: 0m 24s cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt 2>&1 Elapsed: 0m 29s cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt 2>&1 Elapsed: 0m 40s cd /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client mvn -Dmaven.repo.local=/home/jenkins/yetus-m2/hadoop-trunk-patch-1 -Ptest-patch -DskipTests -fae clean install -DskipTests=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true > /testptch/hadoop/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt 2>&1 Elapsed: 0m 20s
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for rebasing the patch with modifications Rohith Sharma K S, Though jenkins report shows compilation passing its failing locally for me because org.apache.hadoop.mapred.ResourceMgrDelegate extends YarnClient but does not implement killApplication(ApplicationId applicationId, String diagnosis). relaunching jenkins again...

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for rebasing the patch with modifications Rohith Sharma K S , Though jenkins report shows compilation passing its failing locally for me because org.apache.hadoop.mapred.ResourceMgrDelegate extends YarnClient but does not implement killApplication(ApplicationId applicationId, String diagnosis) . relaunching jenkins again...
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 50s Maven dependency ordering for branch
        +1 mvninstall 7m 28s trunk passed
        +1 compile 2m 33s trunk passed
        +1 checkstyle 0m 42s trunk passed
        +1 mvnsite 2m 7s trunk passed
        +1 mvneclipse 0m 59s trunk passed
        +1 findbugs 3m 53s trunk passed
        +1 javadoc 1m 27s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 53s the patch passed
        +1 compile 2m 42s the patch passed
        +1 cc 2m 42s the patch passed
        +1 javac 2m 42s the patch passed
        -1 checkstyle 0m 40s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 182 unchanged - 1 fixed = 184 total (was 183)
        +1 mvnsite 2m 11s the patch passed
        +1 mvneclipse 0m 54s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 38s the patch passed
        -1 javadoc 0m 13s hadoop-yarn-client in the patch failed.
        +1 unit 0m 28s hadoop-yarn-api in the patch passed.
        +1 unit 2m 33s hadoop-yarn-common in the patch passed.
        +1 unit 38m 54s hadoop-yarn-server-resourcemanager in the patch passed.
        -1 unit 16m 0s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        94m 11s



        Reason Tests
        Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826571/0002-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 8f4160054dfc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6c60036
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12982/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12982/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 50s Maven dependency ordering for branch +1 mvninstall 7m 28s trunk passed +1 compile 2m 33s trunk passed +1 checkstyle 0m 42s trunk passed +1 mvnsite 2m 7s trunk passed +1 mvneclipse 0m 59s trunk passed +1 findbugs 3m 53s trunk passed +1 javadoc 1m 27s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 53s the patch passed +1 compile 2m 42s the patch passed +1 cc 2m 42s the patch passed +1 javac 2m 42s the patch passed -1 checkstyle 0m 40s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 182 unchanged - 1 fixed = 184 total (was 183) +1 mvnsite 2m 11s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 38s the patch passed -1 javadoc 0m 13s hadoop-yarn-client in the patch failed. +1 unit 0m 28s hadoop-yarn-api in the patch passed. +1 unit 2m 33s hadoop-yarn-common in the patch passed. +1 unit 38m 54s hadoop-yarn-server-resourcemanager in the patch passed. -1 unit 16m 0s hadoop-yarn-client in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 94m 11s Reason Tests Failed junit tests hadoop.yarn.client.TestApplicationClientProtocolOnHA Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826571/0002-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 8f4160054dfc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6c60036 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12982/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12982/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12982/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Rebased the patch with slight modifications from previous. Naganarasimha Garla kindly review the patch.

        Show
        rohithsharma Rohith Sharma K S added a comment - Rebased the patch with slight modifications from previous. Naganarasimha Garla kindly review the patch.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Sure, I will update the patch!

        Show
        rohithsharma Rohith Sharma K S added a comment - Sure, I will update the patch!
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        seems like patch is not getting applied on the trunk, mind rebasing Rohith Sharma K S ?

        Show
        Naganarasimha Naganarasimha G R added a comment - seems like patch is not getting applied on the trunk, mind rebasing Rohith Sharma K S ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 9s YARN-3692 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798487/0001-YARN-3692.patch
        JIRA Issue YARN-3692
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12939/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 9s YARN-3692 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798487/0001-YARN-3692.patch JIRA Issue YARN-3692 Console output https://builds.apache.org/job/PreCommit-YARN-Build/12939/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        Seems reasonable to me. Can someone with more knowledge of the codebase review this?

        Show
        stevel@apache.org Steve Loughran added a comment - Seems reasonable to me. Can someone with more knowledge of the codebase review this?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 6m 39s trunk passed
        +1 compile 1m 43s trunk passed with JDK v1.8.0_77
        +1 compile 2m 4s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 39s trunk passed
        +1 findbugs 3m 19s trunk passed
        +1 javadoc 1m 23s trunk passed with JDK v1.8.0_77
        +1 javadoc 3m 50s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 1m 17s the patch passed
        +1 compile 1m 40s the patch passed with JDK v1.8.0_77
        +1 cc 1m 40s the patch passed
        +1 javac 1m 40s the patch passed
        +1 compile 2m 0s the patch passed with JDK v1.7.0_95
        +1 cc 2m 0s the patch passed
        +1 javac 2m 0s the patch passed
        +1 checkstyle 0m 31s the patch passed
        +1 mvnsite 1m 24s the patch passed
        +1 mvneclipse 0m 35s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 3m 51s the patch passed
        +1 javadoc 1m 18s the patch passed with JDK v1.8.0_77
        +1 javadoc 3m 46s the patch passed with JDK v1.7.0_95
        +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_77.
        +1 unit 1m 50s hadoop-yarn-common in the patch passed with JDK v1.8.0_77.
        -1 unit 64m 19s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
        +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
        +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_95.
        -1 unit 49m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        158m 52s



        Reason Tests
        JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          hadoop.yarn.webapp.TestRMWithCSRFFilter
        JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
        JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          hadoop.yarn.webapp.TestRMWithCSRFFilter
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798487/0001-YARN-3692.patch
        JIRA Issue YARN-3692
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 5d3b5ea49c6c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 903428b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11061/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11061/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 1m 43s trunk passed with JDK v1.8.0_77 +1 compile 2m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 3m 19s trunk passed +1 javadoc 1m 23s trunk passed with JDK v1.8.0_77 +1 javadoc 3m 50s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 1m 40s the patch passed with JDK v1.8.0_77 +1 cc 1m 40s the patch passed +1 javac 1m 40s the patch passed +1 compile 2m 0s the patch passed with JDK v1.7.0_95 +1 cc 2m 0s the patch passed +1 javac 2m 0s the patch passed +1 checkstyle 0m 31s the patch passed +1 mvnsite 1m 24s the patch passed +1 mvneclipse 0m 35s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 51s the patch passed +1 javadoc 1m 18s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 46s the patch passed with JDK v1.7.0_95 +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_77. +1 unit 1m 50s hadoop-yarn-common in the patch passed with JDK v1.8.0_77. -1 unit 64m 19s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. -1 unit 49m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 158m 52s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.webapp.TestRMWithCSRFFilter JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798487/0001-YARN-3692.patch JIRA Issue YARN-3692 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 5d3b5ea49c6c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 903428b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11061/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11061/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11061/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Thanks NGarla_Unused and Jason Lowe for sharing your thoughts..
        1. I will consider defining new api with an addition parameter killApplication(ApplicationId applicationId,String reason)
        2. As of now, will not be deprecating the existing kill api
        3. Generating a useful diagnostic message is a very good improvement.

        Show
        rohithsharma Rohith Sharma K S added a comment - Thanks NGarla_Unused and Jason Lowe for sharing your thoughts.. 1. I will consider defining new api with an addition parameter killApplication(ApplicationId applicationId,String reason) 2. As of now, will not be deprecating the existing kill api 3. Generating a useful diagnostic message is a very good improvement.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        We can still generate a useful diagnostic message automatically on the RM side when one is not provided, such as which user from which host issued the kill command.

        +1 for this approach when no diagnostic message is given by the user/admin

        Show
        Naganarasimha Naganarasimha G R added a comment - We can still generate a useful diagnostic message automatically on the RM side when one is not provided, such as which user from which host issued the kill command. +1 for this approach when no diagnostic message is given by the user/admin
        Hide
        jlowe Jason Lowe added a comment -

        We cannot change the signature of the existing killApplication method or we break backwards compatibility. However as Naganarasimha G R mentioned we can simply add another method that takes the two arguments instead of one. Then we can support both the old method for backwards compatibility and the new method which allows a user-provided diagnostic. I'm not sure we should deprecate the old method just yet. We can still generate a useful diagnostic message automatically on the RM side when one is not provided, such as which user from which host issued the kill command.

        Show
        jlowe Jason Lowe added a comment - We cannot change the signature of the existing killApplication method or we break backwards compatibility. However as Naganarasimha G R mentioned we can simply add another method that takes the two arguments instead of one. Then we can support both the old method for backwards compatibility and the new method which allows a user-provided diagnostic. I'm not sure we should deprecate the old method just yet. We can still generate a useful diagnostic message automatically on the RM side when one is not provided, such as which user from which host issued the kill command.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for sharing you thoughts Rohith Sharma K S,
        +1 for the approach,
        And for YarnClient#killApplication(ApplicationId applicationId), How about overloaded method which takes diagnostic message and anotate existing method as deprecated ?

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for sharing you thoughts Rohith Sharma K S , +1 for the approach, And for YarnClient#killApplication(ApplicationId applicationId) , How about overloaded method which takes diagnostic message and anotate existing method as deprecated ?
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Going through for the implementation, this JIRA required to be handleid from REST and YarnClient. For both, common change is KillApplicationRequest proto i.e adding user diagnostic message. After making required modifications to protos

        1. REST service API can be mofied without breaking compatibility by adding new field in AppState.
        2. YarnClient#killApplication(ApplicationId applicationId) modifying this signature would be compatibile issue since it is already exposed API.
          I would like to hear comments from community about compatibility problem.
        Show
        rohithsharma Rohith Sharma K S added a comment - Going through for the implementation, this JIRA required to be handleid from REST and YarnClient . For both, common change is KillApplicationRequest proto i.e adding user diagnostic message . After making required modifications to protos REST service API can be mofied without breaking compatibility by adding new field in AppState. YarnClient#killApplication(ApplicationId applicationId) modifying this signature would be compatibile issue since it is already exposed API. I would like to hear comments from community about compatibility problem.
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        Tx Rajat Jainfor sharing your use case.
        It was quite long time looking into this JIRA. I will make a progress on this. Similar line of thinking, we can add diagnosis message for failing the attempt too.

        Show
        rohithsharma Rohith Sharma K S added a comment - Tx Rajat Jain for sharing your use case. It was quite long time looking into this JIRA. I will make a progress on this. Similar line of thinking, we can add diagnosis message for failing the attempt too.
        Hide
        rjainqb Rajat Jain added a comment -

        Yes. The user should be able to set a diagnostic message while killing the job. Usually this helps in workflow management in a multi-tenant environment when the workflow scheduler (or the hadoop admin) wants to kill a job - and let the user know the reason why the job was killed. Killing the job by setting a diagnostic message is a very good solution for that. Ideally, we can set the diagnostic message on all such interface:

        • yarn kill -applicationId ... -diagnosticMessage "some message added by admin/workflow"
        • REST API { 'state': 'KILLED', 'diagnosticMessage': 'some message added by admin/workflow'}
        Show
        rjainqb Rajat Jain added a comment - Yes. The user should be able to set a diagnostic message while killing the job. Usually this helps in workflow management in a multi-tenant environment when the workflow scheduler (or the hadoop admin) wants to kill a job - and let the user know the reason why the job was killed. Killing the job by setting a diagnostic message is a very good solution for that. Ideally, we can set the diagnostic message on all such interface: yarn kill -applicationId ... -diagnosticMessage "some message added by admin/workflow" REST API { 'state': 'KILLED', 'diagnosticMessage': 'some message added by admin/workflow'}
        Hide
        rohithsharma Rohith Sharma K S added a comment -

        All the applications are killed by user only. Diagnostic message for KILLED application by user is internal to YARN either it can be from REST or ApplicationClientProtocol who kills it.
        Is this let user set the reason for killing applications?

        Show
        rohithsharma Rohith Sharma K S added a comment - All the applications are killed by user only. Diagnostic message for KILLED application by user is internal to YARN either it can be from REST or ApplicationClientProtocol who kills it. Is this let user set the reason for killing applications?

          People

          • Assignee:
            rohithsharma Rohith Sharma K S
            Reporter:
            rjainqb Rajat Jain
          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development