Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6177

Yarn client should exit with an informative error message if an incompatible Jersey library is used at client

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.0.0-alpha2
    • Fix Version/s: 2.8.0, 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Let yarn client exit with an informative error message if an incompatible Jersey library is used from client side.

      Description

      Per discussion in YARN-5271, lets provide an error message to suggest user to disable timeline service instead of disabling for them.

      1. spark2-job-output-after.out
        5 kB
        Weiwei Yang
      2. spark2-job-output-after-besteffort.out
        8 kB
        Weiwei Yang
      3. spark2-job-output-before.out
        5 kB
        Weiwei Yang
      4. YARN-6177.01.patch
        4 kB
        Weiwei Yang
      5. YARN-6177.02.patch
        4 kB
        Weiwei Yang
      6. YARN-6177.03.patch
        4 kB
        Weiwei Yang
      7. YARN-6177.04.patch
        8 kB
        Weiwei Yang
      8. YARN-6177.05.patch
        8 kB
        Weiwei Yang
      9. YARN-6177.06.patch
        9 kB
        Weiwei Yang

        Activity

        Hide
        cheersyang Weiwei Yang added a comment -
        Show
        cheersyang Weiwei Yang added a comment - cc Wei-Chiu Chuang and Li Lu
        Hide
        cheersyang Weiwei Yang added a comment -

        With the work done in YARN-6137, it looks like now timeline client is only initiated under secure environment and timeline service is enabled, it has smaller chance to hit this issue but still a problem. An clear error message can help user understand what might be the problem and how to work around.

        When play with Spark2 on yarn, it fails to submit a job with

        ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster --driver-memory 4g --executor-memory 2g --executor-cores 1 --queue default lib/spark-examples*.jar 10
        ...
        Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig
        at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        ...

        With the patch, it still fails but with some additional information added

        Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig. It appears that the timeline client failed to initiate because an incompatible dependency in classpath. If timeline service is optional to this client, try to work around by setting yarn.timeline-service.client.best-effort to true.
        at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:190)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:359)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151)
        at org.apache.spark.deploy.yarn.Client.run(Client.scala:1154)
        at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213)
        at org.apache.spark.deploy.yarn.Client.main(Client.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        ...

        I realized set yarn.timeline-service.client.best-effort to true (take advantage of the work from YARN-3144) is better than asking user to disable timeline service, make sense? Hope this will help users. Thanks.

        Show
        cheersyang Weiwei Yang added a comment - With the work done in YARN-6137 , it looks like now timeline client is only initiated under secure environment and timeline service is enabled, it has smaller chance to hit this issue but still a problem. An clear error message can help user understand what might be the problem and how to work around. When play with Spark2 on yarn, it fails to submit a job with ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster --driver-memory 4g --executor-memory 2g --executor-cores 1 --queue default lib/spark-examples*.jar 10 ... Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:181) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:168) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) ... With the patch, it still fails but with some additional information added Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig. It appears that the timeline client failed to initiate because an incompatible dependency in classpath. If timeline service is optional to this client, try to work around by setting yarn.timeline-service.client.best-effort to true. at org.apache.hadoop.yarn.client.api.TimelineClient.createTimelineClient(TimelineClient.java:55) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createTimelineClient(YarnClientImpl.java:190) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getTimelineDelegationToken(YarnClientImpl.java:359) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:151) at org.apache.spark.deploy.yarn.Client.run(Client.scala:1154) at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1213) at org.apache.spark.deploy.yarn.Client.main(Client.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ... I realized set yarn.timeline-service.client.best-effort to true (take advantage of the work from YARN-3144 ) is better than asking user to disable timeline service, make sense? Hope this will help users. Thanks.
        Hide
        cheersyang Weiwei Yang added a comment -

        v2 patch slightly improves the error message.

        Show
        cheersyang Weiwei Yang added a comment - v2 patch slightly improves the error message.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 12m 33s trunk passed
        +1 compile 0m 19s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 20s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82)
        +1 mvnsite 0m 18s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 32s the patch passed
        +1 javadoc 0m 12s the patch passed
        +1 unit 17m 16s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        35m 35s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852216/YARN-6177.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0bbef71dec52 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 649deb7
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14889/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14889/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14889/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 33s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 20s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82) +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 32s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 17m 16s hadoop-yarn-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 35m 35s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852216/YARN-6177.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0bbef71dec52 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 649deb7 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14889/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14889/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14889/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 22m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 33s trunk passed
        +1 compile 0m 18s trunk passed
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 16s the patch passed
        +1 compile 0m 15s the patch passed
        +1 javac 0m 15s the patch passed
        -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82)
        +1 mvnsite 0m 17s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 31s the patch passed
        +1 javadoc 0m 11s the patch passed
        +1 unit 17m 17s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        60m 32s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852216/YARN-6177.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f6e30fd38c83 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 649deb7
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14888/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14888/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14888/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 22m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 33s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 16s the patch passed +1 compile 0m 15s the patch passed +1 javac 0m 15s the patch passed -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82) +1 mvnsite 0m 17s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 31s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 17m 17s hadoop-yarn-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 60m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852216/YARN-6177.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f6e30fd38c83 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 649deb7 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14888/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14888/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14888/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cheersyang Weiwei Yang added a comment -

        Fixed the minor checkstyle issue in v3 patch. And attached some sample job log files to help to explain the issue and the fix:

        1. Without the patch, submit a spark2 job to yarn fails pretty quickly with NoClassDefFound error, see output in spark2-job-output-before.out
        2. With this patch, submit the job again, it still fails with the same error, but with additional clearer error messages, see output in spark2-job-output-after.out
        3. Apply the suggestion given in the error message, set best-effort to true to tolerant this error, then submit the job again and the job succeed, see output in spark2-job-output-after-besteffort.out
        Show
        cheersyang Weiwei Yang added a comment - Fixed the minor checkstyle issue in v3 patch. And attached some sample job log files to help to explain the issue and the fix: Without the patch, submit a spark2 job to yarn fails pretty quickly with NoClassDefFound error, see output in spark2-job-output-before.out With this patch, submit the job again, it still fails with the same error, but with additional clearer error messages, see output in spark2-job-output-after.out Apply the suggestion given in the error message, set best-effort to true to tolerant this error, then submit the job again and the job succeed, see output in spark2-job-output-after-besteffort.out
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 13s trunk passed
        +1 compile 0m 22s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 35s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 21s the patch passed
        +1 compile 0m 20s the patch passed
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 26s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 49s the patch passed
        +1 javadoc 0m 15s the patch passed
        +1 unit 17m 28s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        38m 35s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852337/YARN-6177.03.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 63b5fd10df2f 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 243c0f3
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14906/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14906/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 13s trunk passed +1 compile 0m 22s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 35s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 49s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 17m 28s hadoop-yarn-client in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 38m 35s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852337/YARN-6177.03.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 63b5fd10df2f 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 243c0f3 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14906/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14906/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Wei-Chiu Chuang and Li Lu

        Can you help to review this patch? Thanks

        Show
        cheersyang Weiwei Yang added a comment - Hi Wei-Chiu Chuang and Li Lu Can you help to review this patch? Thanks
        Hide
        gtCarrera9 Li Lu added a comment -

        One quick inquiry: are we catching every throwable and swallow them if timelineServiceBestEffort is set to true? That sounds scary since we're swallowing OutOfMemoryError, etc...

        I think we should limit the range of timelineServiceBestEffort to exceptions, but we still preserve the program's behavior on errors. Meanwhile, we can improve the output message if we hit NoClassDefFoundError to hint users to disable timeline service?

        Show
        gtCarrera9 Li Lu added a comment - One quick inquiry: are we catching every throwable and swallow them if timelineServiceBestEffort is set to true? That sounds scary since we're swallowing OutOfMemoryError, etc... I think we should limit the range of timelineServiceBestEffort to exceptions, but we still preserve the program's behavior on errors. Meanwhile, we can improve the output message if we hit NoClassDefFoundError to hint users to disable timeline service?
        Hide
        cheersyang Weiwei Yang added a comment -

        Thanks Li Lu for the quick review,

        are we catching every throwable and swallow them if timelineServiceBestEffort is set to true?

        that's definitely not my intention! thanks for pointing this out, I am going to correct this and upload a new patch shortly.

        Meanwhile, we can improve the output message if we hit NoClassDefFoundError to hint users to disable timeline service?

        I am proposing the error message like following
        "Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig. It appears that the timeline client failed to initiate because an incompatible dependency in classpath. If timeline service is optional to this client, try to work around by setting yarn.timeline-service.client.best-effort to true."
        Use timeline best effort flag seems a better option for me than disabling it, are you suggesting we should still ask users to disable it?

        Show
        cheersyang Weiwei Yang added a comment - Thanks Li Lu for the quick review, are we catching every throwable and swallow them if timelineServiceBestEffort is set to true? that's definitely not my intention! thanks for pointing this out, I am going to correct this and upload a new patch shortly. Meanwhile, we can improve the output message if we hit NoClassDefFoundError to hint users to disable timeline service? I am proposing the error message like following "Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig. It appears that the timeline client failed to initiate because an incompatible dependency in classpath. If timeline service is optional to this client, try to work around by setting yarn.timeline-service.client.best-effort to true." Use timeline best effort flag seems a better option for me than disabling it, are you suggesting we should still ask users to disable it?
        Hide
        gtCarrera9 Li Lu added a comment -

        Use timeline best effort flag seems a better option for me than disabling it, are you suggesting we should still ask users to disable it?

        Even with our "best effort", I don't think we should keep the program running on errors... Thoughts?

        Show
        gtCarrera9 Li Lu added a comment - Use timeline best effort flag seems a better option for me than disabling it, are you suggesting we should still ask users to disable it? Even with our "best effort", I don't think we should keep the program running on errors... Thoughts?
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Li Lu

        Yes, it won't be running when there is an error, all I am going to do is to wrap that error message to NoClassDefFoundError body and throw it out.

        Show
        cheersyang Weiwei Yang added a comment - Hi Li Lu Yes, it won't be running when there is an error, all I am going to do is to wrap that error message to NoClassDefFoundError body and throw it out.
        Hide
        gtCarrera9 Li Lu added a comment -

        Weiwei Yang Looks fine to me. Thanks!

        Show
        gtCarrera9 Li Lu added a comment - Weiwei Yang Looks fine to me. Thanks!
        Hide
        cheersyang Weiwei Yang added a comment -

        Uploaded v4 patch, fixed the problem in v3 patch that Li Lu pointed out, refactored the UT code to get better coverage. Mocked errors and exceptions to make sure yarn client behaves correctly.

        Show
        cheersyang Weiwei Yang added a comment - Uploaded v4 patch, fixed the problem in v3 patch that Li Lu pointed out, refactored the UT code to get better coverage. Mocked errors and exceptions to make sure yarn client behaves correctly.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 15s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 32s trunk passed
        +1 javadoc 0m 17s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 19s the patch passed
        +1 javac 0m 19s the patch passed
        -0 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82)
        +1 mvnsite 0m 21s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 40s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 16m 59s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        37m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852515/YARN-6177.04.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f3780bcc21c2 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 71c23c9
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14925/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14925/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14925/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 15s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 32s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed -0 checkstyle 0m 14s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 82 unchanged - 0 fixed = 83 total (was 82) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 40s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 16m 59s hadoop-yarn-client in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 37m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852515/YARN-6177.04.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f3780bcc21c2 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 71c23c9 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14925/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14925/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14925/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for patch. LGTM overall, with a few nits:

        + LOG.warn("Ignore a NoClassDefFoundError when attempt to get" --> when attempting

        + + "client, try to work around by setting "
        + + YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT

        Should be yarn.timeline-service.client.best-effort.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for patch. LGTM overall, with a few nits: + LOG.warn("Ignore a NoClassDefFoundError when attempt to get" --> when attempting + + "client, try to work around by setting " + + YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT Should be yarn.timeline-service.client.best-effort.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Wei-Chiu Chuang

        Thanks you for reviewing this. I have fixed the first issue, but second one, are you suggesting to use string yarn.timeline-service.client.best-effort directly instead of using the constant YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT? I am wondering why is that. Thanks

        Show
        cheersyang Weiwei Yang added a comment - Hi Wei-Chiu Chuang Thanks you for reviewing this. I have fixed the first issue, but second one, are you suggesting to use string yarn.timeline-service.client.best-effort directly instead of using the constant YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT ? I am wondering why is that. Thanks
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 12m 33s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 17s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 32s the patch passed
        +1 javadoc 0m 12s the patch passed
        +1 unit 16m 47s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        35m 2s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852533/YARN-6177.05.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 215d7406b65f 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 71c23c9
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14928/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14928/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 33s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 17s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 32s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 16m 47s hadoop-yarn-client in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 35m 2s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12852533/YARN-6177.05.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 215d7406b65f 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 71c23c9 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14928/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14928/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hello Wei-Chiu Chuang and Li Lu

        Just let me know if you have any other comments with v5 patch, thanks!

        Show
        cheersyang Weiwei Yang added a comment - Hello Wei-Chiu Chuang and Li Lu Just let me know if you have any other comments with v5 patch, thanks!
        Hide
        gtCarrera9 Li Lu added a comment -

        Thanks for the hard work Weiwei Yang! Keep using YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT looks fine with me. However, I'm still a little bit hesitate on swallowing the error when timelineServiceBestEffort is set to true. To me handling errors (but not exceptions) is beyond the range of our "best effort". I would like to understand if there's anything I'm missing that makes the community think it is especially appealing to do so.

        Other than this, the patch LGTM.

        Show
        gtCarrera9 Li Lu added a comment - Thanks for the hard work Weiwei Yang ! Keep using YarnConfiguration.TIMELINE_SERVICE_CLIENT_BEST_EFFORT looks fine with me. However, I'm still a little bit hesitate on swallowing the error when timelineServiceBestEffort is set to true. To me handling errors (but not exceptions) is beyond the range of our "best effort". I would like to understand if there's anything I'm missing that makes the community think it is especially appealing to do so. Other than this, the patch LGTM.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Li Lu

        Yes I understand your concern, but i think the patch is providing a reasonable work around if clients ship a jersey2 lib, such as spark2 (more in SPARK-15343), they will 100% hit this. Such services don't really use timeline service but if without this fix, yarn client exits with no information prompted and user needs to study the code, and manually disable timeline client. The error message provided in this patch helps to make this process easier. The patch is not "swallowing" any error (new UT guarantees that), it is just aim to provide useful information plus a work around and user is the decision maker. Does that make sense?

        Show
        cheersyang Weiwei Yang added a comment - Hi Li Lu Yes I understand your concern, but i think the patch is providing a reasonable work around if clients ship a jersey2 lib, such as spark2 (more in SPARK-15343 ), they will 100% hit this. Such services don't really use timeline service but if without this fix, yarn client exits with no information prompted and user needs to study the code, and manually disable timeline client. The error message provided in this patch helps to make this process easier. The patch is not "swallowing" any error (new UT guarantees that), it is just aim to provide useful information plus a work around and user is the decision maker. Does that make sense?
        Hide
        gtCarrera9 Li Lu added a comment -

        Thanks Weiwei Yang. My concern is with these lines:

        379	    } catch (NoClassDefFoundError e) {
        380	      if (timelineServiceBestEffort) {
        381	        LOG.warn("Ignore a NoClassDefFoundError when attempting to get"
        382	            + " delegation token from the timeline server: " + e.getMessage());
        383	        return null;
        384	      }
        385	
        

        So if timelineServiceBestEffort is set to true, we'll leave a message and then proceed? I was think we may not need to treat timelineServiceBestEffort separately here since even with best effort we do not need to keep running on errors.

        Show
        gtCarrera9 Li Lu added a comment - Thanks Weiwei Yang . My concern is with these lines: 379 } catch (NoClassDefFoundError e) { 380 if (timelineServiceBestEffort) { 381 LOG.warn( "Ignore a NoClassDefFoundError when attempting to get" 382 + " delegation token from the timeline server: " + e.getMessage()); 383 return null ; 384 } 385 So if timelineServiceBestEffort is set to true, we'll leave a message and then proceed? I was think we may not need to treat timelineServiceBestEffort separately here since even with best effort we do not need to keep running on errors.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Li Lu

        Yes, if user turns on best effort and it gives a warning message and proceed with this particular error, this is the work around for the issue I described earlier. Option 2 is to disable timeline service in client config, but I think this one is better as this property was introduced to help clients continue to run when the failure with timeline client was not fatal. Without these lines, option 2 is the only work around then.

        Show
        cheersyang Weiwei Yang added a comment - Hi Li Lu Yes, if user turns on best effort and it gives a warning message and proceed with this particular error, this is the work around for the issue I described earlier. Option 2 is to disable timeline service in client config, but I think this one is better as this property was introduced to help clients continue to run when the failure with timeline client was not fatal. Without these lines, option 2 is the only work around then.
        Hide
        gtCarrera9 Li Lu added a comment -

        OK I see. Is it possible to disable timeline service for those affected clients?

        Show
        gtCarrera9 Li Lu added a comment - OK I see. Is it possible to disable timeline service for those affected clients?
        Hide
        cheersyang Weiwei Yang added a comment -

        Hi Li Lu

        Yes, 2 approaches

        1. Set yarn.timeline-service.client.best-effort to true with this patch, so yarn client doesn't treat such failure as a fatal error. Default value is false, so user always hits the error with the prompted message at first time.
        2. Set yarn.timeine-service.enabled to false to disable timeline service from client side config.

        I was suggesting to use #1 because it literally explains the work around better. Disable timeline service is a bit confusing because this property is more like a server side config.

        Show
        cheersyang Weiwei Yang added a comment - Hi Li Lu Yes, 2 approaches Set yarn.timeline-service.client.best-effort to true with this patch, so yarn client doesn't treat such failure as a fatal error. Default value is false, so user always hits the error with the prompted message at first time. Set yarn.timeine-service.enabled to false to disable timeline service from client side config. I was suggesting to use #1 because it literally explains the work around better. Disable timeline service is a bit confusing because this property is more like a server side config.
        Hide
        gtCarrera9 Li Lu added a comment -

        Set yarn.timeline-service.client.best-effort to true with this patch, so yarn client doesn't treat such failure as a fatal error.

        This is actually my concern... My feeling is we may not want dealing Errors as a part of best effort. Not sure about this, cc/Jason Lowe...
        Hi Jason, I saw you committed the original timelineBestEffort patch, so just a quick inquiry to see if you think handling this Error under best effort mode a good idea. Thanks!

        Show
        gtCarrera9 Li Lu added a comment - Set yarn.timeline-service.client.best-effort to true with this patch, so yarn client doesn't treat such failure as a fatal error. This is actually my concern... My feeling is we may not want dealing Errors as a part of best effort. Not sure about this, cc/ Jason Lowe ... Hi Jason, I saw you committed the original timelineBestEffort patch, so just a quick inquiry to see if you think handling this Error under best effort mode a good idea. Thanks!
        Hide
        jlowe Jason Lowe added a comment -

        The best effort setting was primarily targeting the scenario where the timeline server is down and not wanting jobs to fail as a result. It wasn't intended to mask jar conflicts like this.

        Maybe I'm misunderstanding option #1 from above, but we should not be changing the best-effort default value. That's an incompatible and surprising change.

        When someone sets best-effort to true then they desire the timeline server to work but don't want transient errors to block the job's progress. This case is not a transient error – the timeline client is never going to work with that jar conflict in place. Therefore I agree with Li Lu that we should not mask this error. If it's common then I think we should check for it and provide a useful error message stating the user needs to address the classpath conflict or disable the timeline client, but the error should still be fatal.

        Show
        jlowe Jason Lowe added a comment - The best effort setting was primarily targeting the scenario where the timeline server is down and not wanting jobs to fail as a result. It wasn't intended to mask jar conflicts like this. Maybe I'm misunderstanding option #1 from above, but we should not be changing the best-effort default value. That's an incompatible and surprising change. When someone sets best-effort to true then they desire the timeline server to work but don't want transient errors to block the job's progress. This case is not a transient error – the timeline client is never going to work with that jar conflict in place. Therefore I agree with Li Lu that we should not mask this error. If it's common then I think we should check for it and provide a useful error message stating the user needs to address the classpath conflict or disable the timeline client, but the error should still be fatal.
        Hide
        cheersyang Weiwei Yang added a comment -

        Hello Jason Lowe

        Thanks for the input, it's quite helpful. Option #1 was not to change best-effort default value, I meant to say with the patch provided in this ticket, user will be able to work around this jar conflict problem if they want to set "yarn.timeline-service.client.best-effort" to "true". If both of you think the best-effort property was not suitable to handle an error like this, let me upload a new patch and just give the error message (suggest user to disable timeline service to work around). Thanks you!

        Show
        cheersyang Weiwei Yang added a comment - Hello Jason Lowe Thanks for the input, it's quite helpful. Option #1 was not to change best-effort default value, I meant to say with the patch provided in this ticket, user will be able to work around this jar conflict problem if they want to set "yarn.timeline-service.client.best-effort" to "true". If both of you think the best-effort property was not suitable to handle an error like this, let me upload a new patch and just give the error message (suggest user to disable timeline service to work around). Thanks you!
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 12m 55s trunk passed
        +1 compile 0m 19s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 21s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 19s the patch passed
        +1 compile 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 19s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 34s the patch passed
        +1 javadoc 0m 11s the patch passed
        +1 unit 16m 51s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        35m 29s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-6177
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12853093/YARN-6177.06.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 44ad5624d182 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 5690b51
        Default Java 1.8.0_121
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14982/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14982/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 55s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 34s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 16m 51s hadoop-yarn-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 35m 29s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6177 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12853093/YARN-6177.06.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 44ad5624d182 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5690b51 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14982/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14982/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        gtCarrera9 Li Lu added a comment -

        LGTM. Will commit in a few hours if nobody objects.

        Show
        gtCarrera9 Li Lu added a comment - LGTM. Will commit in a few hours if nobody objects.
        Hide
        gtCarrera9 Li Lu added a comment -

        Committing...

        Show
        gtCarrera9 Li Lu added a comment - Committing...
        Hide
        gtCarrera9 Li Lu added a comment -

        Committed to trunk, branch-2, and branch-2.8. Thanks Weiwei Yang for the work! Thanks Jason Lowe and Wei-Chiu Chuang for the review!

        Show
        gtCarrera9 Li Lu added a comment - Committed to trunk, branch-2, and branch-2.8. Thanks Weiwei Yang for the work! Thanks Jason Lowe and Wei-Chiu Chuang for the review!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11267 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11267/)
        YARN-6177. Yarn client should exit with an informative error message if (gtcarrera9: rev 5d339c46f5b16b951afd82afd9e907b9aa2ded9a)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11267 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11267/ ) YARN-6177 . Yarn client should exit with an informative error message if (gtcarrera9: rev 5d339c46f5b16b951afd82afd9e907b9aa2ded9a) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestYarnClient.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
        Hide
        cheersyang Weiwei Yang added a comment -

        Thanks Li Lu, Jason Lowe and Wei-Chiu Chuang for all the comments and suggestions!

        Show
        cheersyang Weiwei Yang added a comment - Thanks Li Lu , Jason Lowe and Wei-Chiu Chuang for all the comments and suggestions!

          People

          • Assignee:
            cheersyang Weiwei Yang
            Reporter:
            cheersyang Weiwei Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development