Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4904 YARN Log tooling enhancements
  3. YARN-5524

Yarn live log aggregation does not throw if command line arg is wrong

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: log-aggregation
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When we used wrong command line arg for specify log file pattern, yarn did not throw any exception instead it pulled entire log onto the client.

      [hive@ctr-e20-1468887904486-0007-01-000003 ~]$ yarn logs -applicationId application_1470931023753_0001 -logFiles hive.*2016-08-11-21.* > logs
      

      NOTE: we are using -logFiles instead of -log_files

      This query should have failed.

      1. YARN-5524.001.patch
        4 kB
        Vrushali C
      2. YARN-5524.002.patch
        4 kB
        Vrushali C
      3. YARN-5524.3.patch
        2 kB
        Xuan Gong

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11012 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11012/)
        YARN-5524. Yarn live log aggregation does not throw if command line arg (naganarasimha_gr: rev 358341398adfe8f59bb2165472b50751de3ffb98)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11012 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11012/ ) YARN-5524 . Yarn live log aggregation does not throw if command line arg (naganarasimha_gr: rev 358341398adfe8f59bb2165472b50751de3ffb98) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the contribution Xuan Gong & initially by Vrushali C. Have committed it to trunk and branch-2.

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the contribution Xuan Gong & initially by Vrushali C . Have committed it to trunk and branch-2.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks Vrushali C, as the approach and the patch was given out by Xuan Gong, i have assigned the jira to him and as the patch LGTM will commit it shortly!

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks Vrushali C , as the approach and the patch was given out by Xuan Gong , i have assigned the jira to him and as the patch LGTM will commit it shortly!
        Hide
        vrushalic Vrushali C added a comment -

        Sorry for the delay, please feel free to reassign the jira to yourself, Xuan Gong

        Show
        vrushalic Vrushali C added a comment - Sorry for the delay, please feel free to reassign the jira to yourself, Xuan Gong
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 50s trunk passed
        +1 compile 0m 30s trunk passed
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 0m 38s trunk passed
        +1 javadoc 0m 18s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 19s the patch passed
        +1 javac 0m 19s the patch passed
        -0 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 3 new + 99 unchanged - 0 fixed = 102 total (was 99)
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 36s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 16m 25s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        39m 10s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5524
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843778/YARN-5524.3.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 60164caa3a43 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ef2dd7b
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14364/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14364/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14364/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 50s trunk passed +1 compile 0m 30s trunk passed +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 38s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed -0 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 3 new + 99 unchanged - 0 fixed = 102 total (was 99) +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 16m 25s hadoop-yarn-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 39m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5524 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843778/YARN-5524.3.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 60164caa3a43 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ef2dd7b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14364/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14364/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/14364/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the patch Xuan Gong, applied the patch and tested it and its able to show the invalid option too.
        Once the jenkins build passes can commit it !

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the patch Xuan Gong , applied the patch and tested it and its able to show the invalid option too. Once the jenkins build passes can commit it !
        Hide
        xgong Xuan Gong added a comment -

        Upload a new patch to fix the issue.

        Show
        xgong Xuan Gong added a comment - Upload a new patch to fix the issue.
        Hide
        xgong Xuan Gong added a comment -

        any update Vrushali C ?

        Show
        xgong Xuan Gong added a comment - any update Vrushali C ?
        Hide
        xgong Xuan Gong added a comment -

        Vrushali C

        Instead of doing

        String unknownOptions = StringUtils.join(commandLine.getArgs(), ' ');
        if (StringUtils.isNotBlank(unknownOptions)) {
          System.err.println("Invalid option(s) specified " + unknownOptions);
          return -1;
        }
        

        we could change

        CommandLine commandLine = parser.parse(opts, args, true);
        to
        CommandLine commandLine = parser.parse(opts, args, false);
        

        So, the GnuParser would detect invalid option by itself.

        Show
        xgong Xuan Gong added a comment - Vrushali C Instead of doing String unknownOptions = StringUtils.join(commandLine.getArgs(), ' '); if (StringUtils.isNotBlank(unknownOptions)) { System .err.println( "Invalid option(s) specified " + unknownOptions); return -1; } we could change CommandLine commandLine = parser.parse(opts, args, true ); to CommandLine commandLine = parser.parse(opts, args, false ); So, the GnuParser would detect invalid option by itself.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Vrushali C,
        IMO i felt it should be CommandLineParser's responsibility to detect invalid option, if validation is not happening would it not be better to support it there so that its applicable across all the CLI Command?

        Show
        Naganarasimha Naganarasimha G R added a comment - Vrushali C , IMO i felt it should be CommandLineParser's responsibility to detect invalid option, if validation is not happening would it not be better to support it there so that its applicable across all the CLI Command?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 48s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 98 unchanged - 1 fixed = 99 total (was 99)
        +1 mvnsite 0m 21s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 32s the patch passed
        +1 javadoc 0m 11s the patch passed
        -1 unit 15m 52s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        27m 45s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823774/YARN-5524.002.patch
        JIRA Issue YARN-5524
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d437d2735b9a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 864f878
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12780/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12780/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 48s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 98 unchanged - 1 fixed = 99 total (was 99) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 32s the patch passed +1 javadoc 0m 11s the patch passed -1 unit 15m 52s hadoop-yarn-client in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 45s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823774/YARN-5524.002.patch JIRA Issue YARN-5524 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d437d2735b9a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 864f878 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12780/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12780/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12780/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 57s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 26s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 98 unchanged - 1 fixed = 99 total (was 99)
        +1 mvnsite 0m 22s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 12s the patch passed
        -1 unit 16m 4s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        28m 11s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823774/YARN-5524.002.patch
        JIRA Issue YARN-5524
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 46a821413a26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 864f878
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12777/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12777/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 57s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 26s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 17s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 98 unchanged - 1 fixed = 99 total (was 99) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 12s the patch passed -1 unit 16m 4s hadoop-yarn-client in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 28m 11s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823774/YARN-5524.002.patch JIRA Issue YARN-5524 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 46a821413a26 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 864f878 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12777/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12777/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12777/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch v2 to address checkstyle warnings (although first patch followed the existing coding style).

        Not sure if the checkstyle warning about function length is to be fixed in this patch. The LogsCLI#run function without this patch is already longer than 150 lines.

        Show
        vrushalic Vrushali C added a comment - Uploading patch v2 to address checkstyle warnings (although first patch followed the existing coding style). Not sure if the checkstyle warning about function length is to be fixed in this patch. The LogsCLI#run function without this patch is already longer than 150 lines.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 35s trunk passed
        +1 compile 0m 25s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 32s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 39s trunk passed
        +1 javadoc 0m 17s trunk passed
        +1 mvninstall 0m 23s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        -1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 5 new + 98 unchanged - 1 fixed = 103 total (was 99)
        +1 mvnsite 0m 26s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 41s the patch passed
        +1 javadoc 0m 15s the patch passed
        +1 unit 16m 26s hadoop-yarn-client in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        31m 24s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823766/YARN-5524.001.patch
        JIRA Issue YARN-5524
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c56ffd212dd5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 03dea65
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12776/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12776/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12776/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 35s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 39s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed -1 checkstyle 0m 15s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 5 new + 98 unchanged - 1 fixed = 103 total (was 99) +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 41s the patch passed +1 javadoc 0m 15s the patch passed +1 unit 16m 26s hadoop-yarn-client in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 31m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823766/YARN-5524.001.patch JIRA Issue YARN-5524 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c56ffd212dd5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 03dea65 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12776/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12776/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12776/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading v1.

        Prasanth Jayachandran does this seem to be along the lines you were thinking? This will check for extra arguments and return an exit of -1 (similar to other incorrect argument situations). Added some test cases.

        Show
        vrushalic Vrushali C added a comment - Uploading v1. Prasanth Jayachandran does this seem to be along the lines you were thinking? This will check for extra arguments and return an exit of -1 (similar to other incorrect argument situations). Added some test cases.
        Hide
        vrushalic Vrushali C added a comment -

        Hmm, I get your point about pulling huge logs. Alright, let me think over this and see if I can add the "unrecognized option" check.

        Also, fixing this means it would break existing behavior for folks who perhaps have been running (unintentionally) with incorrect/extra options.

        Show
        vrushalic Vrushali C added a comment - Hmm, I get your point about pulling huge logs. Alright, let me think over this and see if I can add the "unrecognized option" check. Also, fixing this means it would break existing behavior for folks who perhaps have been running (unintentionally) with incorrect/extra options.
        Hide
        prasanth_j Prasanth Jayachandran added a comment -

        In hive LLAP, we have long running daemons that generate lot of logs. If user explicitly specifies a log file option then the user's intention is fetch the specific logs and not all logs. Also it is easy to provide to wrong option or some typo in options which should not end up pulling all the logs which could be huge. I agree that we should fetch all logs if no options are provided. But if explicit option is provided, it should at least validate the provided option and say "Unrecognized option". I am pretty sure, if we mistype "yarn application -kill" it's not going to kill all applications.

        Show
        prasanth_j Prasanth Jayachandran added a comment - In hive LLAP, we have long running daemons that generate lot of logs. If user explicitly specifies a log file option then the user's intention is fetch the specific logs and not all logs. Also it is easy to provide to wrong option or some typo in options which should not end up pulling all the logs which could be huge. I agree that we should fetch all logs if no options are provided. But if explicit option is provided, it should at least validate the provided option and say "Unrecognized option". I am pretty sure, if we mistype "yarn application -kill" it's not going to kill all applications.
        Hide
        vrushalic Vrushali C added a comment -

        Based on what I am seeing, I don't think this should throw an exception. This was intentionally added as part of YARN-5363 by Vinod Kumar Vavilapalli

        In patch https://issues.apache.org/jira/secure/attachment/12817883/YARN-5363-2016-07-13.1.txt
        you can see that in LogsCLI# fetchAllLogFiles, the default is set to fetch all logs if no option is specified. The fact that the option is spelled differently does not imply the meaning.

        And for what it's worth, looks like the option changed (correctly) from "-logFiles" to "-log_files" in this patch, so perhaps your command was present in history and you happened to run it.

        I think this should be closed as "Not a problem". What do you think Prashanth?

        Show
        vrushalic Vrushali C added a comment - Based on what I am seeing, I don't think this should throw an exception. This was intentionally added as part of YARN-5363 by Vinod Kumar Vavilapalli In patch https://issues.apache.org/jira/secure/attachment/12817883/YARN-5363-2016-07-13.1.txt you can see that in LogsCLI# fetchAllLogFiles, the default is set to fetch all logs if no option is specified. The fact that the option is spelled differently does not imply the meaning. And for what it's worth, looks like the option changed (correctly) from "-logFiles" to "-log_files" in this patch, so perhaps your command was present in history and you happened to run it. I think this should be closed as "Not a problem". What do you think Prashanth?
        Hide
        vrushalic Vrushali C added a comment -

        Looking into this.

        Show
        vrushalic Vrushali C added a comment - Looking into this.

          People

          • Assignee:
            xgong Xuan Gong
            Reporter:
            prasanth_j Prasanth Jayachandran
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development