Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6652

Add configuration property to prevent JHS from loading jobs with a task count greater than X

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: jobhistoryserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Jobs with large number of tasks can have job history files that are large in size and resource-consuming(mainly memory) to parse in Job History Server. If there are many such jobs, the job history server can very easily hang.

      It would be a good usability feature if we added a new config property that could be set to X, where the JHS wouldn't load the details for a job with more than X tasks. The job would still show up on the list of jobs page, but clicking on it would give a warning message that the job is too big, instead of actually loading the job. This way we can prevent users from loading a job that's way too big for the JHS, which currently makes the JHS hang. The default value can be -1 so that it's disabled.

      1. mapreduce6652.001.patch
        26 kB
        Haibo Chen
      2. mapreduce6652.002.patch
        27 kB
        Haibo Chen
      3. mapreduce6652.003.patch
        27 kB
        Haibo Chen
      4. mapreduce6652.004.patch
        27 kB
        Haibo Chen
      5. mapreduce6652.005.patch
        29 kB
        Haibo Chen
      6. mapreduce6652.007.branch2.patch
        29 kB
        Haibo Chen
      7. mapreduce6652.007.patch
        29 kB
        Haibo Chen
      8. mapreduce6652.008.branch2.patch
        29 kB
        Haibo Chen
      9. mapreduce6652.008.patch
        29 kB
        Haibo Chen
      10. mapreduce6652.009.branch2.patch
        29 kB
        Haibo Chen
      11. mapreduce6652.009.patch
        29 kB
        Haibo Chen

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 6m 50s trunk passed
        +1 compile 1m 33s trunk passed with JDK v1.8.0_74
        +1 compile 1m 44s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 1m 25s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 2m 26s trunk passed
        +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 7s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 5s the patch passed
        +1 compile 1m 24s the patch passed with JDK v1.8.0_74
        +1 javac 1m 24s the patch passed
        +1 compile 1m 39s the patch passed with JDK v1.7.0_95
        +1 javac 1m 39s the patch passed
        -1 checkstyle 0m 23s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 4 new + 73 unchanged - 0 fixed = 77 total (was 73)
        +1 mvnsite 1m 16s the patch passed
        +1 mvneclipse 0m 35s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 2m 52s the patch passed
        +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74
        +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95
        -1 unit 1m 56s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_74.
        +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_74.
        -1 unit 5m 41s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_74.
        +1 unit 2m 20s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95.
        +1 unit 0m 49s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95.
        -1 unit 6m 26s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 21s Patch generated 14 ASF License warnings.
        48m 17s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.mapreduce.tools.TestCLI
          hadoop.mapreduce.v2.hs.TestHistoryFileManager
        JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryServer
        JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.hs.TestHistoryFileManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794053/mapreduce6652.001.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux b3fe17bc96c0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / dc951e6
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/testReport/
        asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 50s trunk passed +1 compile 1m 33s trunk passed with JDK v1.8.0_74 +1 compile 1m 44s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 25s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 2m 26s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 7s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 5s the patch passed +1 compile 1m 24s the patch passed with JDK v1.8.0_74 +1 javac 1m 24s the patch passed +1 compile 1m 39s the patch passed with JDK v1.7.0_95 +1 javac 1m 39s the patch passed -1 checkstyle 0m 23s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 4 new + 73 unchanged - 0 fixed = 77 total (was 73) +1 mvnsite 1m 16s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 52s the patch passed +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95 -1 unit 1m 56s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_74. +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_74. -1 unit 5m 41s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_74. +1 unit 2m 20s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95. +1 unit 0m 49s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95. -1 unit 6m 26s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 21s Patch generated 14 ASF License warnings. 48m 17s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.mapreduce.tools.TestCLI   hadoop.mapreduce.v2.hs.TestHistoryFileManager JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryServer JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.hs.TestHistoryFileManager Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794053/mapreduce6652.001.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b3fe17bc96c0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dc951e6 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6382/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        The testCLI failure is unrelated. https://issues.apache.org/jira/browse/MAPREDUCE-6625 reported the issue.

        Show
        haibochen Haibo Chen added a comment - The testCLI failure is unrelated. https://issues.apache.org/jira/browse/MAPREDUCE-6625 reported the issue.
        Hide
        haibochen Haibo Chen added a comment -

        addressed the issues in Hadoop QA comments

        Show
        haibochen Haibo Chen added a comment - addressed the issues in Hadoop QA comments
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 21s Maven dependency ordering for branch
        +1 mvninstall 6m 51s trunk passed
        +1 compile 1m 29s trunk passed with JDK v1.8.0_74
        +1 compile 1m 41s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 1m 17s trunk passed
        +1 mvneclipse 0m 44s trunk passed
        +1 findbugs 2m 43s trunk passed
        +1 javadoc 1m 16s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 30s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 1m 10s the patch passed
        +1 compile 1m 24s the patch passed with JDK v1.8.0_74
        +1 javac 1m 24s the patch passed
        +1 compile 1m 38s the patch passed with JDK v1.7.0_95
        +1 javac 1m 38s the patch passed
        +1 checkstyle 0m 23s the patch passed
        +1 mvnsite 1m 12s the patch passed
        +1 mvneclipse 0m 34s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 2m 52s the patch passed
        +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95
        +1 unit 1m 56s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_74.
        +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_74.
        +1 unit 6m 0s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_74.
        +1 unit 2m 18s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95.
        +1 unit 0m 46s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95.
        +1 unit 6m 20s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95.
        -1 asflicense 0m 20s Patch generated 14 ASF License warnings.
        49m 44s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794264/mapreduce6652.002.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux ea7c3fcf1062 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 33239c9
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/testReport/
        asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 21s Maven dependency ordering for branch +1 mvninstall 6m 51s trunk passed +1 compile 1m 29s trunk passed with JDK v1.8.0_74 +1 compile 1m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 2m 43s trunk passed +1 javadoc 1m 16s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 30s trunk passed with JDK v1.7.0_95 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 10s the patch passed +1 compile 1m 24s the patch passed with JDK v1.8.0_74 +1 javac 1m 24s the patch passed +1 compile 1m 38s the patch passed with JDK v1.7.0_95 +1 javac 1m 38s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 12s the patch passed +1 mvneclipse 0m 34s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 52s the patch passed +1 javadoc 0m 48s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95 +1 unit 1m 56s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_74. +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_74. +1 unit 6m 0s hadoop-mapreduce-client-hs in the patch passed with JDK v1.8.0_74. +1 unit 2m 18s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95. +1 unit 0m 46s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95. +1 unit 6m 20s hadoop-mapreduce-client-hs in the patch passed with JDK v1.7.0_95. -1 asflicense 0m 20s Patch generated 14 ASF License warnings. 49m 44s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12794264/mapreduce6652.002.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ea7c3fcf1062 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 33239c9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6385/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        The afs license warning was generated due to leftover files by MiniDFSCluster. The issues is addressed at MAPREDUCE-6662

        Show
        haibochen Haibo Chen added a comment - The afs license warning was generated due to leftover files by MiniDFSCluster. The issues is addressed at MAPREDUCE-6662
        Hide
        rchiang Ray Chiang added a comment -

        Looks pretty good to me. A few minor nits/suggestions:

        • Rename oversized() to isOversized()
        • In the return value for oversized(), please add parentheses
        • In HsJobBlock#render(), some cleanup and add one other actionable item for supportability:
        html.p()._("The job has a total of " + taskCount + " tasks. ")
            ._("Any job larger than " + oversizedJob.getMaxTasksAllowed() +
            " will not be loaded.")._();
        html.p()._("You can either use the command line tool: 'mapred job -history'"
            + " to view large jobs or adjust the property " +
            JHAdminConfig.MR_HS_LOADED_JOBS_TASKS_MAX + ".")._();
        
        Show
        rchiang Ray Chiang added a comment - Looks pretty good to me. A few minor nits/suggestions: Rename oversized() to isOversized() In the return value for oversized(), please add parentheses In HsJobBlock#render(), some cleanup and add one other actionable item for supportability: html.p()._("The job has a total of " + taskCount + " tasks. ") ._("Any job larger than " + oversizedJob.getMaxTasksAllowed() + " will not be loaded.")._(); html.p()._("You can either use the command line tool: 'mapred job -history'" + " to view large jobs or adjust the property " + JHAdminConfig.MR_HS_LOADED_JOBS_TASKS_MAX + ".")._();
        Hide
        haibochen Haibo Chen added a comment -

        Thanks for your review, Ray. Have updated the patch to reflect your comments.

        Show
        haibochen Haibo Chen added a comment - Thanks for your review, Ray. Have updated the patch to reflect your comments.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 6m 40s trunk passed
        +1 compile 1m 33s trunk passed with JDK v1.8.0_91
        +1 compile 1m 43s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 1m 17s trunk passed
        +1 mvneclipse 0m 36s trunk passed
        +1 findbugs 2m 20s trunk passed
        +1 javadoc 0m 54s trunk passed with JDK v1.8.0_91
        +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 3s the patch passed
        +1 compile 1m 31s the patch passed with JDK v1.8.0_91
        +1 javac 1m 31s the patch passed
        +1 compile 1m 39s the patch passed with JDK v1.7.0_95
        +1 javac 1m 39s the patch passed
        -1 checkstyle 0m 27s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 17 new + 77 unchanged - 2 fixed = 94 total (was 79)
        +1 mvnsite 1m 11s the patch passed
        +1 mvneclipse 0m 32s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 2m 54s the patch passed
        +1 javadoc 0m 47s the patch passed with JDK v1.8.0_91
        +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95
        +1 unit 1m 54s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_91.
        +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91.
        -1 unit 6m 2s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_91.
        +1 unit 2m 18s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95.
        +1 unit 0m 45s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95.
        -1 unit 6m 15s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        47m 41s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.mapreduce.v2.hs.webapp.TestHsJobBlock
        JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.hs.webapp.TestHsJobBlock



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803032/mapreduce6652.003.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux c757df008ea1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 411fb4b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 40s trunk passed +1 compile 1m 33s trunk passed with JDK v1.8.0_91 +1 compile 1m 43s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 2m 20s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 1m 31s the patch passed with JDK v1.8.0_91 +1 javac 1m 31s the patch passed +1 compile 1m 39s the patch passed with JDK v1.7.0_95 +1 javac 1m 39s the patch passed -1 checkstyle 0m 27s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 17 new + 77 unchanged - 2 fixed = 94 total (was 79) +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 2m 54s the patch passed +1 javadoc 0m 47s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95 +1 unit 1m 54s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_91. +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91. -1 unit 6m 2s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_91. +1 unit 2m 18s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95. +1 unit 0m 45s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95. -1 unit 6m 15s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 47m 41s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.mapreduce.v2.hs.webapp.TestHsJobBlock JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.hs.webapp.TestHsJobBlock Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803032/mapreduce6652.003.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux c757df008ea1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 411fb4b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6487/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        looks like new checkstyle checkings have been introduced. Updated my patch accordingly

        Show
        haibochen Haibo Chen added a comment - looks like new checkstyle checkings have been introduced. Updated my patch accordingly
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 30s trunk passed
        +1 compile 1m 23s trunk passed with JDK v1.8.0_91
        +1 compile 1m 37s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 1m 17s trunk passed
        +1 mvneclipse 0m 39s trunk passed
        +1 findbugs 2m 19s trunk passed
        +1 javadoc 0m 52s trunk passed with JDK v1.8.0_91
        +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 2s the patch passed
        +1 compile 1m 24s the patch passed with JDK v1.8.0_91
        +1 javac 1m 24s the patch passed
        +1 compile 1m 37s the patch passed with JDK v1.7.0_95
        +1 javac 1m 37s the patch passed
        -1 checkstyle 0m 27s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79)
        +1 mvnsite 1m 11s the patch passed
        +1 mvneclipse 0m 33s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 47s the patch passed
        +1 javadoc 0m 46s the patch passed with JDK v1.8.0_91
        +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95
        -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_91.
        +1 unit 0m 39s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91.
        -1 unit 20m 14s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_91.
        +1 unit 2m 19s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95.
        +1 unit 0m 46s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95.
        -1 unit 20m 38s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        75m 42s



        Reason Tests
        JDK v1.8.0_91 Failed junit tests hadoop.mapreduce.tools.TestCLI
        JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager
        JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803091/mapreduce6652.004.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux ef5e2b740866 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 996a210
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 30s trunk passed +1 compile 1m 23s trunk passed with JDK v1.8.0_91 +1 compile 1m 37s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 2m 19s trunk passed +1 javadoc 0m 52s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 2s the patch passed +1 compile 1m 24s the patch passed with JDK v1.8.0_91 +1 javac 1m 24s the patch passed +1 compile 1m 37s the patch passed with JDK v1.7.0_95 +1 javac 1m 37s the patch passed -1 checkstyle 0m 27s hadoop-mapreduce-project/hadoop-mapreduce-client: patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 47s the patch passed +1 javadoc 0m 46s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 59s the patch passed with JDK v1.7.0_95 -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_91. +1 unit 0m 39s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_91. -1 unit 20m 14s hadoop-mapreduce-client-hs in the patch failed with JDK v1.8.0_91. +1 unit 2m 19s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_95. +1 unit 0m 46s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_95. -1 unit 20m 38s hadoop-mapreduce-client-hs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 75m 42s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.mapreduce.tools.TestCLI JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803091/mapreduce6652.004.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ef5e2b740866 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 996a210 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6489/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        unit test failures are unrelated.

        Show
        haibochen Haibo Chen added a comment - unit test failures are unrelated.
        Hide
        rkanter Robert Kanter added a comment -

        Looks good overall. A few things:

        • In mapred-default, let's change "-1 as the default value to disable the settings." to "A value of -1 (default) will allow all jobs to be loaded".
        • In TestHistoryFileManager#testHistoryFileInfoLoadNormalSizedJobShouldReturnCompletedJob the comment says "set up a job of which the number of tasks is greater than maximum allowed". However, the max tasks is set to "-1", as in there is no maximum allowed. The comment should be corrected.
          • It would also be good to add a third test where the max tasks is set to something and the job has fewer tasks than the maximum so it gets loaded
        • If you look at CompletedJob, it saves the JobReport so we don't make a new one each time getReport() is called. While constructing the UnparsedJob report is cheaper than that for the CompletedJob, it's still probably a good idea to do this in the UnparsedJob to, to save on object creation.
        • UnparsedJob#getProgress() should return 1.0f instead of -1. The job has finished, so it's at 100%.
        Show
        rkanter Robert Kanter added a comment - Looks good overall. A few things: In mapred-default, let's change "-1 as the default value to disable the settings." to "A value of -1 (default) will allow all jobs to be loaded". In TestHistoryFileManager#testHistoryFileInfoLoadNormalSizedJobShouldReturnCompletedJob the comment says "set up a job of which the number of tasks is greater than maximum allowed". However, the max tasks is set to "-1", as in there is no maximum allowed. The comment should be corrected. It would also be good to add a third test where the max tasks is set to something and the job has fewer tasks than the maximum so it gets loaded If you look at CompletedJob , it saves the JobReport so we don't make a new one each time getReport() is called. While constructing the UnparsedJob report is cheaper than that for the CompletedJob , it's still probably a good idea to do this in the UnparsedJob to, to save on object creation. UnparsedJob#getProgress() should return 1.0f instead of -1 . The job has finished, so it's at 100%.
        Hide
        haibochen Haibo Chen added a comment -

        Thanks Robert Kanter for your reviews. I have updated the patch according to your comments, and also added a new test case as you suggested.

        Show
        haibochen Haibo Chen added a comment - Thanks Robert Kanter for your reviews. I have updated the patch according to your comments, and also added a new test case as you suggested.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 docker 0m 5s Docker failed to build yetus/hadoop:2c91fd8.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808485/mapreduce6652.005.patch
        JIRA Issue MAPREDUCE-6652
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6538/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 0m 5s Docker failed to build yetus/hadoop:2c91fd8. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808485/mapreduce6652.005.patch JIRA Issue MAPREDUCE-6652 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6538/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        Two last things

        • UnparsedJob#getReport() needs to be synchronized now that we're storing the JobReport.
        • It doesn't apply to branch-2. Can you create a branch-2 version of the patch? It looks like a trivial conflict though.
        Show
        rkanter Robert Kanter added a comment - Two last things UnparsedJob#getReport() needs to be synchronized now that we're storing the JobReport . It doesn't apply to branch-2. Can you create a branch-2 version of the patch? It looks like a trivial conflict though.
        Hide
        haibochen Haibo Chen added a comment -

        Apologize for the delay. Attaching one file for trunk, the other for branch-2

        Show
        haibochen Haibo Chen added a comment - Apologize for the delay. Attaching one file for trunk, the other for branch-2
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 6m 31s trunk passed
        +1 compile 1m 33s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 1m 11s trunk passed
        +1 mvneclipse 0m 39s trunk passed
        +1 findbugs 1m 52s trunk passed
        +1 javadoc 0m 55s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 0m 56s the patch passed
        +1 compile 1m 30s the patch passed
        +1 javac 1m 30s the patch passed
        -1 checkstyle 0m 23s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79)
        +1 mvnsite 1m 5s the patch passed
        +1 mvneclipse 0m 31s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 10s the patch passed
        -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198)
        +1 unit 1m 57s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed.
        -1 unit 21m 10s hadoop-mapreduce-client-hs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        46m 8s



        Reason Tests
        Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e2f6409
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811169/mapreduce6652.007.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 0088b6343379 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 127d2c7
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 31s trunk passed +1 compile 1m 33s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 0m 55s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 56s the patch passed +1 compile 1m 30s the patch passed +1 javac 1m 30s the patch passed -1 checkstyle 0m 23s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) +1 mvnsite 1m 5s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 10s the patch passed -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198) +1 unit 1m 57s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed. -1 unit 21m 10s hadoop-mapreduce-client-hs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 46m 8s Reason Tests Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811169/mapreduce6652.007.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 0088b6343379 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 127d2c7 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6561/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        The javadoc and unit test issues are not related.

        Show
        haibochen Haibo Chen added a comment - The javadoc and unit test issues are not related.
        Hide
        rkanter Robert Kanter added a comment -

        +1

        Show
        rkanter Robert Kanter added a comment - +1
        Hide
        rchiang Ray Chiang added a comment - - edited

        Really minor Javadoc nits

        In HistoryFileManager.java

        1. greater than maxTaskForLoadedJob. should be greater than maxTasksForLoadedJob.

        2. Phrasing and punctuation should be cleaned up here:

        @return null if the underlying file was deleted.
        partially parsed job if the job has too many tasks
        fully parsed job otherwise

        will read nicer:

        @return null if the underlying job history file was deleted, an {@link UnparsedJob}
        object representing a partially parsed job if the job tasks exceeds the
        configured maximum, or a {@link CompletedJob} representing a fully
        parsed job.

        Show
        rchiang Ray Chiang added a comment - - edited Really minor Javadoc nits In HistoryFileManager.java 1. greater than maxTaskForLoadedJob. should be greater than maxTasksForLoadedJob. 2. Phrasing and punctuation should be cleaned up here: @return null if the underlying file was deleted. partially parsed job if the job has too many tasks fully parsed job otherwise will read nicer: @return null if the underlying job history file was deleted, an {@link UnparsedJob} object representing a partially parsed job if the job tasks exceeds the configured maximum, or a {@link CompletedJob} representing a fully parsed job.
        Hide
        haibochen Haibo Chen added a comment -

        Thanks @Ray for your comments, uploading a new version including the javadoc changes.

        Show
        haibochen Haibo Chen added a comment - Thanks @Ray for your comments, uploading a new version including the javadoc changes.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 5s MAPREDUCE-6652 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817535/mapreduce6652.008.branch2.patch
        JIRA Issue MAPREDUCE-6652
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6608/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s MAPREDUCE-6652 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817535/mapreduce6652.008.branch2.patch JIRA Issue MAPREDUCE-6652 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6608/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        Retriggered build. Will upload branch-2 patch after the build is done.

        Show
        haibochen Haibo Chen added a comment - Retriggered build. Will upload branch-2 patch after the build is done.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 8m 17s trunk passed
        +1 compile 1m 55s trunk passed
        +1 checkstyle 0m 32s trunk passed
        +1 mvnsite 1m 23s trunk passed
        +1 mvneclipse 0m 44s trunk passed
        +1 findbugs 2m 13s trunk passed
        +1 javadoc 1m 0s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 56s the patch passed
        +1 compile 1m 32s the patch passed
        +1 javac 1m 32s the patch passed
        -1 checkstyle 0m 25s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79)
        +1 mvnsite 1m 9s the patch passed
        +1 mvneclipse 0m 31s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 2m 9s the patch passed
        -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198)
        +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed.
        -1 unit 20m 53s hadoop-mapreduce-client-hs in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        48m 56s



        Reason Tests
        Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817544/mapreduce6652.008.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 1f78ca268221 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ad256be
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 8m 17s trunk passed +1 compile 1m 55s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 23s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 1m 0s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 56s the patch passed +1 compile 1m 32s the patch passed +1 javac 1m 32s the patch passed -1 checkstyle 0m 25s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) +1 mvnsite 1m 9s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 9s the patch passed -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198) +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 41s hadoop-mapreduce-client-common in the patch passed. -1 unit 20m 53s hadoop-mapreduce-client-hs in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 48m 56s Reason Tests Timed out junit tests org.apache.hadoop.mapreduce.v2.hs.TestHistoryFileManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817544/mapreduce6652.008.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 1f78ca268221 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ad256be Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6611/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        haibochen Haibo Chen added a comment -

        Unit test failed because of time out.

        Show
        haibochen Haibo Chen added a comment - Unit test failed because of time out.
        Hide
        rkanter Robert Kanter added a comment -

        +1

        Show
        rkanter Robert Kanter added a comment - +1
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 5s MAPREDUCE-6652 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817836/mapreduce6652.008.branch2.patch
        JIRA Issue MAPREDUCE-6652
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6613/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s MAPREDUCE-6652 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817836/mapreduce6652.008.branch2.patch JIRA Issue MAPREDUCE-6652 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6613/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        Haibo Chen, can you take a look at TestHsJobBlock? The first time I ran it, it timed out. The second time, it really slowed down my computer after a while (eating all my RAM I suppose), and I had to ctrl-c it, which wasn't easy.

        Show
        rkanter Robert Kanter added a comment - Haibo Chen , can you take a look at TestHsJobBlock ? The first time I ran it, it timed out. The second time, it really slowed down my computer after a while (eating all my RAM I suppose), and I had to ctrl-c it, which wasn't easy.
        Hide
        haibochen Haibo Chen added a comment -

        I ran it on my local laptop with mvn test. It was fine. The is the output "Final Memory: 118M/821M". Running TestAppPage, I got "117M/806M", no real difference.

        Show
        haibochen Haibo Chen added a comment - I ran it on my local laptop with mvn test. It was fine. The is the output "Final Memory: 118M/821M". Running TestAppPage, I got "117M/806M", no real difference.
        Hide
        haibochen Haibo Chen added a comment - - edited

        Turns out some of the existing test methods in TestHistoryFileManager set the DFS cluster to safe mode and never let the cluster leave safe mode. Therefore, HistoryFileManager is stuck in creating a folder in HDFS, eventually causing the timeout. This is not caused by this patch. Updating the patch to take DFS out of safe mode in @After method.

        Show
        haibochen Haibo Chen added a comment - - edited Turns out some of the existing test methods in TestHistoryFileManager set the DFS cluster to safe mode and never let the cluster leave safe mode. Therefore, HistoryFileManager is stuck in creating a folder in HDFS, eventually causing the timeout. This is not caused by this patch. Updating the patch to take DFS out of safe mode in @After method.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 34s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 53s Maven dependency ordering for branch
        +1 mvninstall 7m 15s trunk passed
        +1 compile 1m 36s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 1m 14s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 1m 54s trunk passed
        +1 javadoc 0m 56s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 57s the patch passed
        +1 compile 1m 33s the patch passed
        +1 javac 1m 33s the patch passed
        -1 checkstyle 0m 24s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79)
        +1 mvnsite 1m 8s the patch passed
        +1 mvneclipse 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 2m 8s the patch passed
        -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198)
        +1 unit 2m 0s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed.
        +1 unit 6m 40s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        33m 44s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818069/mapreduce6652.009.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 38bb423cfd5c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e549a9a
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 34s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 53s Maven dependency ordering for branch +1 mvninstall 7m 15s trunk passed +1 compile 1m 36s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 14s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 0m 56s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 1m 33s the patch passed +1 javac 1m 33s the patch passed -1 checkstyle 0m 24s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 8s the patch passed -1 javadoc 0m 11s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198) +1 unit 2m 0s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed. +1 unit 6m 40s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 33m 44s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818069/mapreduce6652.009.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 38bb423cfd5c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e549a9a Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6614/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 52s Maven dependency ordering for branch
        +1 mvninstall 7m 12s trunk passed
        +1 compile 1m 38s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 1m 17s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        +1 findbugs 2m 17s trunk passed
        +1 javadoc 1m 1s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 6s the patch passed
        +1 compile 1m 44s the patch passed
        +1 javac 1m 44s the patch passed
        -1 checkstyle 0m 28s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79)
        +1 mvnsite 1m 17s the patch passed
        +1 mvneclipse 0m 35s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 33s the patch passed
        -1 javadoc 0m 12s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198)
        +1 unit 2m 14s hadoop-mapreduce-client-core in the patch passed.
        +1 unit 0m 43s hadoop-mapreduce-client-common in the patch passed.
        +1 unit 6m 57s hadoop-mapreduce-client-hs in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        35m 41s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818069/mapreduce6652.009.patch
        JIRA Issue MAPREDUCE-6652
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux ae2dac2c834e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e549a9a
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
        javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 52s Maven dependency ordering for branch +1 mvninstall 7m 12s trunk passed +1 compile 1m 38s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 17s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 2m 17s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 6s the patch passed +1 compile 1m 44s the patch passed +1 javac 1m 44s the patch passed -1 checkstyle 0m 28s hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) +1 mvnsite 1m 17s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 33s the patch passed -1 javadoc 0m 12s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs generated 5 new + 198 unchanged - 0 fixed = 203 total (was 198) +1 unit 2m 14s hadoop-mapreduce-client-core in the patch passed. +1 unit 0m 43s hadoop-mapreduce-client-common in the patch passed. +1 unit 6m 57s hadoop-mapreduce-client-hs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 35m 41s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818069/mapreduce6652.009.patch JIRA Issue MAPREDUCE-6652 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ae2dac2c834e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e549a9a Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6615/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rkanter Robert Kanter added a comment -

        +1 on the 009 patch. Will commit tomorrow afternoon.

        Show
        rkanter Robert Kanter added a comment - +1 on the 009 patch. Will commit tomorrow afternoon.
        Hide
        rchiang Ray Chiang added a comment -

        Looks good to me. +1 for the 009 patch as well.

        Show
        rchiang Ray Chiang added a comment - Looks good to me. +1 for the 009 patch as well.
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Haibo Chen. And Ray Chiang for reviewing too.

        Committed to trunk and branch-2!

        Show
        rkanter Robert Kanter added a comment - Thanks Haibo Chen . And Ray Chiang for reviewing too. Committed to trunk and branch-2!

          People

          • Assignee:
            haibochen Haibo Chen
            Reporter:
            haibochen Haibo Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development