Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Something tasks failed because of issues on NMs. For example, bad disk/network could cause reducer fetching failure and mappers need to be re-scheduled.

      It will be very helpful to identify such issues if we could print host of failed tasks, which we can simply grep MR AM's log to see what happened.

      1. MAPREDUCE-6574-v1.patch
        1 kB
        Mohammad Shahid Khan
      2. MAPREDUCE-6574-v2.patch
        1 kB
        Mohammad Shahid Khan
      3. MAPREDUCE-6574-v3.patch
        2 kB
        Mohammad Shahid Khan
      4. MAPREDUCE-6574-v4.patch
        2 kB
        Mohammad Shahid Khan

        Activity

        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        Hi Wangda Tan, hope the current patch could address the issue.

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - Hi Wangda Tan , hope the current patch could address the issue.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Mohammad Shahid Khan,

        Thanks for working on this, some comments:

        • I think we may not need to print nodeId for every transitions, DiagnosticInformationUpdater added such info to many different transitions.
        • And DiagnosticInformationUpdater isn't used by transitions which is to FAILED status as well. For example, the one from succeeded -> failed: TooManyFetchFailureTransition
        • Could you print a separate message when status changed from any state to FAILED state with the id and event type? Such as: Attempt X transitioned from state A to B, event type is E and nodeId=x. You may need to check if nodeId == null, IIRC, some task attempts could be failed before allocated to any node.
        Show
        leftnoteasy Wangda Tan added a comment - Hi Mohammad Shahid Khan , Thanks for working on this, some comments: I think we may not need to print nodeId for every transitions, DiagnosticInformationUpdater added such info to many different transitions. And DiagnosticInformationUpdater isn't used by transitions which is to FAILED status as well. For example, the one from succeeded -> failed: TooManyFetchFailureTransition Could you print a separate message when status changed from any state to FAILED state with the id and event type? Such as: Attempt X transitioned from state A to B, event type is E and nodeId=x . You may need to check if nodeId == null, IIRC, some task attempts could be failed before allocated to any node.
        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        Hi Wangda Tan please review the current patch

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - Hi Wangda Tan please review the current patch
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 20s trunk passed
        +1 compile 0m 22s trunk passed with JDK v1.8.0_66
        +1 compile 0m 26s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 30s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 47s trunk passed
        +1 javadoc 0m 15s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 19s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 28s the patch passed
        +1 compile 0m 19s the patch passed with JDK v1.8.0_66
        -1 javac 2m 8s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10).
        +1 javac 0m 19s the patch passed
        +1 compile 0m 25s the patch passed with JDK v1.7.0_91
        -1 javac 2m 33s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11).
        +1 javac 0m 25s the patch passed
        -1 checkstyle 0m 11s Patch generated 4 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 246).
        +1 mvnsite 0m 30s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 54s the patch passed
        +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 20s the patch passed with JDK v1.7.0_91
        -1 unit 9m 23s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
        -1 unit 10m 9s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91.
        -1 asflicense 0m 28s Patch generated 1 ASF License warnings.
        36m 24s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
        JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779291/MAPREDUCE-6574-v2.patch
        JIRA Issue MAPREDUCE-6574
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 561d0931f755 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bb5df27
        findbugs v3.0.0
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/testReport/
        asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 20s trunk passed +1 compile 0m 22s trunk passed with JDK v1.8.0_66 +1 compile 0m 26s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 47s trunk passed +1 javadoc 0m 15s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 19s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 28s the patch passed +1 compile 0m 19s the patch passed with JDK v1.8.0_66 -1 javac 2m 8s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10). +1 javac 0m 19s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_91 -1 javac 2m 33s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11). +1 javac 0m 25s the patch passed -1 checkstyle 0m 11s Patch generated 4 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 246). +1 mvnsite 0m 30s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 20s the patch passed with JDK v1.7.0_91 -1 unit 9m 23s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. -1 unit 10m 9s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 28s Patch generated 1 ASF License warnings. 36m 24s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779291/MAPREDUCE-6574-v2.patch JIRA Issue MAPREDUCE-6574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 561d0931f755 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bb5df27 findbugs v3.0.0 javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6229/console This message was automatically generated.
        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        Checkstyle issue related to current patch, will correct the same.
        javac and ut failure are unrelated.

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - Checkstyle issue related to current patch, will correct the same. javac and ut failure are unrelated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 4s trunk passed
        +1 compile 0m 21s trunk passed with JDK v1.8.0_66
        +1 compile 0m 24s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 11s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 45s trunk passed
        +1 javadoc 0m 16s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 18s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 26s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.8.0_66
        -1 javac 2m 8s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10).
        +1 javac 0m 21s the patch passed
        +1 compile 0m 24s the patch passed with JDK v1.7.0_91
        -1 javac 2m 32s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11).
        +1 javac 0m 24s the patch passed
        -1 checkstyle 0m 12s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 243).
        +1 mvnsite 0m 29s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 53s the patch passed
        +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91
        -1 unit 9m 25s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
        -1 unit 10m 7s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        35m 36s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
        JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779470/MAPREDUCE-6574-v3.patch
        JIRA Issue MAPREDUCE-6574
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ea24de51e44c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6553784
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 4s trunk passed +1 compile 0m 21s trunk passed with JDK v1.8.0_66 +1 compile 0m 24s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 11s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 45s trunk passed +1 javadoc 0m 16s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 18s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 26s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_66 -1 javac 2m 8s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10). +1 javac 0m 21s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_91 -1 javac 2m 32s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11). +1 javac 0m 24s the patch passed -1 checkstyle 0m 12s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 243). +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 53s the patch passed +1 javadoc 0m 17s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91 -1 unit 9m 25s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. -1 unit 10m 7s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 35m 36s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779470/MAPREDUCE-6574-v3.patch JIRA Issue MAPREDUCE-6574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ea24de51e44c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6553784 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6233/console This message was automatically generated.
        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        fixed white space error

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - fixed white space error
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 8m 19s trunk passed
        +1 compile 0m 23s trunk passed with JDK v1.8.0_66
        +1 compile 0m 23s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 46s trunk passed
        +1 javadoc 0m 16s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 18s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 25s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.8.0_66
        -1 javac 2m 7s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10).
        +1 javac 0m 21s the patch passed
        +1 compile 0m 24s the patch passed with JDK v1.7.0_91
        -1 javac 2m 31s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11).
        +1 javac 0m 24s the patch passed
        -1 checkstyle 0m 11s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 243).
        +1 mvnsite 0m 29s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 54s the patch passed
        +1 javadoc 0m 16s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91
        -1 unit 9m 29s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
        -1 unit 10m 2s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        35m 53s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestRecovery
          hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
        JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779529/MAPREDUCE-6574-v4.patch
        JIRA Issue MAPREDUCE-6574
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 46d8a793c80b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8cfd672
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 19s trunk passed +1 compile 0m 23s trunk passed with JDK v1.8.0_66 +1 compile 0m 23s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 46s trunk passed +1 javadoc 0m 16s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 18s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 25s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_66 -1 javac 2m 7s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 10, now 10). +1 javac 0m 21s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_91 -1 javac 2m 31s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 11, now 11). +1 javac 0m 24s the patch passed -1 checkstyle 0m 11s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app (total was 246, now 243). +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 54s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91 -1 unit 9m 29s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. -1 unit 10m 2s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 35m 53s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestRecovery   hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler JDK v1.7.0_91 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779529/MAPREDUCE-6574-v4.patch JIRA Issue MAPREDUCE-6574 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 46d8a793c80b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8cfd672 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt javac hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6238/console This message was automatically generated.
        Hide
        mohdshahidkhan Mohammad Shahid Khan added a comment -

        ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java:1: File length is 2,285 lines (max allowed is 2,000)
        I think its unrelated, as the number of lines were more than 2000 lines from long time.

        UT Failure unrelated.
        For the current patch only log level change ie wy have not added any UT.
        Hi Wangda Tan , please review once.

        Show
        mohdshahidkhan Mohammad Shahid Khan added a comment - ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java:1: File length is 2,285 lines (max allowed is 2,000) I think its unrelated, as the number of lines were more than 2000 lines from long time. UT Failure unrelated. For the current patch only log level change ie wy have not added any UT. Hi Wangda Tan , please review once.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to trunk/branch-2/branch-2.8, thanks Mohammad Shahid Khan!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2/branch-2.8, thanks Mohammad Shahid Khan !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9029 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9029/)
        MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad (wangda: rev 32b3dc11eba8f2d8f22ca502bf30d6aa8839460e)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9029 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9029/ ) MAPREDUCE-6574 . MR AM should print host of failed tasks. (Mohammad (wangda: rev 32b3dc11eba8f2d8f22ca502bf30d6aa8839460e) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java

          People

          • Assignee:
            mohdshahidkhan Mohammad Shahid Khan
            Reporter:
            leftnoteasy Wangda Tan
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development