Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3020

Node link in reduce task attempt page is not working [Job History Page]

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.0
    • Component/s: jobhistoryserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      RM UI -> Applications -> Application(Job History) -> Reduce Tasks -> Task ID -> Node link is not working
      hostname for ReduceAttemptFinishedEvent is coming wrong when loading from history file.

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #858 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/858/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #858 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/858/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #46 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/46/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. (Chackaravarthy via vinodkv)
        svn merge -c r1181793 --ignore-ancestry ../../trunk/

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181794
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #46 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/46/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. (Chackaravarthy via vinodkv) svn merge -c r1181793 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181794 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/37/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. (Chackaravarthy via vinodkv)
        svn merge -c r1181793 --ignore-ancestry ../../trunk/

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181794
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #37 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/37/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. (Chackaravarthy via vinodkv) svn merge -c r1181793 --ignore-ancestry ../../trunk/ vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181794 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #828 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/828/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #828 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/828/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        chackaravarthy added a comment -

        Hi Vinod, Thanks for your wishes

        Show
        chackaravarthy added a comment - Hi Vinod, Thanks for your wishes
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1075/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1075/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1133 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1133/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1133 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1133/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Vinod Kumar Vavilapalli added a comment -

        I just committed this to trunk and branch-0.23. Thanks Chackaravarthy!

        Congratulations on your first patch to be committed!

        Show
        Vinod Kumar Vavilapalli added a comment - I just committed this to trunk and branch-0.23. Thanks Chackaravarthy! Congratulations on your first patch to be committed!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1055 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1055/)
        MAPREDUCE-3020. Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1055 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1055/ ) MAPREDUCE-3020 . Fixed TaskAttemptImpl to log the correct node-address for a finished Reduce task. Contributed by Chackaravarthy. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181793 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Little good patch. +1. Verified the bug without the patch and that the patch works on my single node setup.

        Good job on finding out the root cause BTW smile

        Show
        Vinod Kumar Vavilapalli added a comment - Little good patch. +1. Verified the bug without the patch and that the patch works on my single node setup. Good job on finding out the root cause BTW smile
        Hide
        chackaravarthy added a comment -

        This is trivial change. In Job History Page (Reduce Tasks -> Task ID -> Node link)
        Earlier it was redirecting to wrong page (http://containerMgrAddress). It should be http://hostname

        Show
        chackaravarthy added a comment - This is trivial change. In Job History Page (Reduce Tasks -> Task ID -> Node link) Earlier it was redirecting to wrong page ( http://containerMgrAddress ). It should be http://hostname
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496441/MAPREDUCE-3020.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/856//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/856//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496441/MAPREDUCE-3020.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/856//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/856//console This message is automatically generated.
        Hide
        chackaravarthy added a comment -

        MAPREDUCE-3009 issue exists for this reduce task page also. Already submitted one patch for that issue.

        Show
        chackaravarthy added a comment - MAPREDUCE-3009 issue exists for this reduce task page also. Already submitted one patch for that issue.
        Hide
        chackaravarthy added a comment -

        The issue was : "when creating ReduceAttemptFinishedEvent, instead of passing nodeHostName for hostname, containerMgrAddress was passed"..
        But creation of MapAttemptFinishedEvent is done in right way.Hence changed the code with respect to that.

        Show
        chackaravarthy added a comment - The issue was : "when creating ReduceAttemptFinishedEvent, instead of passing nodeHostName for hostname, containerMgrAddress was passed".. But creation of MapAttemptFinishedEvent is done in right way.Hence changed the code with respect to that.
        Hide
        chackaravarthy added a comment -

        Attached the patch.

        Show
        chackaravarthy added a comment - Attached the patch.

          People

          • Assignee:
            chackaravarthy
            Reporter:
            chackaravarthy
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development