Hadoop Common
  1. Hadoop Common
  2. HADOOP-5276

Upon a lost tracker, the task's start time is reset to 0

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Upon lost tracker, the JIP (via JobTracker.lostTracker() -> JobInProgress.failedTask()) hand crafts task status (marking the attempt as KILLED) and updates the JobInProgress (via JobInProgress.updateTaskStatus()). This status contains the attempt start time as 0. One major impact would be that the listener will overwrite the last start time value with the new one and hence the start tiem will get garbled.

      1. HADOOP-5276-v1.1.patch
        3 kB
        Amar Kamat
      2. HADOOP-5276-v1.2.patch
        3 kB
        Amar Kamat
      3. HADOOP-5276-v1.3.patch
        2 kB
        Amar Kamat

        Issue Links

          Activity

          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #778 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/778/)
          . Moved the comment about 5276 to branch-0.20 section.

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #778 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/778/ ) . Moved the comment about 5276 to branch-0.20 section.
          Hide
          Devaraj Das added a comment -

          i committed this patch to branch-0.20 as well.

          Show
          Devaraj Das added a comment - i committed this patch to branch-0.20 as well.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #766 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/766/ )
          Devaraj Das made changes -
          Assignee Amar Kamat [ amar_kamat ]
          Resolution Fixed [ 1 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Hadoop Flags [Reviewed]
          Status Open [ 1 ] Resolved [ 5 ]
          Hide
          Devaraj Das added a comment -

          I just committed this. Thanks, Amar!

          Show
          Devaraj Das added a comment - I just committed this. Thanks, Amar!
          Amar Kamat made changes -
          Attachment HADOOP-5276-v1.3.patch [ 12400752 ]
          Hide
          Amar Kamat added a comment -

          Updates the patch to trunk. Test patch result :

          [exec] +1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec] 
               [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          
          Show
          Amar Kamat added a comment - Updates the patch to trunk. Test patch result : [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          Hide
          Devaraj Das added a comment -

          This patch doesn't apply anymore. Please submit a new patch.

          Show
          Devaraj Das added a comment - This patch doesn't apply anymore. Please submit a new patch.
          Hide
          Ravi Gummadi added a comment -

          I tested this patch with HADOOP-4191 and I see that correct values of start time and finish time are logged to history file with this patch.

          +1

          Show
          Ravi Gummadi added a comment - I tested this patch with HADOOP-4191 and I see that correct values of start time and finish time are logged to history file with this patch. +1
          Amar Kamat made changes -
          Attachment HADOOP-5276-v1.2.patch [ 12400407 ]
          Hide
          Amar Kamat added a comment -

          Attaching a new patch fixing a minor bug.

          Show
          Amar Kamat added a comment - Attaching a new patch fixing a minor bug.
          Amar Kamat made changes -
          Link This issue blocks HADOOP-4191 [ HADOOP-4191 ]
          Amar Kamat made changes -
          Field Original Value New Value
          Attachment HADOOP-5276-v1.1.patch [ 12400330 ]
          Hide
          Amar Kamat added a comment -

          Attaching a patch that fixes the bug. Modified the test case to check it.

          Show
          Amar Kamat added a comment - Attaching a patch that fixes the bug. Modified the test case to check it.
          Hide
          Amar Kamat added a comment -

          Also noticed that the finish time when a the task is killed upon restart is also 0.

          Show
          Amar Kamat added a comment - Also noticed that the finish time when a the task is killed upon restart is also 0.
          Amar Kamat created issue -

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development