Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I have seen a reducer in RUNNING state, when it was waiting for the commitResponse.
      Task logs show :
      2009-02-12 08:35:36,414 INFO org.apache.hadoop.mapred.TaskRunner: Task:attempt_200902120746_0297_r_000033_0 is done. And is in the process of commiting
      and no logs after that.
      TT logs say:
      2009-02-12 08:35:36,417 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_200902120746_0297_r_000033_0 is in COMMIT_PENDING
      2009-02-12 08:35:36,417 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200902120746_0297_r_000033_0 0.33333334% reduce > sort

      This looks like, the task progress went from COMMIT_PENDING to RUNNING again.

      1. patch-5233-2.txt
        6 kB
        Amareshwari Sriramadasu
      2. patch-5233-1.txt
        3 kB
        Amareshwari Sriramadasu
      3. patch-5233-0.19.txt
        6 kB
        Amareshwari Sriramadasu
      4. patch-5233.txt
        3 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Amareshwari Sriramadasu added a comment -

          I would like to submit one patch for HADOOP-5233, hADOOP-5234 and HADOOP-5235, since they involve one line changes. This would lessen testing overhead.

          Show
          Amareshwari Sriramadasu added a comment - I would like to submit one patch for HADOOP-5233 , hADOOP-5234 and HADOOP-5235 , since they involve one line changes. This would lessen testing overhead.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch incorporating fixing this and HADOOP-5234, HADOOP-5235.
          Also fixed a corner case in JIP with task-cleanup tasks.

          Show
          Amareshwari Sriramadasu added a comment - Patch incorporating fixing this and HADOOP-5234 , HADOOP-5235 . Also fixed a corner case in JIP with task-cleanup tasks.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12400264/patch-5233.txt
          against trunk revision 744894.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12400264/patch-5233.txt against trunk revision 744894. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3869/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Cancelling patch..
          replacing localJobConf with defaultJobConf in TaskTracker.TaskInProgress.cleanup doesn't work. will add not null check in the next patch.

          Show
          Amareshwari Sriramadasu added a comment - Cancelling patch.. replacing localJobConf with defaultJobConf in TaskTracker.TaskInProgress.cleanup doesn't work. will add not null check in the next patch.
          Hide
          Amareshwari Sriramadasu added a comment -

          modified patch

          Show
          Amareshwari Sriramadasu added a comment - modified patch
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12400318/patch-5233-1.txt
          against trunk revision 744894.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12400318/patch-5233-1.txt against trunk revision 744894. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3872/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Attaching patch with proper fix. Earlier patch didnt fix the problem correctly.
          I verified the patch by running 5 gridmix runs back to back.
          Patch applies to both trunk and branch 0.20

          Show
          Amareshwari Sriramadasu added a comment - Attaching patch with proper fix. Earlier patch didnt fix the problem correctly. I verified the patch by running 5 gridmix runs back to back. Patch applies to both trunk and branch 0.20
          Hide
          Amareshwari Sriramadasu added a comment -

          testpatch result :

               [exec]
               [exec] -1 overall.
               [exec]
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec]
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no tests are needed for this patch.
               [exec]
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec]
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec]
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec]
               [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
               [exec]
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec]
          

          All core tests passed on my machine.

          Show
          Amareshwari Sriramadasu added a comment - testpatch result : [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] All core tests passed on my machine.
          Hide
          Devaraj Das added a comment -

          I just committed this to the 0.20 branch and trunk. Thanks, Amareshwari! (After 0.19.1 is released for which voting is going on, we should commit this to 0.19 branch as well)

          Show
          Devaraj Das added a comment - I just committed this to the 0.20 branch and trunk. Thanks, Amareshwari! (After 0.19.1 is released for which voting is going on, we should commit this to 0.19 branch as well)
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #763 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/763/ )
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for branch 0.19

          Show
          Amareshwari Sriramadasu added a comment - Patch for branch 0.19
          Hide
          Devaraj Das added a comment -

          I committed this to the 0.19 branch.

          Show
          Devaraj Das added a comment - I committed this to the 0.19 branch.

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development