Hadoop Common
  1. Hadoop Common
  2. HADOOP-3837

hadop streaming does not use progress reporting to detect hung tasks

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.18.0, 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Changed streaming tasks to adhere to task timeout value specified in the job configuration.

      Description

      Hadoop streaming (StreamJob.java) sets mapred.task.timeout to 0. This effectively means that if tasks hang (either due to bad user code or machine related issue), these tasks never encounters a timeout. This causes the entire job to hang.

      I propose that hadoop streaming not set the timeout to 0. By default, the settings in hadoop-default.xml should be effective for streaming jobs.

      1. streamTimeout.patch
        0.7 kB
        dhruba borthakur
      2. streamTimeout.patch
        0.7 kB
        dhruba borthakur

        Activity

        Hide
        dhruba borthakur added a comment -

        Streaming code does not set the task-timeout value of 0.
        Patch that applies to trunk as well as 0.18 branch.

        Show
        dhruba borthakur added a comment - Streaming code does not set the task-timeout value of 0. Patch that applies to trunk as well as 0.18 branch.
        Hide
        dhruba borthakur added a comment -

        I would like to apply this patch to trunk as well as 0.18.

        Show
        dhruba borthakur added a comment - I would like to apply this patch to trunk as well as 0.18.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12387058/streamTimeout.patch
        against trunk revision 679930.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12387058/streamTimeout.patch against trunk revision 679930. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2962/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        +1 on the patch and +1 on including this in 0.18

        Show
        Devaraj Das added a comment - +1 on the patch and +1 on including this in 0.18
        Hide
        dhruba borthakur added a comment -

        I will commit this to trunk and 0.18 by the end of this week.

        Show
        dhruba borthakur added a comment - I will commit this to trunk and 0.18 by the end of this week.
        Hide
        Doug Cutting added a comment -

        > I will commit this to trunk and 0.18 by the end of this week.

        The week is almost up...

        Show
        Doug Cutting added a comment - > I will commit this to trunk and 0.18 by the end of this week. The week is almost up...
        Hide
        dhruba borthakur added a comment -

        I just committed this.

        Show
        dhruba borthakur added a comment - I just committed this.
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/ )

          People

          • Assignee:
            dhruba borthakur
            Reporter:
            dhruba borthakur
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development