Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Add read and connection timeout to prevent job client hangs

      jobclient can block indefinitely during task log pull if read or connect fails

      at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049)

      • locked <0xeed0f0f0> (a sun.net.www.protocol.http.HttpURLConnection)
        at org.apache.hadoop.mapred.JobClient.getTaskLogs(JobClient.java:1396)
      1. mapred-1868-1.patch
        1 kB
        Krishna Ramachandran
      2. mapred-1868.patch
        1 kB
        Krishna Ramachandran
      3. mapred-1868.20S.patch
        2 kB
        Krishna Ramachandran

        Activity

        Krishna Ramachandran created issue -
        Hide
        Krishna Ramachandran added a comment -

        This proposed fix allows job client connections to timeout during a tasklog pull. Will prevent client calls from hanging because of a bad connection.

        Connection timeout for read/connect set to 60 secs by default.

        Can be configured using key

        "mapreduce.client.tasklog.timeout"

        Patch includes changes for an earlier release
        and for trunk

        Show
        Krishna Ramachandran added a comment - This proposed fix allows job client connections to timeout during a tasklog pull. Will prevent client calls from hanging because of a bad connection. Connection timeout for read/connect set to 60 secs by default. Can be configured using key "mapreduce.client.tasklog.timeout" Patch includes changes for an earlier release and for trunk
        Krishna Ramachandran made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Krishna Ramachandran made changes -
        Attachment mapred-1868.20S.patch [ 12447565 ]
        Hide
        Krishna Ramachandran added a comment -

        proposed fix for trunk

        Show
        Krishna Ramachandran added a comment - proposed fix for trunk
        Krishna Ramachandran made changes -
        Attachment mapred-1868.patch [ 12447566 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12447566/mapred-1868.patch
        against trunk revision 956335.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/255/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12447566/mapred-1868.patch against trunk revision 956335. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/255/console This message is automatically generated.
        Krishna Ramachandran made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Krishna Ramachandran added a comment -

        Revised patch.

        Show
        Krishna Ramachandran added a comment - Revised patch.
        Krishna Ramachandran made changes -
        Attachment mapred-1868-1.patch [ 12447781 ]
        Hide
        Krishna Ramachandran added a comment -

        No automated unit test included as it will require manual steps to test this change

        Show
        Krishna Ramachandran added a comment - No automated unit test included as it will require manual steps to test this change
        Krishna Ramachandran made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks Krishna!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks Krishna!
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
        Konstantin Shvachko made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Krishna Ramachandran
            Reporter:
            Krishna Ramachandran
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development