Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-353

Allow shuffle read and connection timeouts to be configurable

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Expert level config properties mapred.shuffle.connect.timeout and mapred.shuffle.read.timeout that are to be used at cluster level are added by this patch.

      Description

      It would be good for latency-sensitive applications to tune the shuffle read/connection timeouts... in fact this made a huge difference to terasort since we were seeing individual shuffles stuck for upwards of 60s and had to have a very small read timeout.

      1. MR-353.patch
        3 kB
        Ravi Gummadi
      2. MR-353.v1.patch
        3 kB
        Ravi Gummadi
      3. patch-353-ydist.txt
        3 kB
        Jothi Padmanabhan
      4. patch-353-ydist.txt
        3 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Ravi Gummadi
              Reporter:
              Arun C Murthy
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development