Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3297

The way in which ReduceTask/TaskTracker gets completion events during shuffle can be improved

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Reviewed

    Description

      Certain things like poll frequency, number of events fetched in one go, etc. can probably be improved to improve the shuffle performance. This would affect the task->tasktracker and the tasktracker->jobtracker shuffle related RPCs.

      Attachments

        1. 3297.patch
          10 kB
          Devaraj Das
        2. 3297.patch
          10 kB
          Devaraj Das

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-3366 Shuffle/Merge improvements

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3327 Shuffling fetchers waited too long between map output fetch re-tries

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ddas Devaraj Das
              ddas Devaraj Das
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: