Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4699

Change TaskTracker.MapOutputServlet to send only the IFile segment, validate checksum in Reduce

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.20.0
    • None
    • None
    • Reviewed

    Description

      Instead of validating the checksum of the IFile segment in MapOutputServlet, validation may be left to the reduce. While failures may not be detected until late in the reduce, the throughput and CPU improvements should make up for it in the average case.

      Attachments

        1. 4699-0.patch
          6 kB
          Christopher Douglas

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-258 Update MapOutputServlet to use NIO channels

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-1650 Upgrade Jetty to 6.x

          • Major - Major loss of function.
          • Closed

          Activity

            People

              cdouglas Christopher Douglas
              cdouglas Christopher Douglas
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: