Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4699

Change TaskTracker.MapOutputServlet to send only the IFile segment, validate checksum in Reduce

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Instead of validating the checksum of the IFile segment in MapOutputServlet, validation may be left to the reduce. While failures may not be detected until late in the reduce, the throughput and CPU improvements should make up for it in the average case.

        Attachments

        1. 4699-0.patch
          6 kB
          Chris Douglas

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                chris.douglas Chris Douglas
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: