Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-872

map output sorter doesn't compress the outputs before the sort

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 0.10.0
    • None
    • None
    • None

    Description

      The map outputs are not compressed for the in memory sort, even if mapred.compress.map.output is set. This can significantly increase the in memory size of the buffered outputs.

      Attachments

        Issue Links

          is part of

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-2919 Create fewer copies of buffer data during sort/spill

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              ddas Devaraj Das
              omalley Owen O'Malley
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: