Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3365

SequenceFile.Sorter.MergeQueue.next does an unnecessary copy of the key

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.17.0
    • 0.18.0
    • io
    • None
    • Reviewed

    Description

      SequenceFile.Sorter.MergeQueue.next does an unnecessary copy of the rawKey. We should remove that copy, and move code which adjusts the heap from next() to getKey(), which should then just return ms.getKey(), thereby eliminating the extra copy.

      Attachments

        1. 3365.patch
          3 kB
          Devaraj Das
        2. 3365.patch
          3 kB
          Devaraj Das
        3. 3365.patch
          3 kB
          Devaraj Das

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-3366 Shuffle/Merge improvements

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ddas Devaraj Das
              acmurthy Arun Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: