Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4274

MapOutputBuffer should use native byte order for kvmeta

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: performance, task
    • Labels:
      None
    • Target Version/s:

      Description

      I don't have a benchmark to support this, but this should give a small CPU improvement on the map output buffer: currently, we create kvmeta as ByteBuffer.wrap(kvbuffer).asIntBuffer(). According to the javadocs, the resulting int buffer will inherit its byte order from the ByteBuffer it comes from, and the byte buffer defaults to BIG_ENDIAN. Thus, all of our int access to/from the buffer will require byte-swapping.

        Activity

        Todd Lipcon created issue -
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment mapreduce-4274.txt [ 12528357 ]
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Robert Joseph Evans made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 2.0.0 [ 12320354 ]
        Fix Version/s 3.0.0 [ 12320355 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Allen Wittenauer made changes -
        Fix Version/s 3.0.0 [ 12320355 ]
        Allen Wittenauer made changes -
        Fix Version/s 2.0.2-alpha [ 12322471 ]
        Fix Version/s 2.0.0-alpha [ 12320354 ]

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development