Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13291 Lift the scan ceiling
  3. HBASE-13819

Make RPC layer CellBlock buffer a DirectByteBuffer

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: Scanners
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      For master branch(2.0 version), the BoundedByteBufferPool always create Direct (off heap) ByteBuffers and return that.
      For branch-1(1.3 version), byte default the buffers returned will be off heap. This can be changed to return on heap ByteBuffers by configuring 'hbase.ipc.server.reservoir.direct.buffer' to false.
      Show
      For master branch(2.0 version), the BoundedByteBufferPool always create Direct (off heap) ByteBuffers and return that. For branch-1(1.3 version), byte default the buffers returned will be off heap. This can be changed to return on heap ByteBuffers by configuring 'hbase.ipc.server.reservoir.direct.buffer' to false.

      Description

      In RPC layer, when we make a cellBlock to put as RPC payload, we will make an on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto certain number of buffers. This jira aims at testing possibility for making this buffers off heap ones. (DBB) The advantages
      1. Unsafe based writes to off heap is faster than that to on heap. Now we are not using unsafe based writes at all. Even if we add, DBB will be better
      2. When Cells are backed by off heap (HBASE-11425) off heap to off heap writes will be better
      3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer to the socket channel, it will create a temp DBB and copy data to there and only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can avoid this one more level of copying.

      Will do different perf testing with changed and report back.

        Attachments

        1. q.png
          74 kB
          deepankar
        2. HBASE-13819_branch-1.patch
          3 kB
          Michael Stack
        3. HBASE-13819_branch-1.patch
          3 kB
          Anoop Sam John
        4. HBASE-13819_branch-1.patch
          3 kB
          Anoop Sam John
        5. HBASE-13819.patch
          0.7 kB
          Anoop Sam John

          Issue Links

            Activity

              People

              • Assignee:
                anoop.hbase Anoop Sam John
                Reporter:
                anoop.hbase Anoop Sam John
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: