Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15179

Cell/DBB end-to-end on the write-path

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Umbrella jira to make the HBase write path off heap E2E. We have to make sure we have Cells flowing in entire write path. Starting from request received in RPC layer, till the Cells get flushed out as HFiles, we have to keep the Cell data off heap.

      https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit

        Attachments

          Issue Links

          1.
          Reduce garbage created while reading Cells from Codec Decoder Sub-task Resolved Anoop Sam John
          2.
          Allow Dictionary to work with BytebufferedCells Sub-task Resolved ramkrishna.s.vasudevan
          3.
          PrefixTree write path should work with BytebufferedCells (during compaction) Sub-task Resolved Unassigned
          4.
          Reduce garbage while setting response Sub-task Resolved ramkrishna.s.vasudevan
          5.
          Reduce garbage created by path.toString() during Checksum verification Sub-task Resolved ramkrishna.s.vasudevan
          6.
          Try to estimate the cell count for adding into WALEdit Sub-task Resolved ramkrishna.s.vasudevan
          7.
          Do not find the replication scope for every WAL#append() Sub-task Resolved ramkrishna.s.vasudevan
          8.
          Optimization in cloning cells into MSLAB Sub-task Resolved Anoop Sam John
          9.
          Create DBB backed MSLAB pool Sub-task Resolved Anoop Sam John
          10.
          Change the flush related heuristics to work with offheap size configured Sub-task Resolved ramkrishna.s.vasudevan
          11.
          Use Offheap ByteBuffers from BufferPool to read RPC requests. Sub-task Resolved Anoop Sam John
          12.
          PB related changes to work with offheap Sub-task Resolved Anoop Sam John
          13.
          Introduce Cell extension for server side. Sub-task Resolved Anoop Sam John
          14.
          When Cells are not copied to MSLAB, deep clone it while adding to Memstore Sub-task Resolved Anoop Sam John
          15.
          When TagRewriteCell are not copied to MSLAB, deep clone it while adding to Memstore Sub-task Resolved Anoop Sam John
          16.
          Track memstore data size and heap overhead separately Sub-task Resolved Anoop Sam John
          17.
          Handle Offheap cells in CompressedKvEncoder Sub-task Resolved ramkrishna.s.vasudevan
          18.
          Enable Aliasing for CodedInputStream created by ByteInputByteString#newCodedInput Sub-task Resolved Anoop Sam John
          19.
          Avoid unconditional call to getXXXArray() in write path Sub-task Resolved Anoop Sam John
          20.
          Remove Cell variants with ShareableMemory Sub-task Resolved Anoop Sam John
          21.
          Handle ByteBufferCell while making TagRewriteCell Sub-task Resolved Anoop Sam John
          22.
          Make COS#newInstance(ByteOutput, int) as public API. Sub-task Resolved ramkrishna.s.vasudevan
          23.
          Add OffheapMemoryTuner Sub-task Resolved Unassigned
          24.
          Treat Cell data size under global memstore heap size only when that Cell can not be copied to MSLAB Sub-task Resolved Anoop Sam John
          25.
          Add equals/hashcode for OffheapKeyValue Sub-task Resolved ramkrishna.s.vasudevan
          26.
          Add non cached version of OffheapKV for write path Sub-task Resolved ramkrishna.s.vasudevan
          27.
          Always create ByteBufferCells after copying to MSLAB Sub-task Resolved Anoop Sam John
          28.
          OffheapKeyValue#heapSize() implementation is wrong Sub-task Resolved Anoop Sam John
          29.
          Add details of off heap memstore into book. Sub-task Resolved Zheng Hu

            Activity

              People

              • Assignee:
                anoop.hbase Anoop Sam John
                Reporter:
                anoop.hbase Anoop Sam John
              • Votes:
                0 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: