Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15179

Cell/DBB end-to-end on the write-path

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • regionserver
    • None

    Description

      Umbrella jira to make the HBase write path off heap E2E. We have to make sure we have Cells flowing in entire write path. Starting from request received in RPC layer, till the Cells get flushed out as HFiles, we have to keep the Cell data off heap.

      https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-17267 Add OffheapMemoryTuner

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-15879 Introduce Key interface

          • Major - Major loss of function.
          • Closed
          1.
          		 Reduce garbage created while reading Cells from Codec Decoder Sub-task Closed Anoop Sam John
          2.
          		 Allow Dictionary to work with BytebufferedCells Sub-task Closed ramkrishna.s.vasudevan
          3.
          		 PrefixTree write path should work with BytebufferedCells (during compaction) Sub-task Closed Unassigned
          4.
          		 Reduce garbage while setting response Sub-task Closed ramkrishna.s.vasudevan
          5.
          		 Reduce garbage created by path.toString() during Checksum verification Sub-task Closed ramkrishna.s.vasudevan
          6.
          		 Try to estimate the cell count for adding into WALEdit Sub-task Closed ramkrishna.s.vasudevan
          7.
          		 Do not find the replication scope for every WAL#append() Sub-task Closed ramkrishna.s.vasudevan
          8.
          		 Optimization in cloning cells into MSLAB Sub-task Closed Anoop Sam John
          9.
          		 Create DBB backed MSLAB pool Sub-task Closed Anoop Sam John
          10.
          		 Change the flush related heuristics to work with offheap size configured Sub-task Closed ramkrishna.s.vasudevan
          11.
          		 Use Offheap ByteBuffers from BufferPool to read RPC requests. Sub-task Closed Anoop Sam John
          12.
          		 PB related changes to work with offheap Sub-task Closed Anoop Sam John
          13.
          		 Introduce Cell extension for server side. Sub-task Closed Anoop Sam John
          14.
          		 When Cells are not copied to MSLAB, deep clone it while adding to Memstore Sub-task Closed Anoop Sam John
          15.
          		 When TagRewriteCell are not copied to MSLAB, deep clone it while adding to Memstore Sub-task Closed Anoop Sam John
          16.
          		 Track memstore data size and heap overhead separately Sub-task Closed Anoop Sam John
          17.
          		 Handle Offheap cells in CompressedKvEncoder Sub-task Closed ramkrishna.s.vasudevan
          18.
          		 Enable Aliasing for CodedInputStream created by ByteInputByteString#newCodedInput Sub-task Closed Anoop Sam John
          19.
          		 Avoid unconditional call to getXXXArray() in write path Sub-task Closed Anoop Sam John
          20.
          		 Remove Cell variants with ShareableMemory Sub-task Closed Anoop Sam John
          21.
          		 Handle ByteBufferCell while making TagRewriteCell Sub-task Closed Anoop Sam John
          22.
          		 Make COS#newInstance(ByteOutput, int) as public API. Sub-task Closed Unassigned
          23.
          		 Add OffheapMemoryTuner Sub-task Closed Unassigned
          24.
          		 Treat Cell data size under global memstore heap size only when that Cell can not be copied to MSLAB Sub-task Closed Anoop Sam John
          25.
          		 Add equals/hashcode for OffheapKeyValue Sub-task Closed ramkrishna.s.vasudevan
          26.
          		 Add non cached version of OffheapKV for write path Sub-task Closed ramkrishna.s.vasudevan
          27.
          		 Always create ByteBufferCells after copying to MSLAB Sub-task Closed Anoop Sam John
          28.
          		 OffheapKeyValue#heapSize() implementation is wrong Sub-task Closed Anoop Sam John
          29.
          		 Add details of off heap memstore into book. Sub-task Closed Zheng Hu

          Activity

            People

              anoop.hbase Anoop Sam John
              anoop.hbase Anoop Sam John
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: