Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11425 Cell/DBB end-to-end on the read-path
  3. HBASE-12295

Prevent block eviction under us if reads are in progress from the BBs

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: regionserver, Scanners
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      We try to delay the eviction of the block till the cellblocks are formed at the Rpc layer. A simple reference counting mechanism is introduced when ever a block is accessed from the Bucket cache. Once a scanner completes using a block the reference count is decremented. The eviction of the block happens only when the reference count of that block is 0.
      We also introduce a concept of ShareableMemory based on the type of blocks we create from the Block cache. The blocks from the ByteBufferIOEngine directly refer to the buckets in offheap and such blocks are marked SHARED memory type. The blocks from LRU, HDFS and file mode of Bucket cache are all marked EXCLUSIVE because these blocks have their own exclusive memory.
      For the CP case, any cell coming out of SHARED memory block is copied before returning the results, because CPs can use the results as its state so that eviction cannot corrupt the results.
      Show
      We try to delay the eviction of the block till the cellblocks are formed at the Rpc layer. A simple reference counting mechanism is introduced when ever a block is accessed from the Bucket cache. Once a scanner completes using a block the reference count is decremented. The eviction of the block happens only when the reference count of that block is 0. We also introduce a concept of ShareableMemory based on the type of blocks we create from the Block cache. The blocks from the ByteBufferIOEngine directly refer to the buckets in offheap and such blocks are marked SHARED memory type. The blocks from LRU, HDFS and file mode of Bucket cache are all marked EXCLUSIVE because these blocks have their own exclusive memory. For the CP case, any cell coming out of SHARED memory block is copied before returning the results, because CPs can use the results as its state so that eviction cannot corrupt the results.

      Description

      While we try to serve the reads from the BBs directly from the block cache, we need to ensure that the blocks does not get evicted under us while reading. This JIRA is to discuss and implement a strategy for the same.

        Attachments

        1. HBASE-12295_trunk.patch
          64 kB
          ramkrishna.s.vasudevan
        2. HBASE-12295.pdf
          81 kB
          ramkrishna.s.vasudevan
        3. HBASE-12295_1.pdf
          82 kB
          ramkrishna.s.vasudevan
        4. HBASE-12295_1.patch
          228 kB
          ramkrishna.s.vasudevan
        5. HBASE-12295_2.patch
          232 kB
          ramkrishna.s.vasudevan
        6. HBASE-12295_4.patch
          213 kB
          ramkrishna.s.vasudevan
        7. HBASE-12295_4.pdf
          86 kB
          ramkrishna.s.vasudevan
        8. HBASE-12295_5.pdf
          83 kB
          ramkrishna.s.vasudevan
        9. HBASE-12295_9.patch
          181 kB
          ramkrishna.s.vasudevan
        10. HBASE-12295_10.patch
          171 kB
          ramkrishna.s.vasudevan
        11. HBASE-12295_12.patch
          174 kB
          ramkrishna.s.vasudevan
        12. HBASE-12295_14.patch
          176 kB
          ramkrishna.s.vasudevan
        13. HBASE-12295_15.patch
          181 kB
          ramkrishna.s.vasudevan
        14. HBASE-12295_16.patch
          180 kB
          ramkrishna.s.vasudevan
        15. HBASE-12295_16.patch
          180 kB
          ramkrishna.s.vasudevan
        16. HBASE-12295_17.patch
          181 kB
          ramkrishna.s.vasudevan
        17. HBASE-12295_18.patch
          180 kB
          ramkrishna.s.vasudevan
        18. HBASE-12295_21.patch
          181 kB
          ramkrishna.s.vasudevan
        19. HBASE-12295_final.patch
          181 kB
          ramkrishna.s.vasudevan
        20. HBASE-12295_addendum.patch
          4 kB
          ramkrishna.s.vasudevan

          Issue Links

            Activity

              People

              • Assignee:
                ram_krish ramkrishna.s.vasudevan
                Reporter:
                ram_krish ramkrishna.s.vasudevan
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: