XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: regionserver, Scanners
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      The read path from the Hfileblock is now backed by a new data structure represented by an abstract class ByteBuff. The name is similar to nio ByteBuffer.
      The new ByteBuff class has various APIs to read and write primitives to the underlying implementation similar to how nio ByteBuffer provides.
      We have 2 implementations of the ByteBuff now - a single ByteBuffer backed implementation (SingleByteBuff) and a multiple ByteBuffer backed implementation (MultiByteBuff).
      The single byte buffer (SingleByteBuff) backed implementation will be used for the HFileBlocks that come from HDFS and from LRU cachce.
      The blocks that gets created out of BucketCache will be of type MultiByteBuff and the read operations are performed over this data structure which intelligently moves across different Byte buffers that it is formed of.
      Show
      The read path from the Hfileblock is now backed by a new data structure represented by an abstract class ByteBuff. The name is similar to nio ByteBuffer. The new ByteBuff class has various APIs to read and write primitives to the underlying implementation similar to how nio ByteBuffer provides. We have 2 implementations of the ByteBuff now - a single ByteBuffer backed implementation (SingleByteBuff) and a multiple ByteBuffer backed implementation (MultiByteBuff). The single byte buffer (SingleByteBuff) backed implementation will be used for the HFileBlocks that come from HDFS and from LRU cachce. The blocks that gets created out of BucketCache will be of type MultiByteBuff and the read operations are performed over this data structure which intelligently moves across different Byte buffers that it is formed of.

      Description

      In L2 cache (offheap) an HFile block might have been cached into multiple chunks of buffers. If HFileBlock need single BB, we will end up in recreation of bigger BB and copying. Instead we can make HFileBlock to serve data from an array of BBs.

        Attachments

        1. HBASE-12213_jmh.zip
          22 kB
          ramkrishna.s.vasudevan
        2. HBASE-12213_final.patch
          262 kB
          ramkrishna.s.vasudevan
        3. HBASE-12213_9_withBBI.patch
          242 kB
          ramkrishna.s.vasudevan
        4. HBASE-12213_8_withBBI.patch
          198 kB
          ramkrishna.s.vasudevan
        5. HBASE-12213_4.patch
          130 kB
          ramkrishna.s.vasudevan
        6. HBASE-12213_2.patch
          125 kB
          ramkrishna.s.vasudevan
        7. HBASE-12213_14_withBBI.patch
          262 kB
          ramkrishna.s.vasudevan
        8. HBASE-12213_13_withBBI.patch
          263 kB
          ramkrishna.s.vasudevan
        9. HBASE-12213_12_withBBI.patch
          261 kB
          ramkrishna.s.vasudevan
        10. HBASE-12213_12_withBBI.patch
          262 kB
          ramkrishna.s.vasudevan
        11. HBASE-12213_11_withBBI.patch
          260 kB
          ramkrishna.s.vasudevan
        12. HBASE-12213_10_withBBI.patch
          258 kB
          ramkrishna.s.vasudevan
        13. HBASE-12213_1.patch
          134 kB
          ramkrishna.s.vasudevan

          Issue Links

            Activity

              People

              • Assignee:
                ram_krish ramkrishna.s.vasudevan
                Reporter:
                anoop.hbase Anoop Sam John
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: