HBase
  1. HBase
  2. HBASE-10263

make LruBlockCache single/multi/in-memory ratio user-configurable and provide preemptive mode for in-memory type block

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.99.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      4 new configurations are introduced:
      hbase.lru.blockcache.single.percentage / hbase.lru.blockcache.multi.percentage / hbase.lru.blockcache.memory.percentage / hbase.lru.rs.inmemoryforcemode
      the first 3 enable user to config different single/multi/memory ratio in block cache; the last one determines if preemptive mode for memory block is enabled
      Show
      4 new configurations are introduced: hbase.lru.blockcache.single.percentage / hbase.lru.blockcache.multi.percentage / hbase.lru.blockcache.memory.percentage / hbase.lru.rs.inmemoryforcemode the first 3 enable user to config different single/multi/memory ratio in block cache; the last one determines if preemptive mode for memory block is enabled

      Description

      currently the single/multi/in-memory ratio in LruBlockCache is hardcoded 1:2:1, which can lead to somewhat counter-intuition behavior for some user scenario where in-memory table's read performance is much worse than ordinary table when two tables' data size is almost equal and larger than regionserver's cache size (we ever did some such experiment and verified that in-memory table random read performance is two times worse than ordinary table).

      this patch fixes above issue and provides:
      1. make single/multi/in-memory ratio user-configurable
      2. provide a configurable switch which can make in-memory block preemptive, by preemptive means when this switch is on in-memory block can kick out any ordinary block to make room until no ordinary block, when this switch is off (by default) the behavior is the same as previous, using single/multi/in-memory ratio to determine evicting.

      by default, above two changes are both off and the behavior keeps the same as before applying this patch. it's client/user's choice to determine whether or which behavior to use by enabling one of these two enhancements.

      1. HBASE-10263-trunk_v0.patch
        15 kB
        Honghua Feng
      2. HBASE-10263-trunk_v1.patch
        14 kB
        Honghua Feng
      3. HBASE-10263-trunk_v2.patch
        14 kB
        Honghua Feng

        Activity

        Honghua Feng created issue -
        Honghua Feng made changes -
        Field Original Value New Value
        Attachment HBASE-10263-trunk_v0.patch [ 12620929 ]
        Ted Yu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Honghua Feng made changes -
        Attachment HBASE-10263-trunk_v1.patch [ 12621447 ]
        Honghua Feng made changes -
        Component/s io [ 12312135 ]
        Honghua Feng made changes -
        Attachment HBASE-10263-trunk_v2.patch [ 12621740 ]
        Liang Xie made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Resolution Fixed [ 1 ]
        Honghua Feng made changes -
        Release Note 4 new configurations are introduced:
        hbase.lru.blockcache.single.percentage / hbase.lru.blockcache.multi.percentage / hbase.lru.blockcache.memory.percentage / hbase.lru.rs.inmemoryforcemode
        the first 3 enable user to config different single/multi/memory ratio in block cache; the last one determines if preemptive mode for memory block is enabled
        Liang Xie made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Liang Xie made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Fix Version/s 0.98.0 [ 12323143 ]
        Resolution Fixed [ 1 ]
        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Honghua Feng
            Reporter:
            Honghua Feng
          • Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development