Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10263

make LruBlockCache single/multi/in-memory ratio user-configurable and provide preemptive mode for in-memory type block

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.99.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      4 new configurations are introduced:
      hbase.lru.blockcache.single.percentage / hbase.lru.blockcache.multi.percentage / hbase.lru.blockcache.memory.percentage / hbase.lru.rs.inmemoryforcemode
      the first 3 enable user to config different single/multi/memory ratio in block cache; the last one determines if preemptive mode for memory block is enabled
      Show
      4 new configurations are introduced: hbase.lru.blockcache.single.percentage / hbase.lru.blockcache.multi.percentage / hbase.lru.blockcache.memory.percentage / hbase.lru.rs.inmemoryforcemode the first 3 enable user to config different single/multi/memory ratio in block cache; the last one determines if preemptive mode for memory block is enabled

      Description

      currently the single/multi/in-memory ratio in LruBlockCache is hardcoded 1:2:1, which can lead to somewhat counter-intuition behavior for some user scenario where in-memory table's read performance is much worse than ordinary table when two tables' data size is almost equal and larger than regionserver's cache size (we ever did some such experiment and verified that in-memory table random read performance is two times worse than ordinary table).

      this patch fixes above issue and provides:
      1. make single/multi/in-memory ratio user-configurable
      2. provide a configurable switch which can make in-memory block preemptive, by preemptive means when this switch is on in-memory block can kick out any ordinary block to make room until no ordinary block, when this switch is off (by default) the behavior is the same as previous, using single/multi/in-memory ratio to determine evicting.

      by default, above two changes are both off and the behavior keeps the same as before applying this patch. it's client/user's choice to determine whether or which behavior to use by enabling one of these two enhancements.

        Attachments

        1. HBASE-10263-trunk_v2.patch
          14 kB
          Honghua Feng
        2. HBASE-10263-trunk_v1.patch
          14 kB
          Honghua Feng
        3. HBASE-10263-trunk_v0.patch
          15 kB
          Honghua Feng

          Activity

            People

            • Assignee:
              fenghh Honghua Feng
              Reporter:
              fenghh Honghua Feng
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: