HBase
  1. HBase
  2. HBASE-11527

Cluster free memory limit check should consider L2 block cache size also when L2 cache is onheap.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, 2.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      The sum of the heap size % used by Memstore (Configured using 'hbase.regionserver.global.memstore.size' and defaults to 40%) and the that of block cache (Configured using 'hfile.block.cache.size' and defaults to 40%) should leave enough heap size for other normal operations of RS. This is 20%. Also to be noted that when the L2 block cache (ie. Bucket cache) is configured to be on heap, the size of that also comes into this size math. So in that case the sum of memstore heap size, L1 block cache size and L2 cache size (Configured using 'hbase.bucketcache.size')can be at max 80%.
      When automatic tuning of heap memory is enabled (HBASE-5349) at point of time this 80% cap should hold.
      Show
      The sum of the heap size % used by Memstore (Configured using 'hbase.regionserver.global.memstore.size' and defaults to 40%) and the that of block cache (Configured using 'hfile.block.cache.size' and defaults to 40%) should leave enough heap size for other normal operations of RS. This is 20%. Also to be noted that when the L2 block cache (ie. Bucket cache) is configured to be on heap, the size of that also comes into this size math. So in that case the sum of memstore heap size, L1 block cache size and L2 cache size (Configured using 'hbase.bucketcache.size')can be at max 80%. When automatic tuning of heap memory is enabled ( HBASE-5349 ) at point of time this 80% cap should hold.
    1. HBASE-11527.patch
      30 kB
      Anoop Sam John
    2. HBASE-11527_0.99.patch
      36 kB
      Anoop Sam John
    3. HBASE-11527_addendum.patch
      8 kB
      Anoop Sam John

      Activity

      Enis Soztutar made changes -
      Status Resolved [ 5 ] Closed [ 6 ]
      Anoop Sam John made changes -
      Release Note The sum of the heap size % used by Memstore (Configured using 'hbase.regionserver.global.memstore.size' and defaults to 40%) and the that of block cache (Configured using 'hfile.block.cache.size' and defaults to 40%) should leave enough heap size for other normal operations of RS. This is 20%. Also to be noted that when the L2 block cache (ie. Bucket cache) is configured to be on heap, the size of that also comes into this size math. So in that case the sum of memstore heap size, L1 block cache size and L2 cache size (Configured using 'hbase.bucketcache.size')can be at max 80%.
      When automatic tuning of heap memory is enabled (HBASE-5349) at point of time this 80% cap should hold.
      Anoop Sam John made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Resolution Fixed [ 1 ]
      Anoop Sam John made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Fix Version/s 0.99.0 [ 12325675 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527_addendum.patch [ 12660627 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527_addendum.patch [ 12660394 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527_0.99.patch [ 12660626 ]
      Anoop Sam John made changes -
      Status Patch Available [ 10002 ] Open [ 1 ]
      Anoop Sam John made changes -
      Status Reopened [ 4 ] Patch Available [ 10002 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527_addendum.patch [ 12660394 ]
      Anoop Sam John made changes -
      Resolution Fixed [ 1 ]
      Status Resolved [ 5 ] Reopened [ 4 ]
      Anoop Sam John made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Hadoop Flags Reviewed [ 10343 ]
      Resolution Fixed [ 1 ]
      Anoop Sam John made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12659110 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657560 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657584 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657585 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12658589 ]
      Anoop Sam John made changes -
      Status Patch Available [ 10002 ] Open [ 1 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12658668 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12658668 ]
      stack made changes -
      Attachment HBASE-11527.patch [ 12658589 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657585 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657584 ]
      Anoop Sam John made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Anoop Sam John made changes -
      Attachment HBASE-11527.patch [ 12657560 ]
      Anoop Sam John made changes -
      Affects Version/s 0.99.0 [ 12325675 ]
      Anoop Sam John made changes -
      Field Original Value New Value
      Fix Version/s 0.99.0 [ 12325675 ]
      Anoop Sam John created issue -

        People

        • Assignee:
          Anoop Sam John
          Reporter:
          Anoop Sam John
        • Votes:
          0 Vote for this issue
          Watchers:
          7 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development