Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16401 Enable HeapMemoryManager by default in 2.0
  3. HBASE-16650

Wrong usage of BlockCache eviction stat for heap memory tuning

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 2.0.0
    • None
    • None
    • Incompatible change, Reviewed
    • Changed tracking of evictedBlocks count NOT to include evictions of blocks for a removed HFile. HFiles gets removed after compaction

    Description

      1. We use the stat evictedBlocksCount - A block can get evicted because of eviction thread due to lack of space or because of removal of an HFile itself (After a compaction). We should not consider the latter in the tune decision at all. These are actually invalidation of blocks. Should the stat counter itself not use this count of evicted blocks? I think yes. This will give wrong message to users that there are lot of real eviction happening.
      2. In case L1+ L2 combined block cache, what we use is the sum of evictions from both. But we will be tuning L1 size alone. Eviction count from L2 should not affect the tuning of L1

      Attachments

        1. HBASE-16650.patch
          9 kB
          Anoop Sam John
        2. HBASE-16650_V2.patch
          10 kB
          Anoop Sam John
        3. HBASE-16650_V3.patch
          11 kB
          Anoop Sam John
        4. HBASE-16650_V4.patch
          15 kB
          Anoop Sam John
        5. HBASE-16650_V5.patch
          16 kB
          Anoop Sam John

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            anoop.hbase Anoop Sam John
            anoop.hbase Anoop Sam John
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment