Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Show more detail on blockcache deployed in new BlockCache section under Tasks and above Regions; shows blockcache type, what for L1 and L2, help pointers, type-specific and L1/L2 stats as well as dumps of bucket/slab cache stats when bucket/slabs enabled. Keeps block cache age (works in Bucket and LruBlockCache, not for Slab). Adds JSON page with details on block caches and a facility for listing block cache content by file (useful finding hot files).
      Show
      Show more detail on blockcache deployed in new BlockCache section under Tasks and above Regions; shows blockcache type, what for L1 and L2, help pointers, type-specific and L1/L2 stats as well as dumps of bucket/slab cache stats when bucket/slabs enabled. Keeps block cache age (works in Bucket and LruBlockCache, not for Slab). Adds JSON page with details on block caches and a facility for listing block cache content by file (useful finding hot files).

      Description

      Summarized block-cache report for a RegionServer would be helpful. For example ...

      table1
      cf1 100 blocks, totalBytes=yyyyy, averageTimeInCache=XXXX hours
      cf2 200 blocks, totalBytes=zzzzz, averageTimeInCache=XXXX hours

      table2
      cf1 75 blocks, totalBytes=yyyyy, averageTimeInCache=XXXX hours
      cf2 150 blocks, totalBytes=zzzzz, averageTimeInCache=XXXX hours

      ... Etc.

      The current metrics list blockCacheSize and blockCacheFree, but there is no way to know what's in there. Any single block isn't really important, but the patterns of what CF/Table they came from, how big are they, and how long (on average) they've been in the cache, are important.

      No such interface exists in HRegionInterface. But I think it would be helpful from an operational perspective.

      Updated (7-29): Removing suggestion for UI. I would be happy just to get this report on a configured interval dumped to a log file.

        Attachments

        1. 4089.wip.txt
          89 kB
          stack
        2. 4089.wip.v2.txt
          119 kB
          stack
        3. 4089v3.txt
          147 kB
          stack
        4. 4089v4.txt
          149 kB
          stack
        5. 4089v6.txt
          151 kB
          stack
        6. 4089v7.txt
          151 kB
          stack
        7. 4089v7.txt
          151 kB
          stack
        8. 4089v8.txt
          151 kB
          stack
        9. bc_basic.png
          128 kB
          stack
        10. bc_bucketcache.png
          98 kB
          stack
        11. bc_buckets.png
          91 kB
          stack
        12. bc_config.png
          121 kB
          stack
        13. bc_l1.png
          89 kB
          stack
        14. bc_l2_buckets.png
          140 kB
          stack
        15. bc_showing_buckets_start.png
          50 kB
          stack
        16. bc_slabs.png
          199 kB
          stack
        17. bc_slabs.png
          110 kB
          stack
        18. bc_stats.png
          109 kB
          stack
        19. hbase_4089_blockcachereport.pdf
          145 kB
          Doug Meil
        20. java_blockcache_checkpoint_2011_08_11.patch
          13 kB
          Doug Meil
        There are no Sub-Tasks for this issue.

          Activity

            People

            • Assignee:
              stack stack
              Reporter:
              dmeil Doug Meil
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: