HBase
  1. HBase
  2. HBASE-5601

Add per-column-family data block cache hit ratios

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: metrics, monitoring
    • Labels:
      None

      Description

      In addition to the overall block cache hit ratio it would be extremely useful to have per-column-family data block cache hit ratio metrics.

        Issue Links

          Activity

          Mikhail Bautin created issue -
          Hide
          Enis Soztutar added a comment -

          I have started working on this (and other per-table-cf metrics), but soon realized that some of the changes require breaking compatibility with (soon to be released) 0.94 in terms of metric names. So I decided to list the proposed changes first to see whether we can come up with a plan.

          As you know, HBASE-4219 and others added per-cf metrics, but there are a couple of things we can improve on that (great) work. Some ideas I have:

          • Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt)
          • Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX
          • Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction.
          • For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them?
          • Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)?
          • With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc.
          • Add a configuration for disabling per-cf metrics.
          • Thoughts ?
          Show
          Enis Soztutar added a comment - I have started working on this (and other per-table-cf metrics), but soon realized that some of the changes require breaking compatibility with (soon to be released) 0.94 in terms of metric names. So I decided to list the proposed changes first to see whether we can come up with a plan. As you know, HBASE-4219 and others added per-cf metrics, but there are a couple of things we can improve on that (great) work. Some ideas I have: Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt) Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction. For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them? Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)? With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc. Add a configuration for disabling per-cf metrics. Thoughts ?
          Enis Soztutar made changes -
          Field Original Value New Value
          Assignee Enis Soztutar [ enis ]
          Enis Soztutar made changes -
          Link This issue is related to HBASE-5701 [ HBASE-5701 ]
          Hide
          stack added a comment -

          Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt)

          Is it really named the latter? Please fix (Proposal looks good).

          Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX

          hbase-5701 just committed did this (sort-of).

          Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction.

          I don't follow the above. Something didn't make it across.

          For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them?

          Sure, on compaction.

          Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)?

          That would be sweet but maybe for later.

          With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc.

          Having them distinctly named makes sense to me.

          Add a configuration for disabling per-cf metrics.

          Yes. There could be an overwhelming amount.

          ba. Thoughts?

          Sounds excellent. Sounds like work that would be checked in with the singularity, 0.96?

          Show
          stack added a comment - Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt) Is it really named the latter? Please fix (Proposal looks good). Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX hbase-5701 just committed did this (sort-of). Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction. I don't follow the above. Something didn't make it across. For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them? Sure, on compaction. Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)? That would be sweet but maybe for later. With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc. Having them distinctly named makes sense to me. Add a configuration for disabling per-cf metrics. Yes. There could be an overwhelming amount. ba. Thoughts? Sounds excellent. Sounds like work that would be checked in with the singularity, 0.96?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4675/
          -----------------------------------------------------------

          Review request for hbase.

          Summary
          -------

          HBASE-5601. Add per-column-family data block cache hit ratios. First shot at the patch.

          Changes are as follows (some are backwards incompatible):

          • Renamed hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX
          • Renamed the metrics name generation from
            hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt
            -> tbl=usertable.cf=ycsb.bt=Data.at=scan.blockCacheHitCount
            Notice that tbl.TableName is changed to tbl=TableName, cf.Family is
            changed to cf=Family, the fs or compaction prefix is changed to
            at=scan or at=compaction. at stands for access type.
          • Through using CacheAccessStats/CacheSizeStats, metric names
            across per-cf and global are made consistent.
          • For compaction, per-block type metrics are disabled. For compaction
            we do not need to distinguish between block types.
          • Added a configuration "hbase.metrics.dynamic.enabled" for disabling
            dynamic metrics. It is enabled by default.
          • Changed bloom filter metrics from keyMaybeInBloomCnt -> bloomHitCount
          • Removed some code for keeping BC related to META block types

          Some changes in HFileReaders are due to automatic trailing whitespace
          settings in my eclipse. I can regenerate those parts if it is hard to
          review.

          This addresses bug HBASE-5601.
          https://issues.apache.org/jira/browse/HBASE-5601

          Diffs


          src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java e3c4fe4
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessMetrics.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStats.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 2ba3ab2
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeMetrics.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStats.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java ef03075
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java e952ef9
          src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 79ffee0
          src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java aa99add
          src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 91562f7
          src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 87634ba
          src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 1795229
          src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 251ccd4
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java f1a68e0
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java 3b46dd8
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java da08532
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java 5ad7406
          src/main/resources/hbase-default.xml 44ee689
          src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java f7430ee
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java 5f8214e
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java d980948
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java 17284a3
          src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java bd83a71
          src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java dbe425f

          Diff: https://reviews.apache.org/r/4675/diff

          Testing
          -------

          Tested with existing unit tests, and manually using FileContext

          Thanks,

          enis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4675/ ----------------------------------------------------------- Review request for hbase. Summary ------- HBASE-5601 . Add per-column-family data block cache hit ratios. First shot at the patch. Changes are as follows (some are backwards incompatible): Renamed hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX Renamed the metrics name generation from hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.blockCacheHitCount Notice that tbl.TableName is changed to tbl=TableName, cf.Family is changed to cf=Family, the fs or compaction prefix is changed to at=scan or at=compaction. at stands for access type. Through using CacheAccessStats/CacheSizeStats, metric names across per-cf and global are made consistent. For compaction, per-block type metrics are disabled. For compaction we do not need to distinguish between block types. Added a configuration "hbase.metrics.dynamic.enabled" for disabling dynamic metrics. It is enabled by default. Changed bloom filter metrics from keyMaybeInBloomCnt -> bloomHitCount Removed some code for keeping BC related to META block types Some changes in HFileReaders are due to automatic trailing whitespace settings in my eclipse. I can regenerate those parts if it is hard to review. This addresses bug HBASE-5601 . https://issues.apache.org/jira/browse/HBASE-5601 Diffs src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java e3c4fe4 src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessMetrics.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStats.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 2ba3ab2 src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeMetrics.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStats.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java ef03075 src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java e952ef9 src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 79ffee0 src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java aa99add src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 91562f7 src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 87634ba src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 1795229 src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 251ccd4 src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java f1a68e0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java 3b46dd8 src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java da08532 src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java 5ad7406 src/main/resources/hbase-default.xml 44ee689 src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java f7430ee src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java 5f8214e src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java d980948 src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java 17284a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java bd83a71 src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java dbe425f Diff: https://reviews.apache.org/r/4675/diff Testing ------- Tested with existing unit tests, and manually using FileContext Thanks, enis
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4675/
          -----------------------------------------------------------

          (Updated 2012-04-07 03:40:41.839514)

          Review request for hbase.

          Changes
          -------

          Fixed a failing test case.

          Summary
          -------

          HBASE-5601. Add per-column-family data block cache hit ratios. First shot at the patch.

          Changes are as follows (some are backwards incompatible):

          • Renamed hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX
          • Renamed the metrics name generation from
            hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt
            -> tbl=usertable.cf=ycsb.bt=Data.at=scan.blockCacheHitCount
            Notice that tbl.TableName is changed to tbl=TableName, cf.Family is
            changed to cf=Family, the fs or compaction prefix is changed to
            at=scan or at=compaction. at stands for access type.
          • Through using CacheAccessStats/CacheSizeStats, metric names
            across per-cf and global are made consistent.
          • For compaction, per-block type metrics are disabled. For compaction
            we do not need to distinguish between block types.
          • Added a configuration "hbase.metrics.dynamic.enabled" for disabling
            dynamic metrics. It is enabled by default.
          • Changed bloom filter metrics from keyMaybeInBloomCnt -> bloomHitCount
          • Removed some code for keeping BC related to META block types

          Some changes in HFileReaders are due to automatic trailing whitespace
          settings in my eclipse. I can regenerate those parts if it is hard to
          review.

          This addresses bug HBASE-5601.
          https://issues.apache.org/jira/browse/HBASE-5601

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 2ba3ab2
          src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java e3c4fe4
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessMetrics.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStats.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeMetrics.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStats.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java ef03075
          src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStatsImpl.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java e952ef9
          src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 79ffee0
          src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java aa99add
          src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 91562f7
          src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 87634ba
          src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 1795229
          src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 251ccd4
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java f1a68e0
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java 3b46dd8
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java da08532
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java 5ad7406
          src/main/resources/hbase-default.xml 44ee689
          src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java f7430ee
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java 5f8214e
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java d980948
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java 17284a3
          src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java bd83a71
          src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java dbe425f

          Diff: https://reviews.apache.org/r/4675/diff

          Testing
          -------

          Tested with existing unit tests, and manually using FileContext

          Thanks,

          enis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4675/ ----------------------------------------------------------- (Updated 2012-04-07 03:40:41.839514) Review request for hbase. Changes ------- Fixed a failing test case. Summary ------- HBASE-5601 . Add per-column-family data block cache hit ratios. First shot at the patch. Changes are as follows (some are backwards incompatible): Renamed hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX Renamed the metrics name generation from hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.blockCacheHitCount Notice that tbl.TableName is changed to tbl=TableName, cf.Family is changed to cf=Family, the fs or compaction prefix is changed to at=scan or at=compaction. at stands for access type. Through using CacheAccessStats/CacheSizeStats, metric names across per-cf and global are made consistent. For compaction, per-block type metrics are disabled. For compaction we do not need to distinguish between block types. Added a configuration "hbase.metrics.dynamic.enabled" for disabling dynamic metrics. It is enabled by default. Changed bloom filter metrics from keyMaybeInBloomCnt -> bloomHitCount Removed some code for keeping BC related to META block types Some changes in HFileReaders are due to automatic trailing whitespace settings in my eclipse. I can regenerate those parts if it is hard to review. This addresses bug HBASE-5601 . https://issues.apache.org/jira/browse/HBASE-5601 Diffs (updated) src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 2ba3ab2 src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java e3c4fe4 src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessMetrics.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStats.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheAccessStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeMetrics.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStats.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheSizeStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java ef03075 src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStatsImpl.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java e952ef9 src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java 79ffee0 src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java aa99add src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 91562f7 src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 87634ba src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java 1795229 src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java 251ccd4 src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java f1a68e0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java 3b46dd8 src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java da08532 src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java 5ad7406 src/main/resources/hbase-default.xml 44ee689 src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java f7430ee src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java 5f8214e src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java d980948 src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java 17284a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java bd83a71 src/test/java/org/apache/hadoop/hbase/regionserver/metrics/TestSchemaMetrics.java dbe425f Diff: https://reviews.apache.org/r/4675/diff Testing ------- Tested with existing unit tests, and manually using FileContext Thanks, enis
          Otis Gospodnetic made changes -
          Component/s metrics [ 12314281 ]
          Component/s monitoring [ 12315704 ]
          Hide
          stack added a comment -

          Resolving as won't fix for trunk at least (may be in 0.89fb). The items raised by Enis in the description are no longer apt given per-cf metrics were removed by HBASE-6410 (in 0.96) – too expensive to calculate.

          That said, let me respond to the original list since much of it still has merit and I am trying to address in our new per-1.0 context.

          Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt)

          This change looks done though rather than hbase.regionserver.blockCacheHitCount we have hbase.regionserver.server.blockCacheHitCount (The mbean that hosts this metric is hbase.regionserver.server).

          All blockcache metrics have a blockCache prefix in hadoop metrics2 (except 'blockCountHitPercent' which I'll fix).

          Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX

          Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction.
          For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them?

          Not pertinent to trunk.

          Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)?

          Not pertinent anymore.

          With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc.

          We need this. Will file an issue.

          Add a configuration for disabling per-cf metrics.

          No longer pertinent since removal.

          Show
          stack added a comment - Resolving as won't fix for trunk at least (may be in 0.89fb). The items raised by Enis in the description are no longer apt given per-cf metrics were removed by HBASE-6410 (in 0.96) – too expensive to calculate. That said, let me respond to the original list since much of it still has merit and I am trying to address in our new per-1.0 context. Make Per-cf and global block cache metrics naming consistent (hbase.regionserver.blockCacheHitCount vs hbase.RegionServerDynamicStatistics.tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt) This change looks done though rather than hbase.regionserver.blockCacheHitCount we have hbase.regionserver.server.blockCacheHitCount (The mbean that hosts this metric is hbase.regionserver.server). All blockcache metrics have a blockCache prefix in hadoop metrics2 (except 'blockCountHitPercent' which I'll fix). Rename hbase.RegionServerDynamicStatistics.XXX -> hbase.regionserver.dyn.XXX Rename dynamic metric names tbl.usertable.cf.ycsb.bt.Data.fsBlockReadCacheHitCnt -> tbl=usertable.cf=ycsb.bt=Data.at=scan.BlockCacheHitCount. Here at stands for access type, and can be scan or compaction. For at=compaction, we probably do not need per-block type (Data, Index, ...) metric values, shall we aggregate them? Not pertinent to trunk. Shall we extend per-cf metrics into HBASE-5533 (read/write latency histograms)? Not pertinent anymore. With slab cache, we may no longer have one global block cache. Shall we enable per-cache type metrics? This can imply renaming hbase.regionserver.blockCache => hbase.regionserver.lrublockcache, slabblockcache, etc. We need this. Will file an issue. Add a configuration for disabling per-cf metrics. No longer pertinent since removal.
          stack made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Hide
          stack added a comment -

          Oh, reopen if I am wrong in the above Enis Soztutar

          Show
          stack added a comment - Oh, reopen if I am wrong in the above Enis Soztutar
          Hide
          Enis Soztutar added a comment -

          Thanks Stack. Agreed with the resolution. We can continue on this piecemeal on other issues.

          Show
          Enis Soztutar added a comment - Thanks Stack. Agreed with the resolution. We can continue on this piecemeal on other issues.
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          792d 23h 31m 1 stack 21/May/14 21:38

            People

            • Assignee:
              Enis Soztutar
              Reporter:
              Mikhail Bautin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development