Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Client, regionserver
    • Labels:
      None

      Description

      Moving issue w/ no recent movement out of 0.95

        Activity

        Nicolas Spiegelberg created issue -
        Nicolas Spiegelberg made changes -
        Field Original Value New Value
        Fix Version/s 0.94.0 [ 12316419 ]
        Nicolas Spiegelberg made changes -
        Component/s client [ 12312131 ]
        Component/s regionserver [ 12312139 ]
        Hide
        Nicolas Spiegelberg added a comment -

        Need to port from 89-fb. See SVN #1181972

        Show
        Nicolas Spiegelberg added a comment - Need to port from 89-fb. See SVN #1181972
        Hide
        Mubarak Seyed added a comment -

        If no one is working on this issue, i can help porting from 89-fb. Thanks.

        Show
        Mubarak Seyed added a comment - If no one is working on this issue, i can help porting from 89-fb. Thanks.
        Hide
        Otis Gospodnetic added a comment -

        Yes please, +1 fo mo metrix!

        Show
        Otis Gospodnetic added a comment - Yes please, +1 fo mo metrix!
        Hide
        Lars Hofhansl added a comment -

        Any plans for working on this soon? Otherwise please move to 0.96.

        Show
        Lars Hofhansl added a comment - Any plans for working on this soon? Otherwise please move to 0.96.
        Hide
        Lars Hofhansl added a comment -

        Moving out of 0.94, pull back if you disagree.

        Show
        Lars Hofhansl added a comment - Moving out of 0.94, pull back if you disagree.
        Lars Hofhansl made changes -
        Fix Version/s 0.96.0 [ 12320040 ]
        Fix Version/s 0.94.0 [ 12316419 ]
        stack made changes -
        Fix Version/s 0.95.0 [ 12324094 ]
        Fix Version/s 0.96.0 [ 12320040 ]
        Lars Hofhansl made changes -
        Fix Version/s 0.94.0 [ 12316419 ]
        Lars Hofhansl made changes -
        Fix Version/s 0.94.0 [ 12316419 ]
        stack made changes -
        Fix Version/s 0.95.1 [ 12324288 ]
        Fix Version/s 0.95.0 [ 12324094 ]
        stack made changes -
        Fix Version/s 0.95.1 [ 12324288 ]
        Description Moving issue w/ no recent movement out of 0.95
        Hide
        stack added a comment -

        Here is the commit Nicolas is referring to:

        ------------------------------------------------------------------------
        r1181972 | nspiegelberg | 2011-10-11 10:45:00 -0700 (Tue, 11 Oct 2011) | 25 lines

        Refactored and more detailed block read/cache and bloom metrics

        Summary: As we keep adding more granular block read and block cache usage
        statistics, there is a combinatorial explosion of the number of cases we have to
        monitor, especially when we want both per-column family / block type statistics
        and aggregate statistics on one or both of these dimensions. I am trying to
        unclutter HFile readers, LruBlockCache, StoreFile, etc. by creating a
        centralized class that knows how to update all kinds of per-column family/block
        type statistics.

        Test Plan:
        Run all unit tests.
        New unit test.
        Deploy to one region server in dark launch and compare the new output of
        hbaseStats.py to the old one (take a diff of the set of keys).

        Reviewers: pritam, liyintang, jgray, kannan

        Reviewed By: kannan

        CC: , hbase@lists, dist-storage@lists, kannan

        Differential Revision: 321147

        Looking at svn diff -r1181971:1181972... the commit is all about:

        + BlockCategory blockCategory = dataBlock.getBlockType().getCategory();

        ...

        and

        + cfMetrics.updateOnCacheMiss(blockCategory, isCompaction, delta);

        ... and this stuff in a class called ColumnFamilyMetrics:

        + READ_TIME("Read", true),
        + READ_COUNT("BlockReadCnt", true),
        + CACHE_HIT("BlockReadCacheHitCnt", true),
        + CACHE_MISS("BlockReadCacheMissCnt", true),
        +
        + CACHE_SIZE("blockCacheSize", false),
        + CACHED("blockCacheNumCached", false),
        + EVICTED("blockCacheNumEvicted", false);

        We have this. It is differently named, it is CacheStats. So, we have this detail. It came in with HBASE-4027, the slab cache issue. We need more but we have this much now so resolving as implemented.

        Show
        stack added a comment - Here is the commit Nicolas is referring to: ------------------------------------------------------------------------ r1181972 | nspiegelberg | 2011-10-11 10:45:00 -0700 (Tue, 11 Oct 2011) | 25 lines Refactored and more detailed block read/cache and bloom metrics Summary: As we keep adding more granular block read and block cache usage statistics, there is a combinatorial explosion of the number of cases we have to monitor, especially when we want both per-column family / block type statistics and aggregate statistics on one or both of these dimensions. I am trying to unclutter HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-column family/block type statistics. Test Plan: Run all unit tests. New unit test. Deploy to one region server in dark launch and compare the new output of hbaseStats.py to the old one (take a diff of the set of keys). Reviewers: pritam, liyintang, jgray, kannan Reviewed By: kannan CC: , hbase@lists, dist-storage@lists, kannan Differential Revision: 321147 Looking at svn diff -r1181971:1181972... the commit is all about: + BlockCategory blockCategory = dataBlock.getBlockType().getCategory(); ... and + cfMetrics.updateOnCacheMiss(blockCategory, isCompaction, delta); ... and this stuff in a class called ColumnFamilyMetrics: + READ_TIME("Read", true), + READ_COUNT("BlockReadCnt", true), + CACHE_HIT("BlockReadCacheHitCnt", true), + CACHE_MISS("BlockReadCacheMissCnt", true), + + CACHE_SIZE("blockCacheSize", false), + CACHED("blockCacheNumCached", false), + EVICTED("blockCacheNumEvicted", false); We have this. It is differently named, it is CacheStats. So, we have this detail. It came in with HBASE-4027 , the slab cache issue. We need more but we have this much now so resolving as implemented.
        Hide
        stack added a comment -

        Subsumed by HBASE-4027

        Show
        stack added a comment - Subsumed by HBASE-4027
        stack made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Nicolas Spiegelberg
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development