HBase
  1. HBase
  2. HBASE-10048

Add hlog number metric in regionserver

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.96.1, 0.94.15, 0.99.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Add hlog number metric in regionserver.

      We can use this metric to alert about memstore flush because of too many hlogs.

      1. 10048.096.v4.txt
        20 kB
        stack
      2. HBASE-10048-0.94-v1.diff
        3 kB
        Liu Shaohui
      3. HBASE-10048-0.94-v2.diff
        7 kB
        Liu Shaohui
      4. HBASE-10048-0.94-v3.diff
        10 kB
        Liu Shaohui
      5. HBASE-10048-trunk-v1.diff
        9 kB
        Liu Shaohui
      6. HBASE-10048-trunk-v2.diff
        13 kB
        Liu Shaohui
      7. HBASE-10048-trunk-v3.diff
        13 kB
        Liu Shaohui
      8. HBASE-10048-trunk-v4.diff
        27 kB
        Liu Shaohui

        Activity

        stack made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        stack added a comment -

        Released in 0.96.1. Issue closed.

        Show
        stack added a comment - Released in 0.96.1. Issue closed.
        Hide
        Lars Hofhansl added a comment -

        Thanks Liu and Ted.

        Show
        Lars Hofhansl added a comment - Thanks Liu and Ted.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94-security #360 (See https://builds.apache.org/job/HBase-0.94-security/360/)
        HBASE-10048 Add hlog number metric in regionserver (tedyu: rev 1550997)

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94-security #360 (See https://builds.apache.org/job/HBase-0.94-security/360/ ) HBASE-10048 Add hlog number metric in regionserver (tedyu: rev 1550997) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94 #1227 (See https://builds.apache.org/job/HBase-0.94/1227/)
        HBASE-10048 Add hlog number metric in regionserver (tedyu: rev 1550997)

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94 #1227 (See https://builds.apache.org/job/HBase-0.94/1227/ ) HBASE-10048 Add hlog number metric in regionserver (tedyu: rev 1550997) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Ted Yu made changes -
        Fix Version/s 0.94.15 [ 12325559 ]
        Hide
        Ted Yu added a comment -

        Integrated to 0.94

        Thanks Shaohui

        Show
        Ted Yu added a comment - Integrated to 0.94 Thanks Shaohui
        Liu Shaohui made changes -
        Attachment HBASE-10048-0.94-v3.diff [ 12618589 ]
        Hide
        Liu Shaohui added a comment -

        Lars Hofhansl

        Patch for 0.94.
        Sorry for late.

        Show
        Liu Shaohui added a comment - Lars Hofhansl Patch for 0.94. Sorry for late.
        Hide
        Lars Hofhansl added a comment -

        Liu Shaohui, are you still planning a new 0.94 patch?

        Show
        Lars Hofhansl added a comment - Liu Shaohui , are you still planning a new 0.94 patch?
        Ted Yu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96-hadoop2 #144 (See https://builds.apache.org/job/hbase-0.96-hadoop2/144/)
        HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548917)

        • /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
        • /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
        • /hbase/branches/0.96/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.96/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96-hadoop2 #144 (See https://builds.apache.org/job/hbase-0.96-hadoop2/144/ ) HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548917) /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java /hbase/branches/0.96/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.96/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96 #218 (See https://builds.apache.org/job/hbase-0.96/218/)
        HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548917)

        • /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
        • /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
        • /hbase/branches/0.96/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.96/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96 #218 (See https://builds.apache.org/job/hbase-0.96/218/ ) HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548917) /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java /hbase/branches/0.96/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java /hbase/branches/0.96/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.96/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.96/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Hide
        stack added a comment -

        Committed the 0.96 patch. Leaving open for 0.94 patch. Thanks Liu Shaohui

        Show
        stack added a comment - Committed the 0.96 patch. Leaving open for 0.94 patch. Thanks Liu Shaohui
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12617555/10048.096.v4.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 18 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8086//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12617555/10048.096.v4.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8086//console This message is automatically generated.
        stack made changes -
        Attachment 10048.096.v4.txt [ 12617555 ]
        Hide
        stack added a comment -

        Patch for 0.96 is a bit different. Uploading what I applied.

        Show
        stack added a comment - Patch for 0.96 is a bit different. Uploading what I applied.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.98-hadoop2 #2 (See https://builds.apache.org/job/hbase-0.98-hadoop2/2/)
        HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548769)

        • /hbase/branches/0.98/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
        • /hbase/branches/0.98/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
        • /hbase/branches/0.98/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.98/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/branches/0.98/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.98-hadoop2 #2 (See https://builds.apache.org/job/hbase-0.98-hadoop2/2/ ) HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548769) /hbase/branches/0.98/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java /hbase/branches/0.98/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java /hbase/branches/0.98/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.98/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/branches/0.98/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Hide
        Liu Shaohui added a comment -

        Lars Hofhansl

        The total blog size metric which is in trunk patch is not contained in 0.94 patch.
        Please wait and I will update the 0.94 patch.

        Show
        Liu Shaohui added a comment - Lars Hofhansl The total blog size metric which is in trunk patch is not contained in 0.94 patch. Please wait and I will update the 0.94 patch.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4715 (See https://builds.apache.org/job/HBase-TRUNK/4715/)
        HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548768)

        • /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
        • /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4715 (See https://builds.apache.org/job/HBase-TRUNK/4715/ ) HBASE-10048 Add hlog number metric in regionserver (stack: rev 1548768) /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogUtilsForTests.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
        Hide
        stack added a comment -

        Lars Hofhansl I added it to trunk. Will add to 0.94 after I shoehorn it into 0.96 (morrow)

        Show
        stack added a comment - Lars Hofhansl I added it to trunk. Will add to 0.94 after I shoehorn it into 0.96 (morrow)
        Hide
        Lars Hofhansl added a comment -

        That is nice. And something we would monitor. +1 for 0.94.
        Needs to be in trunk (0.99 now) as well, right?

        Show
        Lars Hofhansl added a comment - That is nice. And something we would monitor. +1 for 0.94. Needs to be in trunk (0.99 now) as well, right?
        Hide
        stack added a comment -

        Applied to trunk and to 0.98. 0.96 not going in. Will fix it later. Lars Hofhansl You want this? Its nice.

        Show
        stack added a comment - Applied to trunk and to 0.98. 0.96 not going in. Will fix it later. Lars Hofhansl You want this? Its nice.
        Hide
        Elliott Clark added a comment -

        +1 thanks

        Show
        Elliott Clark added a comment - +1 thanks
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        Fix Version/s 0.96.1 [ 12324961 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Component/s metrics [ 12314281 ]
        Hide
        stack added a comment -

        +1 Patch is great. Elliott Clark Please bless and then I'll commit.

        Show
        stack added a comment - +1 Patch is great. Elliott Clark Please bless and then I'll commit.
        Hide
        Liu Shaohui added a comment -

        chunhui shen Ted Yu Himanshu Vashishtha
        Anyone helps to push this issue? thx

        Show
        Liu Shaohui added a comment - chunhui shen Ted Yu Himanshu Vashishtha Anyone helps to push this issue? thx
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12616734/HBASE-10048-trunk-v4.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 18 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 site. The patch appears to cause mvn site goal to fail.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12616734/HBASE-10048-trunk-v4.diff against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8050//console This message is automatically generated.
        Liu Shaohui made changes -
        Attachment HBASE-10048-trunk-v4.diff [ 12616734 ]
        Hide
        Liu Shaohui added a comment -

        Update for Himanshu Vashishtha 's review.

        rename to getNumLogFiles to getNumRolledLogFiles and add actually getNumLogFiles method which contain current hlog file.

        For there may be multi hlog writers, we should make this metric contain current hlog files.

        Show
        Liu Shaohui added a comment - Update for Himanshu Vashishtha 's review. rename to getNumLogFiles to getNumRolledLogFiles and add actually getNumLogFiles method which contain current hlog file. For there may be multi hlog writers, we should make this metric contain current hlog files.
        Hide
        Liu Shaohui added a comment -

        (I'd avoid doing the "numLogs +1" stuff, as it looks brittle). IMO, it is OKAY to show the rolled WALs and leave out the current one. And, name the metrics accordingly. What do you think Liu Shaohui?

        I agree. thx.
        I will update the patch later.

        Show
        Liu Shaohui added a comment - (I'd avoid doing the "numLogs +1" stuff, as it looks brittle). IMO, it is OKAY to show the rolled WALs and leave out the current one. And, name the metrics accordingly. What do you think Liu Shaohui? I agree. thx. I will update the patch later.
        Hide
        Liu Shaohui added a comment -

        chunhui shenHimanshu Vashishtha
        First, it's easy to get the accurate total size of all hlog files with this patch, so we don't need to guess it.
        Secondly, the accurate log size metric is necessary and very useful to optimize the failover time.

        Show
        Liu Shaohui added a comment - chunhui shen Himanshu Vashishtha First, it's easy to get the accurate total size of all hlog files with this patch, so we don't need to guess it. Secondly, the accurate log size metric is necessary and very useful to optimize the failover time.
        Hide
        chunhui shen added a comment -

        bq,couldn't just we make an intelligent "guess" about log size rather than creating a separate metrics for the same.
        I know the hlog size is about 95% of block size in normal case,
        but we have seen the hlog file size much bigger than block size in abnormal case.

        I think such a metric could show the unhealthy scenario timely, avoiding the hlog size become very huge.

        Maybe we only need the metric for the max size of current hlog.
        Liu Shaohui Any thought about the log size metric?

        Show
        chunhui shen added a comment - bq,couldn't just we make an intelligent "guess" about log size rather than creating a separate metrics for the same. I know the hlog size is about 95% of block size in normal case, but we have seen the hlog file size much bigger than block size in abnormal case. I think such a metric could show the unhealthy scenario timely, avoiding the hlog size become very huge. Maybe we only need the metric for the max size of current hlog. Liu Shaohui Any thought about the log size metric?
        Hide
        Himanshu Vashishtha added a comment -

        +1 to the idea of exposing log numbers, and thanks for the patch.
        (I'd avoid doing the "numLogs +1" stuff, as it looks brittle). IMO, it is OKAY to show the rolled WALs and leave out the current one. And, name the metrics accordingly. What do you think Liu Shaohui?

        chunhui shen: Pardon for chiming in late; but couldn't just we make an intelligent "guess" about log size rather than creating a separate metrics for the same.
        https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L360

        Show
        Himanshu Vashishtha added a comment - +1 to the idea of exposing log numbers, and thanks for the patch. (I'd avoid doing the "numLogs +1" stuff, as it looks brittle). IMO, it is OKAY to show the rolled WALs and leave out the current one. And, name the metrics accordingly. What do you think Liu Shaohui ? chunhui shen : Pardon for chiming in late; but couldn't just we make an intelligent "guess" about log size rather than creating a separate metrics for the same. https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java#L360
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12616374/HBASE-10048-trunk-v3.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 site. The patch appears to cause mvn site goal to fail.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12616374/HBASE-10048-trunk-v3.diff against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8030//console This message is automatically generated.
        Liu Shaohui made changes -
        Attachment HBASE-10048-trunk-v3.diff [ 12616374 ]
        Hide
        Liu Shaohui added a comment -

        Fix the failed tests

        Show
        Liu Shaohui added a comment - Fix the failed tests
        Hide
        Liu Shaohui added a comment -

        It's the problem of code. getFileStatus should be before archiveLogFile

        • archiveLogFile(p);
          this.totalLogSize.addAndGet(-this.fs.getFileStatus(p).getLen());
          + archiveLogFile(p);
        Show
        Liu Shaohui added a comment - It's the problem of code. getFileStatus should be before archiveLogFile archiveLogFile(p); this.totalLogSize.addAndGet(-this.fs.getFileStatus(p).getLen()); + archiveLogFile(p);
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12616354/HBASE-10048-trunk-v2.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 site. The patch appears to cause mvn site goal to fail.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS
        org.apache.hadoop.hbase.client.TestAdmin
        org.apache.hadoop.hbase.regionserver.wal.TestHLog
        org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
        org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
        org.apache.hadoop.hbase.regionserver.wal.TestLogRolling

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12616354/HBASE-10048-trunk-v2.diff against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.regionserver.wal.TestHLog org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed org.apache.hadoop.hbase.regionserver.wal.TestLogRolling Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8028//console This message is automatically generated.
        chunhui shen made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Liu Shaohui made changes -
        Attachment HBASE-10048-trunk-v2.diff [ 12616354 ]
        Hide
        Liu Shaohui added a comment -

        Add hlog file size metric

        chunhui shen

        Show
        Liu Shaohui added a comment - Add hlog file size metric chunhui shen
        Hide
        Liu Shaohui added a comment -

        chunhui shen

        Total size of all logs seems useful too since one hlog would be very big in normal case

        Yes. I will add this metric too. Thx chunhui shen

        Show
        Liu Shaohui added a comment - chunhui shen Total size of all logs seems useful too since one hlog would be very big in normal case Yes. I will add this metric too. Thx chunhui shen
        Hide
        chunhui shen added a comment -
        +      // +1 for the current log
        +      numHLogFiles = regionServer.hlog.getNumLogFiles() + 1;
        

        Need +2 for MetaServer.
        But I think no addition is acceptable here.

        Total size of all logs seems useful too since one hlog would be very big in normal case

        Other looks good to me

        Show
        chunhui shen added a comment - + // +1 for the current log + numHLogFiles = regionServer.hlog.getNumLogFiles() + 1; Need +2 for MetaServer. But I think no addition is acceptable here. Total size of all logs seems useful too since one hlog would be very big in normal case Other looks good to me
        Liu Shaohui made changes -
        Attachment HBASE-10048-trunk-v1.diff [ 12616230 ]
        Hide
        Liu Shaohui added a comment -

        Patch for trunk.
        Don't +1 in getNumLogFiles, for it is used in many others places.

        Show
        Liu Shaohui added a comment - Patch for trunk. Don't +1 in getNumLogFiles, for it is used in many others places.
        Hide
        Ted Yu added a comment -

        Trunk patch please.

        Show
        Ted Yu added a comment - Trunk patch please.
        Liu Shaohui made changes -
        Attachment HBASE-10048-0.94-v2.diff [ 12616204 ]
        Hide
        Liu Shaohui added a comment -

        Ted Yu
        updates:
        add 1 for the current hlog in getNumLogFiles

        Show
        Liu Shaohui added a comment - Ted Yu updates: add 1 for the current hlog in getNumLogFiles
        Hide
        Ted Yu added a comment -

        Okay.

        Can you attach trunk patch ?

        Show
        Ted Yu added a comment - Okay. Can you attach trunk patch ?
        Hide
        Liu Shaohui added a comment -

        Ted Yu

        hlog.getNumLogFiles() only returns the number of all log files but the current one, so we need to add one for the current log.
        Maybe, it's clearer to add 1 in the method of hlog.getNumLogFiles().

        Show
        Liu Shaohui added a comment - Ted Yu hlog.getNumLogFiles() only returns the number of all log files but the current one, so we need to add one for the current log. Maybe, it's clearer to add 1 in the method of hlog.getNumLogFiles().
        Hide
        Ted Yu added a comment -
        +    this.metrics.hlogfiles.set(this.hlog.getNumLogFiles() + 1);
        

        Why '+1' is needed ?

        Trunk patch is welcome.

        Show
        Ted Yu added a comment - + this .metrics.hlogfiles.set( this .hlog.getNumLogFiles() + 1); Why '+1' is needed ? Trunk patch is welcome.
        Liu Shaohui made changes -
        Field Original Value New Value
        Attachment HBASE-10048-0.94-v1.diff [ 12616030 ]
        Hide
        Liu Shaohui added a comment -

        Patch for 0.94.
        After it accepted, i will make a patch for trunk.

        Show
        Liu Shaohui added a comment - Patch for 0.94. After it accepted, i will make a patch for trunk.
        Liu Shaohui created issue -

          People

          • Assignee:
            Liu Shaohui
            Reporter:
            Liu Shaohui
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development