Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.94.7, 0.95.1
    • Component/s: None
    • Labels:
      None

      Description

      Metrics should be added for snapshot.

      From Matteo: Output that we have in SnapshotInfo should be covered:

      "%d HFiles (%d in archive), total size %s (%.2f%% %s shared with the source table)"

      Jesse mentioned snaphot counts, average time to completion.
      I think we should have counts for successful / failed snapshots.

      1. HBASE-7615-v0.patch
        16 kB
        Matteo Bertozzi
      2. HBASE-7615-v0-ui-corrupted.png
        52 kB
        Matteo Bertozzi
      3. HBASE-7615-v0-ui.png
        45 kB
        Matteo Bertozzi
      4. HBASE-7615-v1.patch
        38 kB
        Matteo Bertozzi
      5. HBASE-7615-v2.patch
        38 kB
        Matteo Bertozzi
      6. HBASE-7615-0.94.patch
        40 kB
        Matteo Bertozzi
      7. HBASE-7615-0.94-v1.patch
        41 kB
        Matteo Bertozzi
      8. HBASE-7615-v3.patch
        38 kB
        Matteo Bertozzi
      9. HBASE-7615.patch
        56 kB
        Matteo Bertozzi
      10. HBASE-7615.patch
        58 kB
        Matteo Bertozzi
      11. HBASE-7615-v4.patch
        41 kB
        Matteo Bertozzi
      12. HBASE-7615.patch
        59 kB
        Matteo Bertozzi

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #133 (See https://builds.apache.org/job/HBase-0.94-security/133/)
        HBASE-7615 Add metrics for snapshots (Revision 1464931)

        Result = SUCCESS
        mbertozzi :
        Files :

        • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #133 (See https://builds.apache.org/job/HBase-0.94-security/133/ ) HBASE-7615 Add metrics for snapshots (Revision 1464931) Result = SUCCESS mbertozzi : Files : /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #947 (See https://builds.apache.org/job/HBase-0.94/947/)
        HBASE-7615 Add metrics for snapshots (Revision 1464931)

        Result = SUCCESS
        mbertozzi :
        Files :

        • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #947 (See https://builds.apache.org/job/HBase-0.94/947/ ) HBASE-7615 Add metrics for snapshots (Revision 1464931) Result = SUCCESS mbertozzi : Files : /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Matteo Bertozzi added a comment -

        Committed to 0.94

        Show
        Matteo Bertozzi added a comment - Committed to 0.94
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #54 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/54/)
        HBASE-7615 Add metrics for snapshots (Revision 1464443)

        Result = FAILURE
        mbertozzi :
        Files :

        • /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java
        • /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/branches/0.95/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #54 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/54/ ) HBASE-7615 Add metrics for snapshots (Revision 1464443) Result = FAILURE mbertozzi : Files : /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/branches/0.95/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #476 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/476/)
        HBASE-7615 Add metrics for snapshots (Revision 1464441)

        Result = FAILURE
        mbertozzi :
        Files :

        • /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java
        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #476 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/476/ ) HBASE-7615 Add metrics for snapshots (Revision 1464441) Result = FAILURE mbertozzi : Files : /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #123 (See https://builds.apache.org/job/hbase-0.95/123/)
        HBASE-7615 Add metrics for snapshots (Revision 1464443)

        Result = FAILURE
        mbertozzi :
        Files :

        • /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java
        • /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/branches/0.95/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #123 (See https://builds.apache.org/job/hbase-0.95/123/ ) HBASE-7615 Add metrics for snapshots (Revision 1464443) Result = FAILURE mbertozzi : Files : /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/branches/0.95/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #4012 (See https://builds.apache.org/job/HBase-TRUNK/4012/)
        HBASE-7615 Add metrics for snapshots (Revision 1464441)

        Result = FAILURE
        mbertozzi :
        Files :

        • /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java
        • /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java
        • /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #4012 (See https://builds.apache.org/job/HBase-TRUNK/4012/ ) HBASE-7615 Add metrics for snapshots (Revision 1464441) Result = FAILURE mbertozzi : Files : /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSource.java /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterSourceImpl.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/CloneSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/EnabledTableSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotInfo.java /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/snapshot.jsp /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotManager.java
        Hide
        Matteo Bertozzi added a comment -

        committed to 0.95 and trunk, thanks for the reviews guys.

        Anyone wants it in 94?

        Show
        Matteo Bertozzi added a comment - committed to 0.95 and trunk, thanks for the reviews guys. Anyone wants it in 94?
        Hide
        Elliott Clark added a comment -

        +1 lgtm

        Show
        Elliott Clark added a comment - +1 lgtm
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12576834/HBASE-7615.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12576834/HBASE-7615.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5116//console This message is automatically generated.
        Hide
        Elliott Clark added a comment -

        MetricMutableStat/MutableStat would have been what I used.

        On splits, yeah you are probably correct. I'll file a jira to change those.

        Show
        Elliott Clark added a comment - MetricMutableStat/MutableStat would have been what I used. On splits, yeah you are probably correct. I'll file a jira to change those.
        Hide
        Matteo Bertozzi added a comment -

        Elliott Clark what do you suggest instead? I just want to keep track of min/max/avg time of snapshots...

        As far as I know, we don't have a way to store metrics like [snapshot: time]
        so at the moment just having an idea of how long on avg it takes taking a snapshot or restoring is a step forward.

        I've used the histogram because is the same used by the log splitting, that is quite similar to the snapshot workload... do you think that also the log split metrics doesn't make sense?

        Show
        Matteo Bertozzi added a comment - Elliott Clark what do you suggest instead? I just want to keep track of min/max/avg time of snapshots... As far as I know, we don't have a way to store metrics like [snapshot: time] so at the moment just having an idea of how long on avg it takes taking a snapshot or restoring is a step forward. I've used the histogram because is the same used by the log splitting, that is quite similar to the snapshot workload... do you think that also the log split metrics doesn't make sense?
        Hide
        Elliott Clark added a comment -

        Why the histogram for snapshot time, clone time, and restore time? That comes with a pretty large overhead and I don't really understand what the user will gain by having the extra information of quantiles ?

        Show
        Elliott Clark added a comment - Why the histogram for snapshot time, clone time, and restore time? That comes with a pretty large overhead and I don't really understand what the user will gain by having the extra information of quantiles ?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12576676/HBASE-7615.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.master.TestMasterFailover

        -1 core zombie tests. There are 1 zombie test(s): at org.apache.hadoop.hbase.master.TestMasterFailover.testMasterFailoverWithMockedRIT(TestMasterFailover.java:382)

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12576676/HBASE-7615.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.TestMasterFailover -1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.master.TestMasterFailover.testMasterFailoverWithMockedRIT(TestMasterFailover.java:382) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5104//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        From https://builds.apache.org/job/PreCommit-HBASE-Build/5096/console , looks like compilation failed (on hadoop 2)

        Show
        Ted Yu added a comment - From https://builds.apache.org/job/PreCommit-HBASE-Build/5096/console , looks like compilation failed (on hadoop 2)
        Hide
        Ted Yu added a comment -

        There was no QA report.
        Mind attaching the patch one more time ?

        Thanks

        Show
        Ted Yu added a comment - There was no QA report. Mind attaching the patch one more time ? Thanks
        Hide
        Matteo Bertozzi added a comment -

        I'm planning to commit this one today, any other comments on the patch?

        Show
        Matteo Bertozzi added a comment - I'm planning to commit this one today, any other comments on the patch?
        Hide
        Ted Yu added a comment -

        nit:

        +        <td><%= stats.isSnapshotCorrupted() ? "CORRUPTED" : "ok" %></td>
        

        Consider showing the "CORRUPTED" text in a different color.

        +1 on patch, if tests pass.

        Show
        Ted Yu added a comment - nit: + <td><%= stats.isSnapshotCorrupted() ? "CORRUPTED" : "ok" %></td> Consider showing the "CORRUPTED" text in a different color. +1 on patch, if tests pass.
        Hide
        Matteo Bertozzi added a comment -

        v2 fixes the metricsMaster vs masterMetrics and this.metricsMasters vs metricsMaster inconsistencies

        Show
        Matteo Bertozzi added a comment - v2 fixes the metricsMaster vs masterMetrics and this.metricsMasters vs metricsMaster inconsistencies
        Hide
        Ted Yu added a comment -

        Went through the patch which looks good.

        +        new CloneSnapshotHandler(master, snapshot, hTableDescriptor, metricsMaster).prepare();
        

        nit: both metricsMaster and this.metricsMaster are used. It would be nice to be consistent.

        Hadoop QA is down again - may need to run test suite manually or later.

        Show
        Ted Yu added a comment - Went through the patch which looks good. + new CloneSnapshotHandler(master, snapshot, hTableDescriptor, metricsMaster).prepare(); nit: both metricsMaster and this.metricsMaster are used. It would be nice to be consistent. Hadoop QA is down again - may need to run test suite manually or later.
        Hide
        Matteo Bertozzi added a comment -

        Added snapshot, restore, clone time metrics

        Show
        Matteo Bertozzi added a comment - Added snapshot, restore, clone time metrics
        Hide
        Matteo Bertozzi added a comment -

        sure, the snapshot counters that Jesse mentioned should use metrics2

        but this is something calculated from the disk, unless you're keeping a map of snapshot/stats in memory and you don't check every N second for the corrupted state, you can't obtain this info.

        Show
        Matteo Bertozzi added a comment - sure, the snapshot counters that Jesse mentioned should use metrics2 but this is something calculated from the disk, unless you're keeping a map of snapshot/stats in memory and you don't check every N second for the corrupted state, you can't obtain this info.
        Hide
        Ted Yu added a comment -

        I thought stats would be surfaced through metrics2.

        +      return ((float)hfileSize / (hfileSize + hfileArchiveSize)) * 100;
        

        Do we need to check that hfileSize + hfileArchiveSize != 0 ?

        Show
        Ted Yu added a comment - I thought stats would be surfaced through metrics2. + return (( float )hfileSize / (hfileSize + hfileArchiveSize)) * 100; Do we need to check that hfileSize + hfileArchiveSize != 0 ?
        Hide
        Matteo Bertozzi added a comment -

        added the snapshot info (number of hfile, size and amount of shared data)

        Show
        Matteo Bertozzi added a comment - added the snapshot info (number of hfile, size and amount of shared data)

          People

          • Assignee:
            Matteo Bertozzi
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development