Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.4
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Adds catalogjanitor_enabled, catalogjanitor_run, and catalogjanitor_switch to the shell.

      Description

      When doing surgery or other operational tasks, it's nice to be able to have the .META. table quickly cleaned of split parents. The CatalogJanitor already has controls baked in (currently used in unit tests), I think we should expose this the same way we do with the balancer, that is:

      • start
      • stop
      • request a run

      A client would need to go through HBaseAdmin, and shell commands need to be created.

      1. 4255-5.1.patch
        140 kB
        Devaraj Das
      2. 4255-4.2.patch
        140 kB
        Ted Yu

        Activity

        Hide
        stack stack added a comment -

        +1

        Show
        stack stack added a comment - +1
        Hide
        devaraj Devaraj Das added a comment -
        Show
        devaraj Devaraj Das added a comment - A patch up on https://reviews.apache.org/r/5993/
        Hide
        zhihyu@ebaysf.com Ted Yu added a comment -

        Patch from review board.

        Show
        zhihyu@ebaysf.com Ted Yu added a comment - Patch from review board.
        Hide
        zhihyu@ebaysf.com Ted Yu added a comment -

        @J-D:
        Please take a look at Deravaj's patch.

        Show
        zhihyu@ebaysf.com Ted Yu added a comment - @J-D: Please take a look at Deravaj's patch.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12536980/4255-4.2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536980/4255-4.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//console This message is automatically generated.
        Hide
        stack stack added a comment -

        I added feedback up on rb

        Show
        stack stack added a comment - I added feedback up on rb
        Hide
        devaraj Devaraj Das added a comment -

        This is the patch from RB.

        Show
        devaraj Devaraj Das added a comment - This is the patch from RB.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537036/4255-5.1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537036/4255-5.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//console This message is automatically generated.
        Hide
        zhihyu@ebaysf.com Ted Yu added a comment -

        No test failure from https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/:

        Results :
        
        Tests run: 1021, Failures: 0, Errors: 0, Skipped: 9
        
        Show
        zhihyu@ebaysf.com Ted Yu added a comment - No test failure from https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/: Results : Tests run: 1021, Failures: 0, Errors: 0, Skipped: 9
        Hide
        stack stack added a comment -

        I'll commit in next day unless objection.

        Show
        stack stack added a comment - I'll commit in next day unless objection.
        Hide
        stack stack added a comment -

        I tried the shell commands locally and seems to work.

        Committed to trunk. Thanks for the patch Devaraj.

        Show
        stack stack added a comment - I tried the shell commands locally and seems to work. Committed to trunk. Thanks for the patch Devaraj.
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK #3157 (See https://builds.apache.org/job/HBase-TRUNK/3157/)
        HBASE-4255 Expose CatalogJanitor controls (Revision 1364127)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java
        • /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto
        • /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK #3157 (See https://builds.apache.org/job/HBase-TRUNK/3157/ ) HBASE-4255 Expose CatalogJanitor controls (Revision 1364127) Result = SUCCESS stack : Files : /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb /hbase/trunk/hbase-server/src/main/ruby/shell.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #102 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/102/)
        HBASE-4255 Expose CatalogJanitor controls (Revision 1364127)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java
        • /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto
        • /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #102 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/102/ ) HBASE-4255 Expose CatalogJanitor controls (Revision 1364127) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb /hbase/trunk/hbase-server/src/main/ruby/shell.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Hide
        stack stack added a comment -

        Marking closed.

        Show
        stack stack added a comment - Marking closed.

          People

          • Assignee:
            devaraj Devaraj Das
            Reporter:
            jdcryans Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development