Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.4
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Adds catalogjanitor_enabled, catalogjanitor_run, and catalogjanitor_switch to the shell.

      Description

      When doing surgery or other operational tasks, it's nice to be able to have the .META. table quickly cleaned of split parents. The CatalogJanitor already has controls baked in (currently used in unit tests), I think we should expose this the same way we do with the balancer, that is:

      • start
      • stop
      • request a run

      A client would need to go through HBaseAdmin, and shell commands need to be created.

      1. 4255-4.2.patch
        140 kB
        Ted Yu
      2. 4255-5.1.patch
        140 kB
        Devaraj Das

        Activity

        Hide
        stack added a comment -

        +1

        Show
        stack added a comment - +1
        Hide
        Devaraj Das added a comment -
        Show
        Devaraj Das added a comment - A patch up on https://reviews.apache.org/r/5993/
        Hide
        Ted Yu added a comment -

        Patch from review board.

        Show
        Ted Yu added a comment - Patch from review board.
        Hide
        Ted Yu added a comment -

        @J-D:
        Please take a look at Deravaj's patch.

        Show
        Ted Yu added a comment - @J-D: Please take a look at Deravaj's patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12536980/4255-4.2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536980/4255-4.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2397//console This message is automatically generated.
        Hide
        stack added a comment -

        I added feedback up on rb

        Show
        stack added a comment - I added feedback up on rb
        Hide
        Devaraj Das added a comment -

        This is the patch from RB.

        Show
        Devaraj Das added a comment - This is the patch from RB.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537036/4255-5.1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537036/4255-5.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2400//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        No test failure from https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/:

        Results :
        
        Tests run: 1021, Failures: 0, Errors: 0, Skipped: 9
        
        Show
        Ted Yu added a comment - No test failure from https://builds.apache.org/job/PreCommit-HBASE-Build/2400//testReport/: Results : Tests run: 1021, Failures: 0, Errors: 0, Skipped: 9
        Hide
        stack added a comment -

        I'll commit in next day unless objection.

        Show
        stack added a comment - I'll commit in next day unless objection.
        Hide
        stack added a comment -

        I tried the shell commands locally and seems to work.

        Committed to trunk. Thanks for the patch Devaraj.

        Show
        stack added a comment - I tried the shell commands locally and seems to work. Committed to trunk. Thanks for the patch Devaraj.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #3157 (See https://builds.apache.org/job/HBase-TRUNK/3157/)
        HBASE-4255 Expose CatalogJanitor controls (Revision 1364127)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java
        • /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto
        • /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #3157 (See https://builds.apache.org/job/HBase-TRUNK/3157/ ) HBASE-4255 Expose CatalogJanitor controls (Revision 1364127) Result = SUCCESS stack : Files : /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb /hbase/trunk/hbase-server/src/main/ruby/shell.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #102 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/102/)
        HBASE-4255 Expose CatalogJanitor controls (Revision 1364127)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java
        • /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto
        • /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb
        • /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #102 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/102/ ) HBASE-4255 Expose CatalogJanitor controls (Revision 1364127) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/MasterAdminProtocol.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java /hbase/trunk/hbase-server/src/main/protobuf/MasterAdmin.proto /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb /hbase/trunk/hbase-server/src/main/ruby/shell.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_enabled.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_run.rb /hbase/trunk/hbase-server/src/main/ruby/shell/commands/catalogjanitor_switch.rb
        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development