HBase
  1. HBase
  2. HBASE-7930

hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Today when master and HBCK are reporting empty REGIONINFO_QUALIFIER .META. rows, we need to manually delete them from the .META. region. We need to enhance hbck to do that automatically.

      1. HBASE-7930-v2-trunk.patch
        10 kB
        Jean-Marc Spaggiari
      2. HBASE-7930-v1-trunk.patch
        10 kB
        Jean-Marc Spaggiari
      3. HBASE-7930-v0-trunk.patch
        4 kB
        Jean-Marc Spaggiari

        Activity

        Hide
        Dave Latham added a comment -

        Are there things that could generate this WARNing where the fix is not deleting the rows, but rather trying to repair them? For example, if there is still corresponding data on HDFS?

        Show
        Dave Latham added a comment - Are there things that could generate this WARNing where the fix is not deleting the rows, but rather trying to repair them? For example, if there is still corresponding data on HDFS?
        Hide
        Jean-Marc Spaggiari added a comment -

        hi Dave,

        This is the code generating this warning:

            byte [] bytes =
              result.getValue(HConstants.CATALOG_FAMILY, HConstants.REGIONINFO_QUALIFIER);
            if (bytes == null) {
              LOG.warn("REGIONINFO_QUALIFIER is empty in " + result);
              return null;
            }
        

        And it's the only place where this is printed into the logs. I "grepped" the code and was not able to find anywhere else where this is printed. So this warning is displayed only when we are not able to find the REGIONINFO_QUALIFIER in CATALOG_FAMILY the row.

        From what is remaining in the table:
        entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:server, timestamp=1361046940033, value=node1:60020
        entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:serverstartcode, timestamp=1361046940033, value=1361046923347
        We don't know where to look into HDFS. If we remove those 2 lines and HBCK found a region in the HDFS which is not into the .META., it will reconstruct the entries in the .META. based in the HDFS content.

        So I think we can safely remove those lines from the .META. and I don't think there is anything else which can generate this WARNing.

        Show
        Jean-Marc Spaggiari added a comment - hi Dave, This is the code generating this warning: byte [] bytes = result.getValue(HConstants.CATALOG_FAMILY, HConstants.REGIONINFO_QUALIFIER); if (bytes == null ) { LOG.warn( "REGIONINFO_QUALIFIER is empty in " + result); return null ; } And it's the only place where this is printed into the logs. I "grepped" the code and was not able to find anywhere else where this is printed. So this warning is displayed only when we are not able to find the REGIONINFO_QUALIFIER in CATALOG_FAMILY the row. From what is remaining in the table: entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:server, timestamp=1361046940033, value=node1:60020 entry,keyvalue,1360762149493.288611f36e195c950754ba88e9402950. column=info:serverstartcode, timestamp=1361046940033, value=1361046923347 We don't know where to look into HDFS. If we remove those 2 lines and HBCK found a region in the HDFS which is not into the .META., it will reconstruct the entries in the .META. based in the HDFS content. So I think we can safely remove those lines from the .META. and I don't think there is anything else which can generate this WARNing.
        Hide
        Ted Yu added a comment -

        Jimmy Xiang,Jonathan Hsieh:
        Can you take a look ?

        Show
        Ted Yu added a comment - Jimmy Xiang , Jonathan Hsieh : Can you take a look ?
        Hide
        Jean-Marc Spaggiari added a comment -

        I have not yet attached the patch.

        I build it and tested it successfuly on 0.94.5. I'm re-building because I change one line and I want to make sure it's still good. I should be able to submit it in the next 2h. The patch from trunk will got in the same timeframe.

        Show
        Jean-Marc Spaggiari added a comment - I have not yet attached the patch. I build it and tested it successfuly on 0.94.5. I'm re-building because I change one line and I want to make sure it's still good. I should be able to submit it in the next 2h. The patch from trunk will got in the same timeframe.
        Hide
        Jean-Marc Spaggiari added a comment -

        Patch for trunk attached. Also fixing one FindBug report. Will post patch for 0.94 when Hadoop QA will be done. I have tested 0.94 against my cluster. I had 6 empty entries in the META. All got removed. Have recreated on manually and retried, passed again.

        Show
        Jean-Marc Spaggiari added a comment - Patch for trunk attached. Also fixing one FindBug report. Will post patch for 0.94 when Hadoop QA will be done. I have tested 0.94 against my cluster. I had 6 empty entries in the META. All got removed. Have recreated on manually and retried, passed again.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12570865/HBASE-7930-v0-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.util.TestHBaseFsck

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570865/HBASE-7930-v0-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.util.TestHBaseFsck Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4540//console This message is automatically generated.
        Hide
        Jean-Marc Spaggiari added a comment -

        I removed the hash code which is failing. I will look at it later most probably in another JIRA.

        Show
        Jean-Marc Spaggiari added a comment - I removed the hash code which is failing. I will look at it later most probably in another JIRA.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12570884/HBASE-7930-v0-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570884/HBASE-7930-v0-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4541//console This message is automatically generated.
        Hide
        Jonathan Hsieh added a comment - - edited

        Hey, Jean-Marc Spaggiari patch looks reasonable – can you add a simple test case for this fix? There are plenty of examples in TestHBaseFsck and this should be relatively easy to include.

        Show
        Jonathan Hsieh added a comment - - edited Hey, Jean-Marc Spaggiari patch looks reasonable – can you add a simple test case for this fix? There are plenty of examples in TestHBaseFsck and this should be relatively easy to include.
        Hide
        Jean-Marc Spaggiari added a comment -

        Thanks for the review Jonathan Hsieh. I will add a test a re-submit the patch.

        Show
        Jean-Marc Spaggiari added a comment - Thanks for the review Jonathan Hsieh . I will add a test a re-submit the patch.
        Hide
        Jean-Marc Spaggiari added a comment -

        Tests now included in the patch.

        Show
        Jean-Marc Spaggiari added a comment - Tests now included in the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12571750/HBASE-7930-v1-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.util.TestHBaseFsck
        org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571750/HBASE-7930-v1-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.util.TestHBaseFsck org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4637//console This message is automatically generated.
        Hide
        Jean-Marc Spaggiari added a comment -

        Test is now working fine locally. Let's see if Hadoop QA agrees

        Show
        Jean-Marc Spaggiari added a comment - Test is now working fine locally. Let's see if Hadoop QA agrees
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12571779/HBASE-7930-v2-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571779/HBASE-7930-v2-trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4641//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        Jimmy Xiang:
        Can you take a look at Jean-Marc's patch ?

        Thanks

        Show
        Ted Yu added a comment - Jimmy Xiang : Can you take a look at Jean-Marc's patch ? Thanks
        Hide
        Jimmy Xiang added a comment -

        For the test, can we use the utility assertErrors() and assertNoErrors() in TestHBaseFsck? By this way, we will be sure the list of expected errors.

        + assertTrue(hbck.getErrors().getErrorList().contains(ERROR_CODE.EMPTY_META_CELL));

        + assertFalse(hbck.getErrors().getErrorList().contains(ERROR_CODE.EMPTY_META_CELL));

        Should we also put this fix in OfflineMetaRepair?

        Show
        Jimmy Xiang added a comment - For the test, can we use the utility assertErrors() and assertNoErrors() in TestHBaseFsck? By this way, we will be sure the list of expected errors. + assertTrue(hbck.getErrors().getErrorList().contains(ERROR_CODE.EMPTY_META_CELL)); + assertFalse(hbck.getErrors().getErrorList().contains(ERROR_CODE.EMPTY_META_CELL)); Should we also put this fix in OfflineMetaRepair?
        Hide
        Jean-Marc Spaggiari added a comment -

        First thing, thanks for looking at it!

        For the test, can we use the utility assertErrors() and assertNoErrors() in TestHBaseFsck? By this way, we will be sure the list of expected errors.

        Unfortunatly we can't. asserErrors() verifies that ALL the errors you are giving it are there, and only those errors. However here, there is some other errors triggered because of the entries removed. I used asserTrue to focus on the specific ERROR_CODE.EMPTY_META_CELL and not be "polluted" by the other errors. I can still build the full list of errors triggered if required, and user assertErrors, but that will also validate some other errors which are not related to this specific test case.

        Should we also put this fix in OfflineMetaRepair?

        I don't think we need. OfflineMetaRepair will call rebuildMeta which will call sidelineOldMeta and createNewMeta. That will backup the meta and create a proper one, which will not have any empty REGIONINFO_QUALIFIER.

        Show
        Jean-Marc Spaggiari added a comment - First thing, thanks for looking at it! For the test, can we use the utility assertErrors() and assertNoErrors() in TestHBaseFsck? By this way, we will be sure the list of expected errors. Unfortunatly we can't. asserErrors() verifies that ALL the errors you are giving it are there, and only those errors. However here, there is some other errors triggered because of the entries removed. I used asserTrue to focus on the specific ERROR_CODE.EMPTY_META_CELL and not be "polluted" by the other errors. I can still build the full list of errors triggered if required, and user assertErrors, but that will also validate some other errors which are not related to this specific test case. Should we also put this fix in OfflineMetaRepair? I don't think we need. OfflineMetaRepair will call rebuildMeta which will call sidelineOldMeta and createNewMeta. That will backup the meta and create a proper one, which will not have any empty REGIONINFO_QUALIFIER.
        Hide
        Jimmy Xiang added a comment -

        I see. +1

        Show
        Jimmy Xiang added a comment - I see. +1
        Hide
        Ted Yu added a comment -

        Ran the test with patch v2 locally and it passed:

        Running org.apache.hadoop.hbase.util.TestHBaseFsck
        2013-03-15 13:38:10.148 java[68610:1203] Unable to load realm info from SCDynamicStore
        Tests run: 35, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 212.76 sec
        

        Integrated to 0.95 and trunk.

        Thanks for the patch, Jean-Marc.

        Thanks for the review, Jimmy.

        Show
        Ted Yu added a comment - Ran the test with patch v2 locally and it passed: Running org.apache.hadoop.hbase.util.TestHBaseFsck 2013-03-15 13:38:10.148 java[68610:1203] Unable to load realm info from SCDynamicStore Tests run: 35, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 212.76 sec Integrated to 0.95 and trunk. Thanks for the patch, Jean-Marc. Thanks for the review, Jimmy.
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #77 (See https://builds.apache.org/job/hbase-0.95/77/)
        HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457102)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #77 (See https://builds.apache.org/job/hbase-0.95/77/ ) HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457102) Result = FAILURE tedyu : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #3963 (See https://builds.apache.org/job/HBase-TRUNK/3963/)
        HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457103)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #3963 (See https://builds.apache.org/job/HBase-TRUNK/3963/ ) HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457103) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #449 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/449/)
        HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457103)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #449 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/449/ ) HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457103) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #28 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/28/)
        HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457102)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
        • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #28 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/28/ ) HBASE-7930 hbck should provide an option to fix .META. rows with empty REGIONINFO_QUALIFIER (Jean-Marc) (Revision 1457102) Result = FAILURE tedyu : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
        Hide
        stack added a comment -

        This was integrated a while back.

        Show
        stack added a comment - This was integrated a while back.
        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.

          People

          • Assignee:
            Jean-Marc Spaggiari
            Reporter:
            Jean-Marc Spaggiari
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development