Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15827

Backport HBASE-15125 'HBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary' to 1.2

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.2.2
    • None
    • None
    • Reviewed

    Description

      There is a bug in HBaseFsck's adoptHdfsOrphan function.At the last of this function will create a region,which want to cover all the orphan regions.But the end key of this new region was set incorrectly.Correct region's boundary should be [startKey,endKey),but this function create a region with boundary of [startKey,endKey],this bug will leads to scan operation omit some data.
      I think we should create the region like bellow,

          // create new region on hdfs. move data into place.
          HRegionInfo hri = new HRegionInfo(template.getTableName(), orphanRegionRange.getFirst(),
              Bytes.add(orphanRegionRange.getSecond(), new byte[1]));
      

      This is to backport to branch-1.2

      Attachments

        Issue Links

          Activity

            yuzhihong@gmail.com Ted Yu added a comment -

            TestHBaseFsck passed with the patch on branch-1.2

            yuzhihong@gmail.com Ted Yu added a comment - TestHBaseFsck passed with the patch on branch-1.2

            +1 LGTM

            syuanjiang Stephen Yuan Jiang added a comment - +1 LGTM
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            -1 patch 0m 4s HBASE-15827 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for help.



            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 4s HBASE-15827 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803968/15827-branch-1.2.patch JIRA Issue HBASE-15827 Console output https://builds.apache.org/job/PreCommit-HBASE-Build/1895/console Powered by Apache Yetus 0.2.1 http://yetus.apache.org This message was automatically generated.
            yuzhihong@gmail.com Ted Yu added a comment -

            Got some test failures on the backport to branch-1.1 :

            testRegionHole(org.apache.hadoop.hbase.util.TestHBaseFsck)  Time elapsed: 5.866 sec  <<< FAILURE!
            java.lang.AssertionError: expected:<[HOLE_IN_REGION_CHAIN]> but was:<[HOLE_IN_REGION_CHAIN, EXPIRED_TABLE_LOCK]>
            	at org.junit.Assert.fail(Assert.java:88)
            	at org.junit.Assert.failNotEquals(Assert.java:834)
            	at org.junit.Assert.assertEquals(Assert.java:118)
            	at org.junit.Assert.assertEquals(Assert.java:144)
            	at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99)
            	at org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionHole(TestHBaseFsck.java:1189)
            
            testNotInMetaOrDeployedHole(org.apache.hadoop.hbase.util.TestHBaseFsck)  Time elapsed: 5.877 sec  <<< FAILURE!
            java.lang.AssertionError: expected:<[NOT_IN_META_OR_DEPLOYED, HOLE_IN_REGION_CHAIN]> but was:<[NOT_IN_META_OR_DEPLOYED, HOLE_IN_REGION_CHAIN, EXPIRED_TABLE_LOCK]>
            	at org.junit.Assert.fail(Assert.java:88)
            	at org.junit.Assert.failNotEquals(Assert.java:834)
            	at org.junit.Assert.assertEquals(Assert.java:118)
            	at org.junit.Assert.assertEquals(Assert.java:144)
            	at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99)
            	at org.apache.hadoop.hbase.util.TestHBaseFsck.testNotInMetaOrDeployedHole(TestHBaseFsck.java:1307)
            
            testMissingFirstRegion(org.apache.hadoop.hbase.util.TestHBaseFsck)  Time elapsed: 5.869 sec  <<< FAILURE!
            java.lang.AssertionError: expected:<[FIRST_REGION_STARTKEY_NOT_EMPTY]> but was:<[FIRST_REGION_STARTKEY_NOT_EMPTY, EXPIRED_TABLE_LOCK]>
            	at org.junit.Assert.fail(Assert.java:88)
            	at org.junit.Assert.failNotEquals(Assert.java:834)
            	at org.junit.Assert.assertEquals(Assert.java:118)
            	at org.junit.Assert.assertEquals(Assert.java:144)
            	at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99)
            	at org.apache.hadoop.hbase.util.TestHBaseFsck.testMissingFirstRegion(TestHBaseFsck.java:1867)
            

            Will investigate later.

            yuzhihong@gmail.com Ted Yu added a comment - Got some test failures on the backport to branch-1.1 : testRegionHole(org.apache.hadoop.hbase.util.TestHBaseFsck) Time elapsed: 5.866 sec <<< FAILURE! java.lang.AssertionError: expected:<[HOLE_IN_REGION_CHAIN]> but was:<[HOLE_IN_REGION_CHAIN, EXPIRED_TABLE_LOCK]> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99) at org.apache.hadoop.hbase.util.TestHBaseFsck.testRegionHole(TestHBaseFsck.java:1189) testNotInMetaOrDeployedHole(org.apache.hadoop.hbase.util.TestHBaseFsck) Time elapsed: 5.877 sec <<< FAILURE! java.lang.AssertionError: expected:<[NOT_IN_META_OR_DEPLOYED, HOLE_IN_REGION_CHAIN]> but was:<[NOT_IN_META_OR_DEPLOYED, HOLE_IN_REGION_CHAIN, EXPIRED_TABLE_LOCK]> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99) at org.apache.hadoop.hbase.util.TestHBaseFsck.testNotInMetaOrDeployedHole(TestHBaseFsck.java:1307) testMissingFirstRegion(org.apache.hadoop.hbase.util.TestHBaseFsck) Time elapsed: 5.869 sec <<< FAILURE! java.lang.AssertionError: expected:<[FIRST_REGION_STARTKEY_NOT_EMPTY]> but was:<[FIRST_REGION_STARTKEY_NOT_EMPTY, EXPIRED_TABLE_LOCK]> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.assertErrors(HbckTestingUtil.java:99) at org.apache.hadoop.hbase.util.TestHBaseFsck.testMissingFirstRegion(TestHBaseFsck.java:1867) Will investigate later.
            elserj Josh Elser added a comment -

            Thanks for being proactive on the backport, syuanjiang and tedyu! I just went to look if someone had did this (after Stephen's comment on HBASE-15125) and was happy to see you all already have this in the works

            elserj Josh Elser added a comment - Thanks for being proactive on the backport, syuanjiang and tedyu ! I just went to look if someone had did this (after Stephen's comment on HBASE-15125 ) and was happy to see you all already have this in the works
            busbey Sean Busbey added a comment -

            this went into branch-1.2 in commit f778736b658b28b24ce7bb7428ab3b268d742765

            busbey Sean Busbey added a comment - this went into branch-1.2 in commit f778736b658b28b24ce7bb7428ab3b268d742765

            People

              yuzhihong@gmail.com Ted Yu
              yuzhihong@gmail.com Ted Yu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: