HBase
  1. HBase
  2. HBASE-4563

When error occurs in this.parent.close(false) of split, the split region cannot write or read

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.90.4, 0.92.0
    • Fix Version/s: 0.90.5
    • Component/s: regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Follow below steps to replay the problem:
      1. change the SplitTransaction.java as below,just like mock the hdfs error.

      SplitTransaction.java
            List<StoreFile> hstoreFilesToSplit = this.parent.close(false);
            throw new IOException("some unexpected error in close store files");
         

      2. update the regionserver code,restart;
      3. create a table & put some data to the table;
      4. split the table;
      5. scan the table,then it'll fail.

      We can fix the bug just use the patch.

      1. test-4563-trunk.txt
        31 kB
        bluedavy
      2. test-4563-0.92.txt
        30 kB
        bluedavy
      3. test-4563-0.90.txt
        21 kB
        bluedavy
      4. HBASE-4563-trunk.patch
        2 kB
        bluedavy
      5. HBASE-4563-0.92.patch
        2 kB
        bluedavy
      6. HBASE-4563-0.90.patch
        2 kB
        bluedavy

        Activity

        Hide
        bluedavy added a comment -

        patch & test report.

        Show
        bluedavy added a comment - patch & test report.
        Hide
        ramkrishna.s.vasudevan added a comment -

        Thanks for the patch.

        +	catch(IOException e){
        +	  throw e;
        +	}
        
        

        Why should we catch and again throw exception?

        Show
        ramkrishna.s.vasudevan added a comment - Thanks for the patch. + catch (IOException e){ + throw e; + } Why should we catch and again throw exception?
        Hide
        ramkrishna.s.vasudevan added a comment -

        Is it possible to write a testcase with the scenario you have told in the description?

        Minor comment.

        +	}
        +	catch(IOException e){
        +	  throw e;
        +	}
        +	finally{
        

        Catch block, finally block should start in the same line like
        + } catch(IOException e)

        { + throw e; + }

        finally{

        Show
        ramkrishna.s.vasudevan added a comment - Is it possible to write a testcase with the scenario you have told in the description? Minor comment. + } + catch (IOException e){ + throw e; + } + finally { Catch block, finally block should start in the same line like + } catch(IOException e) { + throw e; + } finally{
        Hide
        bluedavy added a comment -

        throw exception so CompactSplitThread can catch it then do rollback...

        Show
        bluedavy added a comment - throw exception so CompactSplitThread can catch it then do rollback...
        Hide
        Ted Yu added a comment -

        Ramkrishna's comment @ 10/Oct/11 10:35 was saying that the IOException caught didn't get transformed into another exception type.
        So the catch block isn't needed.

        Show
        Ted Yu added a comment - Ramkrishna's comment @ 10/Oct/11 10:35 was saying that the IOException caught didn't get transformed into another exception type. So the catch block isn't needed.
        Hide
        bluedavy added a comment -

        @Ted Yu
        Yes,,I'll change it & attach the patch again.

        Show
        bluedavy added a comment - @Ted Yu Yes, ,I'll change it & attach the patch again.
        Hide
        ramkrishna.s.vasudevan added a comment -
        +      }
        +	} finally {
        +      this.journal.add(JournalEntry.CLOSED_PARENT_REGION);
        

        Formatting issue is there in the latest patch

        HBASE-4563fortrunk.patch and HBASE-4563for0.92.patch. If you submit for trunk the same can be taken for 0.92. I will leave it to others for commenting before we can commit.

        Show
        ramkrishna.s.vasudevan added a comment - + } + } finally { + this .journal.add(JournalEntry.CLOSED_PARENT_REGION); Formatting issue is there in the latest patch HBASE-4563 fortrunk.patch and HBASE-4563 for0.92.patch. If you submit for trunk the same can be taken for 0.92. I will leave it to others for commenting before we can commit.
        Hide
        Ted Yu added a comment -

        Please run through unit tests for 0.92 patch.

        Show
        Ted Yu added a comment - Please run through unit tests for 0.92 patch.
        Hide
        Ted Yu added a comment -

        Thanks for the detailed test report.
        +1 on patches.
        There is some formatting:

        +	} finally {
        +      this.journal.add(JournalEntry.CLOSED_PARENT_REGION);
        +	}
        

        The right curly brace before finally should be moved left by 3 spaces.
        Same with the closing right curly brace.

        See HBASE-3678 for Formatter for Eclipse.

        Show
        Ted Yu added a comment - Thanks for the detailed test report. +1 on patches. There is some formatting: + } finally { + this .journal.add(JournalEntry.CLOSED_PARENT_REGION); + } The right curly brace before finally should be moved left by 3 spaces. Same with the closing right curly brace. See HBASE-3678 for Formatter for Eclipse.
        Hide
        Lars Hofhansl added a comment -

        +1 on patches (pending all tests pass and Ted's suggested formatting fix).

        Show
        Lars Hofhansl added a comment - +1 on patches (pending all tests pass and Ted's suggested formatting fix).
        Hide
        bluedavy added a comment -

        I fix the formatter.

        Show
        bluedavy added a comment - I fix the formatter.
        Hide
        bluedavy added a comment -

        wait for committer commit to the svn.

        Show
        bluedavy added a comment - wait for committer commit to the svn.
        Hide
        Lars Hofhansl added a comment -

        @Ted... You wanna commit, or should I? I'm happy to.

        Show
        Lars Hofhansl added a comment - @Ted... You wanna commit, or should I? I'm happy to.
        Hide
        Ted Yu added a comment -

        @Lars:
        Go ahead.

        Show
        Ted Yu added a comment - @Lars: Go ahead.
        Hide
        Lars Hofhansl added a comment -

        Committed to 0.90, 0.92, and trunk

        Show
        Lars Hofhansl added a comment - Committed to 0.90, 0.92, and trunk
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2329 (See https://builds.apache.org/job/HBase-TRUNK/2329/)
        HBASE-4563 When error occurs in this.parent.close(false) of split, the split region cannot write or read

        larsh :
        Files :

        • /hbase/trunk/CHANGES.txt
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2329 (See https://builds.apache.org/job/HBase-TRUNK/2329/ ) HBASE-4563 When error occurs in this.parent.close(false) of split, the split region cannot write or read larsh : Files : /hbase/trunk/CHANGES.txt /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #66 (See https://builds.apache.org/job/HBase-0.92/66/)
        HBASE-4563 When error occurs in this.parent.close(false) of split, the split region cannot write or read

        larsh :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #66 (See https://builds.apache.org/job/HBase-0.92/66/ ) HBASE-4563 When error occurs in this.parent.close(false) of split, the split region cannot write or read larsh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java

          People

          • Assignee:
            bluedavy
            Reporter:
            bluedavy
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development