Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0, 1.1.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I had this patch laying around that somehow dropped from my plate. We should skip replaying compaction / flush and region open event markers if the files (from flush or compaction) can no longer be found from the secondary. If we do not skip, the replay will be retried forever, effectively blocking the replication further.

      Bulk load already does this, we just need to do it for flush / compaction and region open events as well.

      1. hbase-13515_v1.patch
        11 kB
        Enis Soztutar
      2. hbase-13515_v1.patch
        11 kB
        Enis Soztutar

        Activity

        Hide
        enis Enis Soztutar added a comment -

        Attaching straightforward patch.

        Show
        enis Enis Soztutar added a comment - Attaching straightforward patch.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726685/hbase-13515_v1.patch
        against master branch at commit eb82b8b3098d6a9ac62aa50189f9d4b289f38472.
        ATTACHMENT ID: 12726685

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0)

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//testReport/
        Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//artifact/patchprocess/newFindbugsWarnings.html
        Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//artifact/patchprocess/checkstyle-aggregate.html

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726685/hbase-13515_v1.patch against master branch at commit eb82b8b3098d6a9ac62aa50189f9d4b289f38472. ATTACHMENT ID: 12726685 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13746//console This message is automatically generated.
        Hide
        devaraj Devaraj Das added a comment -

        LGTM

        Show
        devaraj Devaraj Das added a comment - LGTM
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726758/hbase-13515_v1.patch
        against master branch at commit eb82b8b3098d6a9ac62aa50189f9d4b289f38472.
        ATTACHMENT ID: 12726758

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0)

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//testReport/
        Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//artifact/patchprocess/newFindbugsWarnings.html
        Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//artifact/patchprocess/checkstyle-aggregate.html

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726758/hbase-13515_v1.patch against master branch at commit eb82b8b3098d6a9ac62aa50189f9d4b289f38472. ATTACHMENT ID: 12726758 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13749//console This message is automatically generated.
        Hide
        enis Enis Soztutar added a comment -

        I have committed this. Thanks Devaraj for review.

        Show
        enis Enis Soztutar added a comment - I have committed this. Thanks Devaraj for review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in HBase-1.1 #416 (See https://builds.apache.org/job/HBase-1.1/416/)
        HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 48053f16053bc28273c71f7544bd27c4411512ea)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in HBase-1.1 #416 (See https://builds.apache.org/job/HBase-1.1/416/ ) HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 48053f16053bc28273c71f7544bd27c4411512ea) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #6396 (See https://builds.apache.org/job/HBase-TRUNK/6396/)
        HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 4e0de088c5f2fe13ea12219ffa4b85bdb7e26553)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #6396 (See https://builds.apache.org/job/HBase-TRUNK/6396/ ) HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 4e0de088c5f2fe13ea12219ffa4b85bdb7e26553) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in HBase-1.2 #11 (See https://builds.apache.org/job/HBase-1.2/11/)
        HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 0c6b1c93889bc00949fd776a5d091e66427bb1b2)

        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in HBase-1.2 #11 (See https://builds.apache.org/job/HBase-1.2/11/ ) HBASE-13515 Handle FileNotFoundException in region replica replay for flush/compaction events (enis: rev 0c6b1c93889bc00949fd776a5d091e66427bb1b2) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
        Hide
        ndimiduk Nick Dimiduk added a comment -

        Closing issues released in 1.1.0.

        Show
        ndimiduk Nick Dimiduk added a comment - Closing issues released in 1.1.0.

          People

          • Assignee:
            enis Enis Soztutar
            Reporter:
            enis Enis Soztutar
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development