Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0, 1.1.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As mentioned in parent issue, and design docs for phase-1 (HBASE-10070) and Phase-2 (HBASE-11183), implement asynchronous WAL replication from the WAL files of the primary region to the secondary region replicas.

      The WAL replication will build upon the pluggable replication framework introduced in HBASE-11367, and the distributed WAL replay.

      Upon having some experience with the patch, we changed the design so that there is only one replication queue for doing the async wal replication to secondary replicas rather than having a queue per region replica. This is due to the fact that, we do not want to tail the logs of every region server for a single region replica.

      Handling of flushes/compactions and memstore accounting will be handled in other subtasks.

      1. hbase-11568_v2.patch
        105 kB
        Enis Soztutar
      2. hbase-11568_v3.patch
        106 kB
        Enis Soztutar
      3. 11568-branch-1.txt
        109 kB
        Devaraj Das
      4. 11568-2-branch-1.txt
        109 kB
        Devaraj Das

        Issue Links

          Activity

          Enis Soztutar created issue -
          Enis Soztutar made changes -
          Field Original Value New Value
          Remote Link This issue links to "review (Web Link)" [ 16237 ]
          Hide
          Enis Soztutar added a comment -

          I have a patch which I attached to RB. I've tested this smaller setups and it seems that it does the job. It builds upon the replication and log replay features that we already have.

          We would need the flush handling patch to be close to final to test this on a larger scale, but it should be fine without it, because any issues we find can be handled in follow up jiras as well.

          Show
          Enis Soztutar added a comment - I have a patch which I attached to RB. I've tested this smaller setups and it seems that it does the job. It builds upon the replication and log replay features that we already have. We would need the flush handling patch to be close to final to test this on a larger scale, but it should be fine without it, because any issues we find can be handled in follow up jiras as well.
          Hide
          Enis Soztutar added a comment -

          Let's try the v2 patch from RB against hadoopqa.

          Show
          Enis Soztutar added a comment - Let's try the v2 patch from RB against hadoopqa.
          Enis Soztutar made changes -
          Attachment hbase-11568_v2.patch [ 12662166 ]
          Enis Soztutar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12662166/hbase-11568_v2.patch
          against trunk revision .
          ATTACHMENT ID: 12662166

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12662166/hbase-11568_v2.patch against trunk revision . ATTACHMENT ID: 12662166 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10465//console This message is automatically generated.
          Hide
          Enis Soztutar added a comment -

          v3 patch from RB.

          Show
          Enis Soztutar added a comment - v3 patch from RB.
          Enis Soztutar made changes -
          Attachment hbase-11568_v3.patch [ 12662580 ]
          Hide
          Jeffrey Zhong added a comment -

          v3 patch addressed my previous comments & looks good to me(+1). Thanks.

          Show
          Jeffrey Zhong added a comment - v3 patch addressed my previous comments & looks good to me(+1). Thanks.
          Hide
          Enis Soztutar added a comment -

          I've committed this to master. Thanks Jeffrey.

          Show
          Enis Soztutar added a comment - I've committed this to master. Thanks Jeffrey.
          Enis Soztutar made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #5413 (See https://builds.apache.org/job/HBase-TRUNK/5413/)
          HBASE-11568 Async WAL replication for region replicas (enis: rev e28ec72464ccb266c49e5ef42dd87f938e71ffde)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
          • hbase-common/src/main/resources/hbase-default.xml
          • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5413 (See https://builds.apache.org/job/HBase-TRUNK/5413/ ) HBASE-11568 Async WAL replication for region replicas (enis: rev e28ec72464ccb266c49e5ef42dd87f938e71ffde) hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java hbase-common/src/main/resources/hbase-default.xml hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          Hide
          Devaraj Das added a comment -

          Reopening for branch-1 commit

          Show
          Devaraj Das added a comment - Reopening for branch-1 commit
          Devaraj Das made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Devaraj Das added a comment -

          Rebased patch for branch-1.

          Show
          Devaraj Das added a comment - Rebased patch for branch-1.
          Devaraj Das made changes -
          Attachment 11568-branch-1.txt [ 12696643 ]
          Devaraj Das made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Devaraj Das made changes -
          Fix Version/s 1.1.0 [ 12329043 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12696643/11568-branch-1.txt
          against branch-1 branch at commit 8aeb3acaf959e2905191fd6c92fa56300f7d3597.
          ATTACHMENT ID: 12696643

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 12 warning messages.

          -1 checkstyle. The applied patch generated 3785 checkstyle errors (more than the master's current 3764 errors).

          -1 findbugs. The patch appears to introduce 4 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/checkstyle-aggregate.html

          Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/patchJavadocWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12696643/11568-branch-1.txt against branch-1 branch at commit 8aeb3acaf959e2905191fd6c92fa56300f7d3597. ATTACHMENT ID: 12696643 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 12 warning messages. -1 checkstyle . The applied patch generated 3785 checkstyle errors (more than the master's current 3764 errors). -1 findbugs . The patch appears to introduce 4 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12703//console This message is automatically generated.
          Hide
          Devaraj Das added a comment -

          Fixes the UT failure.

          Show
          Devaraj Das added a comment - Fixes the UT failure.
          Devaraj Das made changes -
          Attachment 11568-2-branch-1.txt [ 12696838 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12696838/11568-2-branch-1.txt
          against branch-1 branch at commit 96cdc7987e8894b304a3201f67cb0b9595c68cc3.
          ATTACHMENT ID: 12696838

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 12 warning messages.

          -1 checkstyle. The applied patch generated 3789 checkstyle errors (more than the master's current 3768 errors).

          -1 findbugs. The patch appears to introduce 4 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          -1 core zombie tests. There are 1 zombie test(s): at org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache.testCacheMultiThreadedEviction(TestSingleSizeCache.java:73)

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/checkstyle-aggregate.html

          Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/patchJavadocWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12696838/11568-2-branch-1.txt against branch-1 branch at commit 96cdc7987e8894b304a3201f67cb0b9595c68cc3. ATTACHMENT ID: 12696838 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 12 warning messages. -1 checkstyle . The applied patch generated 3789 checkstyle errors (more than the master's current 3768 errors). -1 findbugs . The patch appears to introduce 4 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . -1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache.testCacheMultiThreadedEviction(TestSingleSizeCache.java:73) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12710//console This message is automatically generated.
          Hide
          Enis Soztutar added a comment -

          +1. I checked the patch against the master patch and current code.
          Two nits we can address on commit.

          • RegionReplicaReplicationEndpoint.java javadoc refers to now-gone HLogSplitter. This coming from the old patch, we can replace that with WALSplitter. Also the InterfaceAudience should be the one in the hbase package, not hadoop package.
          • in LogReplayOutputSink.flush() method, dataAvailable.notifyAll() is wrapped with the sync block. It is not needed since all callers for the flush() method already has the monitor, but we can still have that as an extra guarantee, since it is not documented well. master code already has this (I think added after the patch), so we can keep that sync block in the branch-1 as well.
                   synchronized(dataAvailable) {
            -          dataAvailable.notifyAll();
            -        }
            +        controller.dataAvailable.notifyAll();
            
          Show
          Enis Soztutar added a comment - +1. I checked the patch against the master patch and current code. Two nits we can address on commit. RegionReplicaReplicationEndpoint.java javadoc refers to now-gone HLogSplitter. This coming from the old patch, we can replace that with WALSplitter. Also the InterfaceAudience should be the one in the hbase package, not hadoop package. in LogReplayOutputSink.flush() method, dataAvailable.notifyAll() is wrapped with the sync block. It is not needed since all callers for the flush() method already has the monitor, but we can still have that as an extra guarantee, since it is not documented well. master code already has this (I think added after the patch), so we can keep that sync block in the branch-1 as well. synchronized (dataAvailable) { - dataAvailable.notifyAll(); - } + controller.dataAvailable.notifyAll();
          Hide
          Devaraj Das added a comment -

          Committed to branch-1 as well.

          Show
          Devaraj Das added a comment - Committed to branch-1 as well.
          Devaraj Das made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-1.1 #149 (See https://builds.apache.org/job/HBase-1.1/149/)
          HBASE-11568. Async WAL replication for region replicas (Enis Soztutar) (ddas: rev 055f5a95e46e55412e05892f86fc0e0e2792356c)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALMethods.java
          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java
          • hbase-common/src/main/resources/hbase-default.xml
          • hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
          • hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-1.1 #149 (See https://builds.apache.org/job/HBase-1.1/149/ ) HBASE-11568 . Async WAL replication for region replicas (Enis Soztutar) (ddas: rev 055f5a95e46e55412e05892f86fc0e0e2792356c) hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerNoMaster.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RegionReplicaReplicationEndpoint.java hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALMethods.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/ReplicationProtbufUtil.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpoint.java hbase-common/src/main/resources/hbase-default.xml hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java
          Hide
          stack added a comment -
          Show
          stack added a comment - Broke the branch-1 build? https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.1/149/
          Hide
          Devaraj Das added a comment -

          stack, I fixed it a few minutes back.

          Show
          Devaraj Das added a comment - stack , I fixed it a few minutes back.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-1.1 #150 (See https://builds.apache.org/job/HBase-1.1/150/)
          HBASE-11568. Addendum to add a file that I missed earlier. (ddas: rev 78c50af3ec2f98053a4c736183b15499e574c113)

          • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-1.1 #150 (See https://builds.apache.org/job/HBase-1.1/150/ ) HBASE-11568 . Addendum to add a file that I missed earlier. (ddas: rev 78c50af3ec2f98053a4c736183b15499e574c113) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionAdminServiceCallable.java
          Hide
          stack added a comment -

          Devaraj Das Grand.

          Show
          stack added a comment - Devaraj Das Grand.

            People

            • Assignee:
              Enis Soztutar
              Reporter:
              Enis Soztutar
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development