Details

    • Type: Task Task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. 5965.txt
      40 kB
      stack
    2. 5965v2.txt
      58 kB
      stack
    3. 5965v3.txt
      60 kB
      stack
    4. 5965v4.txt
      60 kB
      stack
    5. 5965v5.txt
      96 kB
      Chris Trezzo

      Issue Links

        Activity

        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #154 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/154/)
        HBASE-5965 Move replication znodes to pb (Revision 1378714)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
        • /hbase/trunk/hbase-server/src/main/protobuf/ZooKeeper.proto
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #154 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/154/ ) HBASE-5965 Move replication znodes to pb (Revision 1378714) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java /hbase/trunk/hbase-server/src/main/protobuf/ZooKeeper.proto /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #3293 (See https://builds.apache.org/job/HBase-TRUNK/3293/)
        HBASE-5965 Move replication znodes to pb (Revision 1378714)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
        • /hbase/trunk/hbase-server/src/main/protobuf/ZooKeeper.proto
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #3293 (See https://builds.apache.org/job/HBase-TRUNK/3293/ ) HBASE-5965 Move replication znodes to pb (Revision 1378714) Result = FAILURE stack : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java /hbase/trunk/hbase-server/src/main/protobuf/ZooKeeper.proto /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
        Hide
        Chris Trezzo added a comment -

        Here is the follow up jira: HBASE-6687

        Show
        Chris Trezzo added a comment - Here is the follow up jira: HBASE-6687
        Hide
        stack added a comment -

        Committed on trunk. Thanks for the patch mighty Chris.

        Show
        stack added a comment - Committed on trunk. Thanks for the patch mighty Chris.
        Hide
        Chris Trezzo added a comment -

        Sorry that was confusing. v5 is the patch that was up on RB. I will make a separate issue for handling deserialization issues.

        Thanks,
        Chris

        Show
        Chris Trezzo added a comment - Sorry that was confusing. v5 is the patch that was up on RB. I will make a separate issue for handling deserialization issues. Thanks, Chris
        Hide
        stack added a comment -

        Hey Chris: What is difference that v5 adds (review suggests moving handling of failed deser of clusterkey to another issue...). Thanks boss.

        Show
        stack added a comment - Hey Chris: What is difference that v5 adds (review suggests moving handling of failed deser of clusterkey to another issue...). Thanks boss.
        Hide
        Chris Trezzo added a comment -

        Addressing the issues that stack brought up in the above review.

        Show
        Chris Trezzo added a comment - Addressing the issues that stack brought up in the above review.
        Hide
        Chris Trezzo added a comment -

        I think we got all of them. Committing these sounds good to me. Thanks!

        Chris

        Show
        Chris Trezzo added a comment - I think we got all of them. Committing these sounds good to me. Thanks! Chris
        Hide
        stack added a comment -

        Patch up in RB looks good to me. You think we got them all Chris? We should commit this at least and if any outstanding, we can do in another issue?

        Show
        stack added a comment - Patch up in RB looks good to me. You think we got them all Chris? We should commit this at least and if any outstanding, we can do in another issue?
        Hide
        Chris Trezzo added a comment -

        Posted a review request: https://reviews.apache.org/r/6805/

        It introduced one new find bug saying that ReplicationZookeeper.ENABLED_ZNODE_BYTES should be package protected because it can be maliciously changed, but it is a static final variable...

        Chris

        Show
        Chris Trezzo added a comment - Posted a review request: https://reviews.apache.org/r/6805/ It introduced one new find bug saying that ReplicationZookeeper.ENABLED_ZNODE_BYTES should be package protected because it can be maliciously changed, but it is a static final variable... Chris
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542384/5965v5.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        -1 javadoc. The javadoc tool appears to have generated 94 warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.TestCheckTestClasses

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542384/5965v5.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 94 warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestCheckTestClasses Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2689//console This message is automatically generated.
        Hide
        Chris Trezzo added a comment -

        Here is another version of the patch. I have all replication tests passing locally (except for TestReplication.queueFailover).

        I also noticed that the lock znode during region server failover writes the owner of the lock in the znode value. I PB'd this, but didn't find any place where it is actually read. If this value is purely for post-mortem manual inspection of state, then maybe it might be better not to PB it?

        Chris

        Show
        Chris Trezzo added a comment - Here is another version of the patch. I have all replication tests passing locally (except for TestReplication.queueFailover). I also noticed that the lock znode during region server failover writes the owner of the lock in the znode value. I PB'd this, but didn't find any place where it is actually read. If this value is purely for post-mortem manual inspection of state, then maybe it might be better not to PB it? Chris
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12542367/5965v4.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).

        -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.replication.TestReplication
        org.apache.hadoop.hbase.security.access.TestTablePermissions
        org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
        org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink
        org.apache.hadoop.hbase.replication.TestMasterReplication
        org.apache.hadoop.hbase.master.cleaner.TestLogsCleaner

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542367/5965v4.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.security.access.TestTablePermissions org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink org.apache.hadoop.hbase.replication.TestMasterReplication org.apache.hadoop.hbase.master.cleaner.TestLogsCleaner Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2685//console This message is automatically generated.
        Hide
        stack added a comment -

        Here is a list by Chris over in HBASE-5450 on what needs converting. Let me make sure this patch catches them all:

        The value of the replication status znode (TRUE or FALSE)
        The cluster key for a peer cluster, containing the zk quorum, zk port and hbase base znode (zk1.host.com,zk2.host.com,zk3.host.com:2181:/hbase)
        The peer state znode (ENABLED or DISABLED)
        The HLog position (a long containing the position in the hlog)
        
        Show
        stack added a comment - Here is a list by Chris over in HBASE-5450 on what needs converting. Let me make sure this patch catches them all: The value of the replication status znode (TRUE or FALSE) The cluster key for a peer cluster, containing the zk quorum, zk port and hbase base znode (zk1.host.com,zk2.host.com,zk3.host.com:2181:/hbase) The peer state znode (ENABLED or DISABLED) The HLog position (a long containing the position in the hlog)
        Hide
        stack added a comment -

        Trying it against hadoopqa to see how this updated patch does.

        Show
        stack added a comment - Trying it against hadoopqa to see how this updated patch does.
        Hide
        stack added a comment -

        Updated for trunk.

        Show
        stack added a comment - Updated for trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526219/5965v3.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.master.TestLogsCleaner
        org.apache.hadoop.hbase.client.TestFromClientSide
        org.apache.hadoop.hbase.replication.TestReplication
        org.apache.hadoop.hbase.TestDrainingServer
        org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
        org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink
        org.apache.hadoop.hbase.replication.TestMasterReplication

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526219/5965v3.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestLogsCleaner org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1822//console This message is automatically generated.
        Hide
        stack added a comment -

        Retry

        Show
        stack added a comment - Retry
        Hide
        stack added a comment -

        Missed a deserialization spot.

        Show
        stack added a comment - Missed a deserialization spot.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526203/5965v2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1820//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526203/5965v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1820//console This message is automatically generated.
        Hide
        stack added a comment -

        Added ReplicationState and ReplicationPeer pb messages. Refactored replication code to use this stuff (mostly just changes in ReplicationZooKeeper).

        Show
        stack added a comment - Added ReplicationState and ReplicationPeer pb messages. Refactored replication code to use this stuff (mostly just changes in ReplicationZooKeeper).
        Hide
        stack added a comment -

        A start. Not finished.

        Show
        stack added a comment - A start. Not finished.

          People

          • Assignee:
            Chris Trezzo
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development