Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.4.0
    • Fix Version/s: 3.6.0
    • Component/s: server
    • Labels:
      None
    • Environment:
    • Release Note:
      ZOOKEEPER-892. Remote replication of ZooKeeper data (Anirban Roy)
    • Tags:
      zkrepl replication zoorepl

      Description

      ZooKeeper is a highly available and scalable system for distributed synchrony and is frequently used for cluster management. In its current incarnation it has issues with communication and data replication across extended geographic locations. Presently, the only way to distribute ZooKeeper across multiple data centers is to maintain a cross-colo Quorum using Observer members, leading to unnecessary consumption of bandwidth and performance impacts. As the title suggests, this work aims to to provide replication of ZooKeeper data from one site to others using a new type of ZooKeeper member called a Publisher. The broad idea is to have a complete instance of the current ZooKeeper at each geographic location in a master-slave setup. The Publisher will be a part of the Master ZooKeeper Site and will push changes to a FIFO queue and make it available to any interested client. The slave ZooKeeper runs client application called Replicator which receives and replays the changes to slave instance. Multiple slave Replicators can subscribes to the master Publisher and receive changes with guaranteed ordering. It will be asynchronous, non-intrusive, loosely-coupled and can be applied to a subset of the data. This scheme will bring about many of the benefits of database replication such as resilience to site failure and localized serving across data centers. In short, the goal is to provide remote (sub-tree) data replication with guaranteed ordering, without affecting the Master ZooKeeper performance.

      1. zoorepl.pdf
        548 kB
        Anirban Roy
      2. zoorepl.patch
        176 kB
        Anirban Roy
      3. zoorepl.patch
        176 kB
        Anirban Roy

        Activity

        Hide
        Patrick Hunt added a comment -

        Sounds like an interesting feature, can you compare it to observers?
        http://www.cloudera.com/blog/2009/12/observers-making-zookeeper-scale-even-further/

        Henry & Flavio might have additional insight on this. Esp around the "subset of data" feature. This is a great idea, however in practice it might be tricky (esp ensuring that if you dynamically change the subset (expand say) that both the servers and clients are setup to handle this properly.) Def think through the various scenarios this encompasses re "correct" operation.

        BTW: Previously I've seen this done on the client side rather than on the server side. In particular users have setup "bridge" clients whose job it is to copy data from a local cluster to a remote one. in this scenario you have 2 clusters that are distinct/separate. And clients (one or more on each cluster, depending on reliability requirements) that copy a subset of the data. You lose out on ordering guarantees though (across the clusters I mean) so that may not be what you are looking for here.

        Also keep in mind reliability/availability. If you have a single "replicated" server in the remote location you should also consider what happens if that server is unavailable to server clients for whatever reason.

        Show
        Patrick Hunt added a comment - Sounds like an interesting feature, can you compare it to observers? http://www.cloudera.com/blog/2009/12/observers-making-zookeeper-scale-even-further/ Henry & Flavio might have additional insight on this. Esp around the "subset of data" feature. This is a great idea, however in practice it might be tricky (esp ensuring that if you dynamically change the subset (expand say) that both the servers and clients are setup to handle this properly.) Def think through the various scenarios this encompasses re "correct" operation. BTW: Previously I've seen this done on the client side rather than on the server side. In particular users have setup "bridge" clients whose job it is to copy data from a local cluster to a remote one. in this scenario you have 2 clusters that are distinct/separate. And clients (one or more on each cluster, depending on reliability requirements) that copy a subset of the data. You lose out on ordering guarantees though (across the clusters I mean) so that may not be what you are looking for here. Also keep in mind reliability/availability. If you have a single "replicated" server in the remote location you should also consider what happens if that server is unavailable to server clients for whatever reason.
        Hide
        Anirban Roy added a comment -

        Sorry for not coming back to this ticket in between.

        Thanks Patrick for your insight. I'm going to attach the design doc where we discussed different approaches including the one with observer. In summary, the master and slave zookeeper are designed to be loosely coupled(run independent of each other without creating any bottleneck) with a fast and reliable JMS layer in between. The reliability, availability, auto-failover and more importantly the guaranteed ordering of updates were taken care in the design.

        Please go through and put your comment/suggestion.

        Show
        Anirban Roy added a comment - Sorry for not coming back to this ticket in between. Thanks Patrick for your insight. I'm going to attach the design doc where we discussed different approaches including the one with observer. In summary, the master and slave zookeeper are designed to be loosely coupled(run independent of each other without creating any bottleneck) with a fast and reliable JMS layer in between. The reliability, availability, auto-failover and more importantly the guaranteed ordering of updates were taken care in the design. Please go through and put your comment/suggestion.
        Hide
        Anirban Roy added a comment -

        The feature "ZooRepl: Remote Replication of ZooKeeper data" is implemented.

        Show
        Anirban Roy added a comment - The feature "ZooRepl: Remote Replication of ZooKeeper data" is implemented.
        Hide
        Anirban Roy added a comment -

        ZooRepl patch

        Show
        Anirban Roy added a comment - ZooRepl patch
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12475339/zoorepl.patch
        against trunk revision 1087821.

        -1 @author. The patch appears to contain 20 @author tags which the Zookeeper community has agreed to not allow in code contributions.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The patch appears to cause tar ant target to fail.

        -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/218//testReport/
        Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/218//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475339/zoorepl.patch against trunk revision 1087821. -1 @author. The patch appears to contain 20 @author tags which the Zookeeper community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/218//testReport/ Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/218//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        ZooRepl design

        Show
        Anirban Roy added a comment - ZooRepl design
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12475340/zoorepl.pdf
        against trunk revision 1087821.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/219//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475340/zoorepl.pdf against trunk revision 1087821. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/219//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        Updated ZooRepl patch

        Show
        Anirban Roy added a comment - Updated ZooRepl patch
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12475688/zoorepl.patch
        against trunk revision 1089617.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The patch appears to cause tar ant target to fail.

        -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/222//testReport/
        Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/222//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475688/zoorepl.patch against trunk revision 1089617. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/222//testReport/ Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/222//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        Fixed dependencies with ActiveMQ, Apache Commons Codec and Jackson JSON util.

        Show
        Anirban Roy added a comment - Fixed dependencies with ActiveMQ, Apache Commons Codec and Jackson JSON util.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12475782/zoorepl.patch
        against trunk revision 1089617.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475782/zoorepl.patch against trunk revision 1089617. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/223//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        Patch with slf4j logging.

        Show
        Anirban Roy added a comment - Patch with slf4j logging.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12477395/zoorepl.patch
        against trunk revision 1091841.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

        -1 javac. The patch appears to cause tar ant target to fail.

        -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477395/zoorepl.patch against trunk revision 1091841. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/234//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        This patch build was successful with the following command -

        ant -Djavac.args="-Xlint -Xmaxwarns 1000" clean compile test tar

        Show
        Anirban Roy added a comment - This patch build was successful with the following command - ant -Djavac.args="-Xlint -Xmaxwarns 1000" clean compile test tar
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478034/zoorepl.patch
        against trunk revision 1097865.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478034/zoorepl.patch against trunk revision 1097865. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 20 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/237//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        Removed findbug warnings.

        Show
        Anirban Roy added a comment - Removed findbug warnings.
        Hide
        Anirban Roy added a comment -

        Resubmitting the patch.

        Show
        Anirban Roy added a comment - Resubmitting the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478243/zoorepl.patch
        against trunk revision 1099329.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478243/zoorepl.patch against trunk revision 1099329. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/249//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478364/zoorepl.patch
        against trunk revision 1099329.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478364/zoorepl.patch against trunk revision 1099329. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/250//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        Minor changes related to JMS selector.

        Show
        Anirban Roy added a comment - Minor changes related to JMS selector.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch
        against trunk revision 1103811.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch against trunk revision 1103811. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/267//console This message is automatically generated.
        Hide
        Anirban Roy added a comment -

        The complete patch is available now. Resolving the task, please provide any feedback/suggestion.

        Show
        Anirban Roy added a comment - The complete patch is available now. Resolving the task, please provide any feedback/suggestion.
        Hide
        Mahadev konar added a comment -

        Anirban, the patch hasnt been checked in yet, you should not mark it resolved.

        Show
        Mahadev konar added a comment - Anirban, the patch hasnt been checked in yet, you should not mark it resolved.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch
        against trunk revision 1103811.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch against trunk revision 1103811. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/270//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch
        against trunk revision 1103811.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/271//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479547/zoorepl.patch against trunk revision 1103811. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/271//console This message is automatically generated.
        Hide
        Patrick Hunt added a comment -

        According to jenkins the patch is out of date with trunk, Anirban can you take a look?

        Show
        Patrick Hunt added a comment - According to jenkins the patch is out of date with trunk, Anirban can you take a look?
        Hide
        Anirban Roy added a comment -

        The previous one was probably out of sync, not sure why. Corrected now.

        Show
        Anirban Roy added a comment - The previous one was probably out of sync, not sure why. Corrected now.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12480083/zoorepl.patch
        against trunk revision 1125581.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480083/zoorepl.patch against trunk revision 1125581. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/280//console This message is automatically generated.
        Hide
        Mahadev konar added a comment -

        Moving it out, since this isnt a blocker or critical.

        Show
        Mahadev konar added a comment - Moving it out, since this isnt a blocker or critical.
        Hide
        Flavio Junqueira added a comment -

        I can't believe we have had this patch available for so long. It is almost two years old! Is there still interest in having this feature in? If so, we should revisit the idea and review the patch. For now, I'm cancelling the patch.

        Show
        Flavio Junqueira added a comment - I can't believe we have had this patch available for so long. It is almost two years old! Is there still interest in having this feature in? If so, we should revisit the idea and review the patch. For now, I'm cancelling the patch.
        Hide
        Alexander Shraer added a comment -

        Hi Flavio,

        So the main advantage of this scheme over simply using observers is that the proposed feature can replicate only a portion of the data, correct ? In that case, and if such a functionality is still needed, maybe its worth to consider a simpler alternative - a static or dynamic way for an observer to specify the subtree(s) in which it is interested. The leader can then inform the observer only about things in which it is interested.

        Show
        Alexander Shraer added a comment - Hi Flavio, So the main advantage of this scheme over simply using observers is that the proposed feature can replicate only a portion of the data, correct ? In that case, and if such a functionality is still needed, maybe its worth to consider a simpler alternative - a static or dynamic way for an observer to specify the subtree(s) in which it is interested. The leader can then inform the observer only about things in which it is interested.
        Hide
        Alexander Shraer added a comment -

        Actually, the second advantage of the proposed scheme over observers is probably that the leader is not dealing with all the remote replication, which is nice.

        Show
        Alexander Shraer added a comment - Actually, the second advantage of the proposed scheme over observers is probably that the leader is not dealing with all the remote replication, which is nice.
        Hide
        Flavio Junqueira added a comment -

        Right, I haven't looked at this issue in a while. If I remember correctly, one nice property is that it builds on ZooKeeper and doesn't require changes. If we filter directly in ZooKeeper, we either do it on the leader or on the observer. If we choose the leader, then it is more work and configuration for the leader. If we choose the observer, then we are sending cross-colo messages unnecessarily.

        Overall it sounds like a reasonable design, but the question is if anyone is interested in using this feature. It is ok with me to have it in, but I'd like to have a champion for it.

        Show
        Flavio Junqueira added a comment - Right, I haven't looked at this issue in a while. If I remember correctly, one nice property is that it builds on ZooKeeper and doesn't require changes. If we filter directly in ZooKeeper, we either do it on the leader or on the observer. If we choose the leader, then it is more work and configuration for the leader. If we choose the observer, then we are sending cross-colo messages unnecessarily. Overall it sounds like a reasonable design, but the question is if anyone is interested in using this feature. It is ok with me to have it in, but I'd like to have a champion for it.

          People

          • Assignee:
            Anirban Roy
            Reporter:
            Anirban Roy
          • Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 2,688h
              2,688h
              Remaining:
              Remaining Estimate - 2,688h
              2,688h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development