ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-834

Allow ephemeral znodes to have children created only by the owner session.

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.6.0
    • Component/s: c client, java client, server
    • Labels:
      None

      Description

      Ephemeral znodes are automatically removed when the client session is closed or expires and this behavior makes them very useful when you want to publish status information from active / connected clients.

      But there is a catch. Right now ephemerals can't have children znodes and because of that clients need to serialize status information as byte strings. This serialization renders that information almost invisible to generic zookeeper clients and hard / inefficient to update.

      Most of the time the status information can be expressed as a bunch of (key, value) pairs and we could easily store that using child znodes. Any ZooKeeper client can read that info without the need to reverse the serialization process and we can also easily update it.

      I suggest that the server should allow the ephemeral znodes to have children znodes. Each child should also be an ephemeral znode owned by the same session - parent ephemeralOwner session.

      Mail Archive:
      http://www.mail-archive.com/zookeeper-dev@hadoop.apache.org/msg09819.html

      Another discussion about the same topic:
      http://www.mail-archive.com/zookeeper-dev@hadoop.apache.org/msg08165.html

      1. ZOOKEEPER-834.patch
        35 kB
        Rakesh R
      2. ZOOKEEPER-834.2.patch
        22 kB
        Laxman
      3. ZOOKEEPER-834.1.patch
        16 kB
        Laxman
      4. ZOOKEEPER-834.patch
        16 kB
        Laxman

        Activity

        Hide
        Laxman added a comment -

        Initial patch for review

        Show
        Laxman added a comment - Initial patch for review
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495071/ZOOKEEPER-834.patch
        against trunk revision 1172406.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/559//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495071/ZOOKEEPER-834.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/559//console This message is automatically generated.
        Hide
        Patrick Hunt added a comment -

        Patch is not applying cleanly to trunk.

        Show
        Patrick Hunt added a comment - Patch is not applying cleanly to trunk.
        Hide
        Laxman added a comment -

        Refreshed the patch for trunk.

        Show
        Laxman added a comment - Refreshed the patch for trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495204/ZOOKEEPER-834.patch
        against trunk revision 1172406.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 12 javac compiler warnings (more than the trunk's current 9 warnings).

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495204/ZOOKEEPER-834.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 12 javac compiler warnings (more than the trunk's current 9 warnings). +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495221/ZOOKEEPER-834.1.patch
        against trunk revision 1172406.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495221/ZOOKEEPER-834.1.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//console This message is automatically generated.
        Hide
        Andrei Savu added a comment -

        I think this patch also needs a test that shows that when a session is closed an ephemeral subtree is garbage collected as expected.

        Show
        Andrei Savu added a comment - I think this patch also needs a test that shows that when a session is closed an ephemeral subtree is garbage collected as expected.
        Hide
        Laxman added a comment -

        Thanks for the review comments.

        I have added two more test cases for showing the session closure and session expiration. Please review.

        Show
        Laxman added a comment - Thanks for the review comments. I have added two more test cases for showing the session closure and session expiration. Please review.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch
        against trunk revision 1172406.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//console This message is automatically generated.
        Hide
        Andrei Savu added a comment -

        Looks good to me. Do we expect to see large ephemeral subtrees? Would it be a good idea to replace the HashSet with a TreeSet so that we can avoid allocating more memory for sorting before removal?

        Show
        Andrei Savu added a comment - Looks good to me. Do we expect to see large ephemeral subtrees? Would it be a good idea to replace the HashSet with a TreeSet so that we can avoid allocating more memory for sorting before removal?
        Hide
        Thomas Koch added a comment -

        Could you please be so kind to add the Code for review to https://reviews.apache.org/r/new/ ? Thank you!

        Show
        Thomas Koch added a comment - Could you please be so kind to add the Code for review to https://reviews.apache.org/r/new/ ? Thank you!
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2037/
        -----------------------------------------------------------

        Review request for zookeeper, Patrick Hunt, Benjamin Reed, and Mahadev Konar.

        Summary
        -------

        Allow ephemeral znodes to have children created only by the owner session.

        This addresses bug ZOOKEEPER-834.
        https://issues.apache.org/jira/browse/ZOOKEEPER-834

        Diffs


        ./src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml 1173537
        ./src/java/main/org/apache/zookeeper/KeeperException.java 1173537
        ./src/java/main/org/apache/zookeeper/ZooKeeperMain.java 1173537
        ./src/java/main/org/apache/zookeeper/server/DataTree.java 1173537
        ./src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 1173537
        ./src/java/test/org/apache/zookeeper/test/EphemeralNodeTest.java PRE-CREATION

        Diff: https://reviews.apache.org/r/2037/diff

        Testing
        -------

        Patch includes tests for different scenarios. Passing in qa bot.

        Thanks,

        Laxman

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2037/ ----------------------------------------------------------- Review request for zookeeper, Patrick Hunt, Benjamin Reed, and Mahadev Konar. Summary ------- Allow ephemeral znodes to have children created only by the owner session. This addresses bug ZOOKEEPER-834 . https://issues.apache.org/jira/browse/ZOOKEEPER-834 Diffs ./src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml 1173537 ./src/java/main/org/apache/zookeeper/KeeperException.java 1173537 ./src/java/main/org/apache/zookeeper/ZooKeeperMain.java 1173537 ./src/java/main/org/apache/zookeeper/server/DataTree.java 1173537 ./src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 1173537 ./src/java/test/org/apache/zookeeper/test/EphemeralNodeTest.java PRE-CREATION Diff: https://reviews.apache.org/r/2037/diff Testing ------- Patch includes tests for different scenarios. Passing in qa bot. Thanks, Laxman
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch
        against trunk revision 1214571.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch against trunk revision 1214571. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//console This message is automatically generated.
        Hide
        Rakesh R added a comment -

        Hi, looks interesting feature and would like to take this ahead. I will do backgnd analysis and rework the patch in couple of days.

        Show
        Rakesh R added a comment - Hi, looks interesting feature and would like to take this ahead. I will do backgnd analysis and rework the patch in couple of days.
        Hide
        Michi Mutsuzaki added a comment -

        I'm canceling the patch since it no longer applies cleanly.

        Show
        Michi Mutsuzaki added a comment - I'm canceling the patch since it no longer applies cleanly.
        Hide
        Rakesh R added a comment -

        Few days back I had an offline discussion with Laxman about this issue. As per that I'm attaching a new patch to take the flow ahead.

        Show
        Rakesh R added a comment - Few days back I had an offline discussion with Laxman about this issue. As per that I'm attaching a new patch to take the flow ahead.
        Hide
        Rakesh R added a comment -

        Following are the changes included in latest patch:

        • Provided validations for create2 opcode
        • refined existing testcases
        • added testcases for zkcli, multi, delete, create2 cases

        Review request : https://reviews.apache.org/r/19950/

        Show
        Rakesh R added a comment - Following are the changes included in latest patch: Provided validations for create2 opcode refined existing testcases added testcases for zkcli, multi, delete, create2 cases Review request : https://reviews.apache.org/r/19950/
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12638296/ZOOKEEPER-834.patch
        against trunk revision 1583783.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12638296/ZOOKEEPER-834.patch against trunk revision 1583783. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//console This message is automatically generated.
        Hide
        Michi Mutsuzaki added a comment -

        I'd like to hear from the community before moving forward with this. Reading the discussion thread, it wasn't clear if we reached a consensus on whether to implement ZOOKEEPER-834, ZOOKEEPER-723, both, or neither.

        Show
        Michi Mutsuzaki added a comment - I'd like to hear from the community before moving forward with this. Reading the discussion thread, it wasn't clear if we reached a consensus on whether to implement ZOOKEEPER-834 , ZOOKEEPER-723 , both, or neither.

          People

          • Assignee:
            Rakesh R
            Reporter:
            Andrei Savu
          • Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:

              Development