Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-834

Allow ephemeral znodes to have children created only by the owner session.

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.6.0
    • Component/s: c client, java client, server
    • Labels:
      None

      Description

      Ephemeral znodes are automatically removed when the client session is closed or expires and this behavior makes them very useful when you want to publish status information from active / connected clients.

      But there is a catch. Right now ephemerals can't have children znodes and because of that clients need to serialize status information as byte strings. This serialization renders that information almost invisible to generic zookeeper clients and hard / inefficient to update.

      Most of the time the status information can be expressed as a bunch of (key, value) pairs and we could easily store that using child znodes. Any ZooKeeper client can read that info without the need to reverse the serialization process and we can also easily update it.

      I suggest that the server should allow the ephemeral znodes to have children znodes. Each child should also be an ephemeral znode owned by the same session - parent ephemeralOwner session.

      Mail Archive:
      http://www.mail-archive.com/zookeeper-dev@hadoop.apache.org/msg09819.html

      Another discussion about the same topic:
      http://www.mail-archive.com/zookeeper-dev@hadoop.apache.org/msg08165.html

      1. ZOOKEEPER-834.patch
        16 kB
        Laxman
      2. ZOOKEEPER-834.1.patch
        16 kB
        Laxman
      3. ZOOKEEPER-834.2.patch
        22 kB
        Laxman
      4. ZOOKEEPER-834.patch
        35 kB
        Rakesh R

        Issue Links

          Activity

          Hide
          michim Michi Mutsuzaki added a comment -

          I'd like to hear from the community before moving forward with this. Reading the discussion thread, it wasn't clear if we reached a consensus on whether to implement ZOOKEEPER-834, ZOOKEEPER-723, both, or neither.

          Show
          michim Michi Mutsuzaki added a comment - I'd like to hear from the community before moving forward with this. Reading the discussion thread, it wasn't clear if we reached a consensus on whether to implement ZOOKEEPER-834 , ZOOKEEPER-723 , both, or neither.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12638296/ZOOKEEPER-834.patch
          against trunk revision 1583783.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12638296/ZOOKEEPER-834.patch against trunk revision 1583783. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/2023//console This message is automatically generated.
          Hide
          rakeshr Rakesh R added a comment -

          Following are the changes included in latest patch:

          • Provided validations for create2 opcode
          • refined existing testcases
          • added testcases for zkcli, multi, delete, create2 cases

          Review request : https://reviews.apache.org/r/19950/

          Show
          rakeshr Rakesh R added a comment - Following are the changes included in latest patch: Provided validations for create2 opcode refined existing testcases added testcases for zkcli, multi, delete, create2 cases Review request : https://reviews.apache.org/r/19950/
          Hide
          rakeshr Rakesh R added a comment -

          Few days back I had an offline discussion with Laxman about this issue. As per that I'm attaching a new patch to take the flow ahead.

          Show
          rakeshr Rakesh R added a comment - Few days back I had an offline discussion with Laxman about this issue. As per that I'm attaching a new patch to take the flow ahead.
          Hide
          michim Michi Mutsuzaki added a comment -

          I'm canceling the patch since it no longer applies cleanly.

          Show
          michim Michi Mutsuzaki added a comment - I'm canceling the patch since it no longer applies cleanly.
          Hide
          rakeshr Rakesh R added a comment -

          Hi, looks interesting feature and would like to take this ahead. I will do backgnd analysis and rework the patch in couple of days.

          Show
          rakeshr Rakesh R added a comment - Hi, looks interesting feature and would like to take this ahead. I will do backgnd analysis and rework the patch in couple of days.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch
          against trunk revision 1214571.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch against trunk revision 1214571. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/831//console This message is automatically generated.
          Hide
          jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2037/
          -----------------------------------------------------------

          Review request for zookeeper, Patrick Hunt, Benjamin Reed, and Mahadev Konar.

          Summary
          -------

          Allow ephemeral znodes to have children created only by the owner session.

          This addresses bug ZOOKEEPER-834.
          https://issues.apache.org/jira/browse/ZOOKEEPER-834

          Diffs


          ./src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml 1173537
          ./src/java/main/org/apache/zookeeper/KeeperException.java 1173537
          ./src/java/main/org/apache/zookeeper/ZooKeeperMain.java 1173537
          ./src/java/main/org/apache/zookeeper/server/DataTree.java 1173537
          ./src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 1173537
          ./src/java/test/org/apache/zookeeper/test/EphemeralNodeTest.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2037/diff

          Testing
          -------

          Patch includes tests for different scenarios. Passing in qa bot.

          Thanks,

          Laxman

          Show
          jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2037/ ----------------------------------------------------------- Review request for zookeeper, Patrick Hunt, Benjamin Reed, and Mahadev Konar. Summary ------- Allow ephemeral znodes to have children created only by the owner session. This addresses bug ZOOKEEPER-834 . https://issues.apache.org/jira/browse/ZOOKEEPER-834 Diffs ./src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml 1173537 ./src/java/main/org/apache/zookeeper/KeeperException.java 1173537 ./src/java/main/org/apache/zookeeper/ZooKeeperMain.java 1173537 ./src/java/main/org/apache/zookeeper/server/DataTree.java 1173537 ./src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java 1173537 ./src/java/test/org/apache/zookeeper/test/EphemeralNodeTest.java PRE-CREATION Diff: https://reviews.apache.org/r/2037/diff Testing ------- Patch includes tests for different scenarios. Passing in qa bot. Thanks, Laxman
          Hide
          thkoch Thomas Koch added a comment -

          Could you please be so kind to add the Code for review to https://reviews.apache.org/r/new/ ? Thank you!

          Show
          thkoch Thomas Koch added a comment - Could you please be so kind to add the Code for review to https://reviews.apache.org/r/new/ ? Thank you!
          Hide
          savu.andrei Andrei Savu added a comment -

          Looks good to me. Do we expect to see large ephemeral subtrees? Would it be a good idea to replace the HashSet with a TreeSet so that we can avoid allocating more memory for sorting before removal?

          Show
          savu.andrei Andrei Savu added a comment - Looks good to me. Do we expect to see large ephemeral subtrees? Would it be a good idea to replace the HashSet with a TreeSet so that we can avoid allocating more memory for sorting before removal?
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch
          against trunk revision 1172406.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495342/ZOOKEEPER-834.2.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/573//console This message is automatically generated.
          Hide
          lakshman Laxman added a comment -

          Thanks for the review comments.

          I have added two more test cases for showing the session closure and session expiration. Please review.

          Show
          lakshman Laxman added a comment - Thanks for the review comments. I have added two more test cases for showing the session closure and session expiration. Please review.
          Hide
          savu.andrei Andrei Savu added a comment -

          I think this patch also needs a test that shows that when a session is closed an ephemeral subtree is garbage collected as expected.

          Show
          savu.andrei Andrei Savu added a comment - I think this patch also needs a test that shows that when a session is closed an ephemeral subtree is garbage collected as expected.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495221/ZOOKEEPER-834.1.patch
          against trunk revision 1172406.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495221/ZOOKEEPER-834.1.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/568//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495204/ZOOKEEPER-834.patch
          against trunk revision 1172406.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The applied patch generated 12 javac compiler warnings (more than the trunk's current 9 warnings).

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495204/ZOOKEEPER-834.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 12 javac compiler warnings (more than the trunk's current 9 warnings). +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/567//console This message is automatically generated.
          Hide
          lakshman Laxman added a comment -

          Refreshed the patch for trunk.

          Show
          lakshman Laxman added a comment - Refreshed the patch for trunk.
          Hide
          phunt Patrick Hunt added a comment -

          Patch is not applying cleanly to trunk.

          Show
          phunt Patrick Hunt added a comment - Patch is not applying cleanly to trunk.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495071/ZOOKEEPER-834.patch
          against trunk revision 1172406.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/559//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495071/ZOOKEEPER-834.patch against trunk revision 1172406. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/559//console This message is automatically generated.
          Hide
          lakshman Laxman added a comment -

          Initial patch for review

          Show
          lakshman Laxman added a comment - Initial patch for review

            People

            • Assignee:
              rakeshr Rakesh R
              Reporter:
              savu.andrei Andrei Savu
            • Votes:
              3 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:

                Development