HBase
  1. HBase
  2. HBASE-10448

ZKUtil create and watch methods don't set watch in some cases

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.96.0, 0.96.1.1
    • Fix Version/s: 0.98.0, 0.96.2, 0.99.0, 0.94.17
    • Component/s: Zookeeper
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      While using the ZKUtil methods during testing, I found that watch was not set when it should be set based on the methods and method comments:
      createNodeIfNotExistsAndWatch
      createEphemeralNodeAndWatch

      For example, in createNodeIfNotExistsAndWatch():

       public static boolean createNodeIfNotExistsAndWatch(
            ZooKeeperWatcher zkw, String znode, byte [] data)
        throws KeeperException {
          try {
            zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
                CreateMode.PERSISTENT);
          } catch (KeeperException.NodeExistsException nee) {
            try {
              zkw.getRecoverableZooKeeper().exists(znode, zkw);
            } catch (InterruptedException e) {
              zkw.interruptedException(e);
              return false;
            }
            return false;
          } catch (InterruptedException e) {
            zkw.interruptedException(e);
            return false;
          }
          return true;
        }
      

      The watch is only set via exists() call when the node already exists.
      Similarly in createEphemeralNodeAndWatch():

        public static boolean createEphemeralNodeAndWatch(ZooKeeperWatcher zkw,
            String znode, byte [] data)
        throws KeeperException {
          try {
            zkw.getRecoverableZooKeeper().create(znode, data, createACL(zkw, znode),
                CreateMode.EPHEMERAL);
          } catch (KeeperException.NodeExistsException nee) {
            if(!watchAndCheckExists(zkw, znode)) {
              // It did exist but now it doesn't, try again
              return createEphemeralNodeAndWatch(zkw, znode, data);
            }
            return false;
          } catch (InterruptedException e) {
            LOG.info("Interrupted", e);
            Thread.currentThread().interrupt();
          }
          return true;
        }
      

        Issue Links

          Activity

          Hide
          Jerry He added a comment -

          I wonder how the callers/users of these 2 methods actually worked.
          A guess is that they don't really depend on the watches being set in the cases that the watches are not set.

          Show
          Jerry He added a comment - I wonder how the callers/users of these 2 methods actually worked. A guess is that they don't really depend on the watches being set in the cases that the watches are not set.
          Hide
          Jerry He added a comment -

          Found this HBASE-8937 that reported similar problem.

          Show
          Jerry He added a comment - Found this HBASE-8937 that reported similar problem.
          Hide
          Jerry He added a comment -

          Attached a patch that set the watch whether or not we get NodeExistsException.
          I tried to avoid any other change in behavior of these two methods.

          Show
          Jerry He added a comment - Attached a patch that set the watch whether or not we get NodeExistsException. I tried to avoid any other change in behavior of these two methods.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12626450/HBASE-10448-trunk.patch
          against trunk revision .
          ATTACHMENT ID: 12626450

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop1.1. The patch compiles against the hadoop 1.1 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:

          -1 core zombie tests. There are 1 zombie test(s): at org.apache.hadoop.hbase.io.encoding.TestEncodedSeekers.testEncodedSeeker(TestEncodedSeekers.java:125)

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12626450/HBASE-10448-trunk.patch against trunk revision . ATTACHMENT ID: 12626450 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop1.1 . The patch compiles against the hadoop 1.1 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.io.encoding.TestEncodedSeekers.testEncodedSeeker(TestEncodedSeekers.java:125) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8570//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          A guess is that they don't really depend on the watches being set in the cases that the watches are not set.

          You are right, I don't follow how these *AndWatch methods lived up to the expectations of their naming.

          +1

          Show
          Andrew Purtell added a comment - A guess is that they don't really depend on the watches being set in the cases that the watches are not set. You are right, I don't follow how these *AndWatch methods lived up to the expectations of their naming. +1
          Hide
          Ted Yu added a comment -

          I tried to apply patch on 0.98 but there was conflict in hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java.rej

          Also, please trim the trailing ^M at line end.

          Thanks

          Show
          Ted Yu added a comment - I tried to apply patch on 0.98 but there was conflict in hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java.rej Also, please trim the trailing ^M at line end. Thanks
          Hide
          Ted Yu added a comment -

          Integrated to 0.98 and trunk.

          Thanks for the patch, Jerry.

          Show
          Ted Yu added a comment - Integrated to 0.98 and trunk. Thanks for the patch, Jerry.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK #4873 (See https://builds.apache.org/job/HBase-TRUNK/4873/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563507)

          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4873 (See https://builds.apache.org/job/HBase-TRUNK/4873/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563507) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Jerry He added a comment -

          Thanks, Ted, Andrew.
          Could you also mark HBASE-8937 as resolved as dup so that we have closure on that one too?

          Show
          Jerry He added a comment - Thanks, Ted, Andrew. Could you also mark HBASE-8937 as resolved as dup so that we have closure on that one too?
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.98 #119 (See https://builds.apache.org/job/HBase-0.98/119/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563506)

          • /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.98 #119 (See https://builds.apache.org/job/HBase-0.98/119/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563506) /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #111 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/111/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563506)

          • /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #111 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/111/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563506) /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          stack added a comment -

          Good one Jerry He Committed to 0.96 branch.

          Show
          stack added a comment - Good one Jerry He Committed to 0.96 branch.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in hbase-0.96 #276 (See https://builds.apache.org/job/hbase-0.96/276/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (stack: rev 1563567)

          • /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - SUCCESS: Integrated in hbase-0.96 #276 (See https://builds.apache.org/job/hbase-0.96/276/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (stack: rev 1563567) /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in hbase-0.96-hadoop2 #190 (See https://builds.apache.org/job/hbase-0.96-hadoop2/190/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (stack: rev 1563567)

          • /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in hbase-0.96-hadoop2 #190 (See https://builds.apache.org/job/hbase-0.96-hadoop2/190/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (stack: rev 1563567) /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Lars Hofhansl added a comment -

          Lemme check if this is an issue in 0.94 as well.

          Show
          Lars Hofhansl added a comment - Lemme check if this is an issue in 0.94 as well.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #75 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/75/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563507)

          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #75 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/75/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases (Jerry He) (tedyu: rev 1563507) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Lars Hofhansl added a comment -

          Committed to 0.94 as well. (I hope we are not relying anywhere in the current behavior).

          Show
          Lars Hofhansl added a comment - Committed to 0.94 as well. (I hope we are not relying anywhere in the current behavior).
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #6 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/6/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783)

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #6 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/6/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-0.94-security #396 (See https://builds.apache.org/job/HBase-0.94-security/396/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783)

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-0.94-security #396 (See https://builds.apache.org/job/HBase-0.94-security/396/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-0.94-JDK7 #35 (See https://builds.apache.org/job/HBase-0.94-JDK7/35/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783)

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-0.94-JDK7 #35 (See https://builds.apache.org/job/HBase-0.94-JDK7/35/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-0.94 #1269 (See https://builds.apache.org/job/HBase-0.94/1269/)
          HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783)

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-0.94 #1269 (See https://builds.apache.org/job/HBase-0.94/1269/ ) HBASE-10448 ZKUtil create and watch methods don't set watch in some cases. (Jerry He) (larsh: rev 1563783) /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
          Hide
          Andrew Purtell added a comment -

          (I hope we are not relying anywhere in the current behavior).

          I doubt it, but if we are that would be a bug that needs fixing anyhow.

          Show
          Andrew Purtell added a comment - (I hope we are not relying anywhere in the current behavior). I doubt it, but if we are that would be a bug that needs fixing anyhow.

            People

            • Assignee:
              Jerry He
              Reporter:
              Jerry He
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development