Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3469

ZKRMStateStore: Avoid setting watches that are not required

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.1, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In ZKRMStateStore, most operations(e.g. getDataWithRetries, getDataWithRetries, getDataWithRetries) set watches on znode. Large watches will cause problem such as ZOOKEEPER-706: large numbers of watches can cause session re-establishment to fail.

      Although there is a workaround that setting jute.maxbuffer to a larger value, we need to adjust this value once there are more app and attempts stored in ZK. And those watches are useless now. It might be better that do not set watches.

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724219/YARN-3469.01.patch
          against trunk revision 6495940.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7271//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7271//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724219/YARN-3469.01.patch against trunk revision 6495940. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7271//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7271//console This message is automatically generated.
          Hide
          kasha Karthik Kambatla added a comment -

          When working on YARN-2716, I was wondering about the same. I think not setting watches makes sense. I ll let Jian He also comment before committing this.

          Show
          kasha Karthik Kambatla added a comment - When working on YARN-2716 , I was wondering about the same. I think not setting watches makes sense. I ll let Jian He also comment before committing this.
          Hide
          hex108 Jun Gong added a comment -

          Karthik Kambatla Thank you for the review and comment.

          Show
          hex108 Jun Gong added a comment - Karthik Kambatla Thank you for the review and comment.
          Hide
          jianhe Jian He added a comment -

          lgtm, thanks Jun Gong and Karthik Kambatla

          Show
          jianhe Jian He added a comment - lgtm, thanks Jun Gong and Karthik Kambatla
          Hide
          kasha Karthik Kambatla added a comment -

          +1. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - +1. Checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks Jun Gong for the patch and Jian for the review. Just committed to trunk and branch-2.

          Show
          kasha Karthik Kambatla added a comment - Thanks Jun Gong for the patch and Jian for the review. Just committed to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7564 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7564/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7564 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7564/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #160 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/160/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #160 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/160/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2092 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2092/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2092 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2092/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #151 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/151/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #151 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/151/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #894 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/894/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #894 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/894/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/161/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/161/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2110 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2110/)
          YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2110 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2110/ ) YARN-3469 . ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) (kasha: rev e516706b896743e47e2852be81944eb5613e3e76) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7576 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7576/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7576 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7576/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/163/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/163/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2095 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2095/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2095 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2095/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #897 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/897/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #897 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/897/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #154 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/154/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #154 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/154/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/164/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/164/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2113 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2113/)
          Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1)

          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2113 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2113/ ) Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 (kasha: rev 5cda6fffd39fcb997b6e3cbcd0a2e2dce07153e1) hadoop-yarn-project/CHANGES.txt
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Pulled this into 2.7 and (1) it seems useful for production clusters and (2) YARN-3385 got conflicts on 2.7 branch without this patch.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.7 and (1) it seems useful for production clusters and (2) YARN-3385 got conflicts on 2.7 branch without this patch.
          Hide
          chenchun Chun Chen added a comment -

          Just a note. I saw ZOOKEEPER-706 has been resolved recently after 5 years.

          Show
          chenchun Chun Chen added a comment - Just a note. I saw ZOOKEEPER-706 has been resolved recently after 5 years.
          Hide
          kasha Karthik Kambatla added a comment -

          2.8 uses Curator and all watch handling is now implicit.

          Show
          kasha Karthik Kambatla added a comment - 2.8 uses Curator and all watch handling is now implicit.
          Hide
          king0719 Chuan Jin added a comment -

          Our cluster uses 2.7.3 branch which merges the Curator related codes from 2.8.0 branch, but we still encounter this problem. I also notice the ZOOKEEPER-706 , and replace the old zookeeper jar(3.4.6) to a newer one(3.4.10). It does solve my problem.
          I think it's a good reason to update the zookeeper version in hadoop trunk.

          Show
          king0719 Chuan Jin added a comment - Our cluster uses 2.7.3 branch which merges the Curator related codes from 2.8.0 branch, but we still encounter this problem. I also notice the ZOOKEEPER-706 , and replace the old zookeeper jar(3.4.6) to a newer one(3.4.10). It does solve my problem. I think it's a good reason to update the zookeeper version in hadoop trunk.

            People

            • Assignee:
              hex108 Jun Gong
              Reporter:
              hex108 Jun Gong
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development