Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14773

Extend ZKCuratorManager API for more reusability

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-beta1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-10631 needs some minor changes in ZKCuratorManager:

      • boolean delete()
      • getData(path, stat)
      • createRootDirRecursively(path)
      1. HADOOP-14773-000.patch
        5 kB
        Íñigo Goiri
      2. HADOOP-14773-001.patch
        7 kB
        Íñigo Goiri

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #12194 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12194/)
          HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo (subu: rev 75dd866bfb8b63cb9f13179d4365b05c48e0907d)

          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestZKCuratorManager.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #12194 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12194/ ) HADOOP-14773 . Extend ZKCuratorManager API for more reusability. (Íñigo (subu: rev 75dd866bfb8b63cb9f13179d4365b05c48e0907d) (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestZKCuratorManager.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/ZKRMStateStore.java
          Hide
          subru Subru Krishnan added a comment -

          Thanks Íñigo Goiri for your contribution, I have committed this to trunk/branch-2.

          Show
          subru Subru Krishnan added a comment - Thanks Íñigo Goiri for your contribution, I have committed this to trunk/branch-2.
          Hide
          elgoiri Íñigo Goiri added a comment - - edited

          TestRPC works fine in my machine and TestContainerAllocation failed in other builds.
          I don't think any of them are related to this patch.

          Show
          elgoiri Íñigo Goiri added a comment - - edited TestRPC works fine in my machine and TestContainerAllocation failed in other builds. I don't think any of them are related to this patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          0 mvndep 0m 18s Maven dependency ordering for branch
          +1 mvninstall 15m 52s trunk passed
          +1 compile 17m 16s trunk passed
          +1 checkstyle 2m 3s trunk passed
          +1 mvnsite 2m 12s trunk passed
          +1 findbugs 2m 52s trunk passed
          +1 javadoc 1m 27s trunk passed
                Patch Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for patch
          +1 mvninstall 1m 24s the patch passed
          +1 compile 11m 45s the patch passed
          +1 javac 11m 45s the patch passed
          +1 checkstyle 2m 7s root: The patch generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2)
          +1 mvnsite 2m 28s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 3s the patch passed
          +1 javadoc 1m 30s the patch passed
                Other Tests
          -1 unit 9m 15s hadoop-common in the patch failed.
          -1 unit 44m 18s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 36s The patch does not generate ASF License warnings.
          141m 14s



          Reason Tests
          Failed junit tests hadoop.ipc.TestRPC
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14773
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881994/HADOOP-14773-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 10549a3eefed 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dadb0c2
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 15m 52s trunk passed +1 compile 17m 16s trunk passed +1 checkstyle 2m 3s trunk passed +1 mvnsite 2m 12s trunk passed +1 findbugs 2m 52s trunk passed +1 javadoc 1m 27s trunk passed       Patch Compile Tests 0 mvndep 0m 20s Maven dependency ordering for patch +1 mvninstall 1m 24s the patch passed +1 compile 11m 45s the patch passed +1 javac 11m 45s the patch passed +1 checkstyle 2m 7s root: The patch generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) +1 mvnsite 2m 28s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 3s the patch passed +1 javadoc 1m 30s the patch passed       Other Tests -1 unit 9m 15s hadoop-common in the patch failed. -1 unit 44m 18s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 36s The patch does not generate ASF License warnings. 141m 14s Reason Tests Failed junit tests hadoop.ipc.TestRPC   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14773 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881994/HADOOP-14773-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 10549a3eefed 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dadb0c2 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13037/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 38s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 25m 48s trunk passed
          +1 compile 26m 12s trunk passed
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 2m 26s trunk passed
          +1 findbugs 2m 30s trunk passed
          +1 javadoc 1m 26s trunk passed
                Patch Compile Tests
          +1 mvninstall 1m 13s the patch passed
          +1 compile 20m 42s the patch passed
          +1 javac 20m 42s the patch passed
          +1 checkstyle 1m 1s the patch passed
          +1 mvnsite 2m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 56s the patch passed
          +1 javadoc 1m 27s the patch passed
                Other Tests
          -1 unit 12m 18s hadoop-common in the patch failed.
          +1 asflicense 0m 55s The patch does not generate ASF License warnings.
          106m 3s



          Reason Tests
          Failed junit tests hadoop.ipc.TestRPC
            hadoop.security.TestRaceWhenRelogin



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14773
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881843/HADOOP-14773-000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d628ace0b2d3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dadb0c2
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 38s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 25m 48s trunk passed +1 compile 26m 12s trunk passed +1 checkstyle 1m 4s trunk passed +1 mvnsite 2m 26s trunk passed +1 findbugs 2m 30s trunk passed +1 javadoc 1m 26s trunk passed       Patch Compile Tests +1 mvninstall 1m 13s the patch passed +1 compile 20m 42s the patch passed +1 javac 20m 42s the patch passed +1 checkstyle 1m 1s the patch passed +1 mvnsite 2m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 56s the patch passed +1 javadoc 1m 27s the patch passed       Other Tests -1 unit 12m 18s hadoop-common in the patch failed. +1 asflicense 0m 55s The patch does not generate ASF License warnings. 106m 3s Reason Tests Failed junit tests hadoop.ipc.TestRPC   hadoop.security.TestRaceWhenRelogin Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14773 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12881843/HADOOP-14773-000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d628ace0b2d3 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dadb0c2 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13036/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Tweaked ZKRMStateStore. I'll leave CuratorService for another JIRA as it has some complexity.

          Show
          elgoiri Íñigo Goiri added a comment - Tweaked ZKRMStateStore . I'll leave CuratorService for another JIRA as it has some complexity.
          Hide
          subru Subru Krishnan added a comment - - edited

          Thanks Íñigo Goiri for the patch. It looks fairly straightforward, one minor comment - can you update ZKRMStateStore/CuratorService etc to use the ZKCuratorManager.

          Otherwise +1 (pending Yetus).

          Show
          subru Subru Krishnan added a comment - - edited Thanks Íñigo Goiri for the patch. It looks fairly straightforward, one minor comment - can you update ZKRMStateStore/CuratorService etc to use the ZKCuratorManager . Otherwise +1 (pending Yetus).

            People

            • Assignee:
              elgoiri Íñigo Goiri
              Reporter:
              elgoiri Íñigo Goiri
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development