Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None

      Description

      As discussed in the mailing list, we’d like to introduce Apache Kerby into Hadoop. Initially it’s good to start with upgrading Hadoop MiniKDC with Kerby offerings. Apache Kerby (https://github.com/apache/directory-kerby), as an Apache Directory sub project, is a Java Kerberos binding. It provides a SimpleKDC server that borrowed ideas from MiniKDC and implemented all the facilities existing in MiniKDC. Currently MiniKDC depends on the old Kerberos implementation in Directory Server project, but the implementation is stopped being maintained. Directory community has a plan to replace the implementation using Kerby. MiniKDC can use Kerby SimpleKDC directly to avoid depending on the full of Directory project. Kerby also provides nice identity backends such as the lightweight memory based one and the very simple json one for easy development and test environments.

      1. HADOOP-12911-v1.patch
        44 kB
        Jiajia Li
      2. HADOOP-12911-v2.patch
        44 kB
        Jiajia Li
      3. HADOOP-12911-v3.patch
        44 kB
        Jiajia Li
      4. HADOOP-12911-v4.patch
        44 kB
        Jiajia Li
      5. HADOOP-12911-v5.patch
        49 kB
        Jiajia Li
      6. HADOOP-12911-v6.patch
        48 kB
        Jiajia Li
      7. HADOOP-12911-v7.patch
        45 kB
        Jiajia Li
      8. HaDOOP-12911-v8.patch
        45 kB
        Jiajia Li

        Issue Links

          Activity

          Hide
          jiajia Jiajia Li added a comment -

          Upload the patch v1.
          In this patch, replaced the Directory sever project dependencies with Kerby 1.0.0-RC2 modules. Please help to review, thanks.

          Show
          jiajia Jiajia Li added a comment - Upload the patch v1. In this patch, replaced the Directory sever project dependencies with Kerby 1.0.0-RC2 modules. Please help to review, thanks.
          Hide
          jiajia Jiajia Li added a comment -

          The MiniKdc performance will speed up after upgrading.
          I've run TestMiniKdc as example in my desktop:
          before: Time elapsed: 58.185 sec
          after: Time elapsed: 0.77 sec

          Show
          jiajia Jiajia Li added a comment - The MiniKdc performance will speed up after upgrading. I've run TestMiniKdc as example in my desktop: before: Time elapsed: 58.185 sec after: Time elapsed: 0.77 sec
          Hide
          drankye Kai Zheng added a comment -

          Thanks Jiajia Li for the great work. The performance speedup sounds great, even though it's only relevant to the tests.

          Show
          drankye Kai Zheng added a comment - Thanks Jiajia Li for the great work. The performance speedup sounds great, even though it's only relevant to the tests.
          Hide
          stevel@apache.org Steve Loughran added a comment -
          1. could you submit a new patch that doesn't re-order imports. Ideally, turn that feature off in your IDE. It makes merging patches and cherry picking very hard.
          1. Can you supplement this patch with (linked) JIRAs for HDFS and YARN projects, which contain the same patches as here. This ensures that the new miniKDC is tested in these modules.
          Show
          stevel@apache.org Steve Loughran added a comment - could you submit a new patch that doesn't re-order imports. Ideally, turn that feature off in your IDE. It makes merging patches and cherry picking very hard. Can you supplement this patch with (linked) JIRAs for HDFS and YARN projects, which contain the same patches as here. This ensures that the new miniKDC is tested in these modules.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 22s Maven dependency ordering for branch
          +1 mvninstall 9m 32s trunk passed
          +1 compile 10m 59s trunk passed with JDK v1.8.0_74
          +1 compile 9m 48s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 29s trunk passed
          +1 mvnsite 4m 50s trunk passed
          +1 mvneclipse 2m 0s trunk passed
          0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project
          +1 findbugs 7m 32s trunk passed
          +1 javadoc 4m 37s trunk passed with JDK v1.8.0_74
          +1 javadoc 5m 11s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 32s Maven dependency ordering for patch
          -1 mvninstall 0m 15s hadoop-auth in the patch failed.
          -1 mvninstall 0m 25s hadoop-common in the patch failed.
          -1 compile 0m 31s root in the patch failed with JDK v1.8.0_74.
          -1 javac 0m 31s root in the patch failed with JDK v1.8.0_74.
          -1 compile 0m 26s root in the patch failed with JDK v1.7.0_95.
          -1 javac 0m 26s root in the patch failed with JDK v1.7.0_95.
          -1 checkstyle 1m 25s root: patch generated 14 new + 105 unchanged - 9 fixed = 119 total (was 114)
          -1 mvnsite 0m 17s hadoop-auth in the patch failed.
          -1 mvnsite 0m 35s hadoop-common in the patch failed.
          +1 mvneclipse 1m 34s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project
          -1 findbugs 0m 44s hadoop-common-project/hadoop-minikdc generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          -1 findbugs 0m 13s hadoop-auth in the patch failed.
          -1 findbugs 0m 25s hadoop-common in the patch failed.
          -1 javadoc 6m 33s hadoop-common-project_hadoop-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1)
          +1 javadoc 4m 29s the patch passed with JDK v1.8.0_74
          -1 javadoc 12m 19s hadoop-common-project_hadoop-common-jdk1.7.0_95 with JDK v1.7.0_95 generated 4 new + 13 unchanged - 0 fixed = 17 total (was 13)
          +1 javadoc 5m 21s the patch passed with JDK v1.7.0_95
          +1 unit 0m 6s hadoop-project in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74.
          -1 unit 0m 16s hadoop-auth in the patch failed with JDK v1.8.0_74.
          -1 unit 0m 26s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_74.
          -1 unit 83m 25s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 71m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 15s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          -1 unit 0m 15s hadoop-auth in the patch failed with JDK v1.7.0_95.
          -1 unit 0m 24s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.7.0_95.
          -1 unit 77m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 unit 72m 55s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 35s Patch does not generate ASF License warnings.
          398m 22s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-minikdc
            Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java:[line 442]
            Exceptional return value of java.io.File.delete() ignored in org.apache.hadoop.minikdc.MiniKdc.createPrincipal(File, String[]) At MiniKdc.java:ignored in org.apache.hadoop.minikdc.MiniKdc.createPrincipal(File, String[]) At MiniKdc.java:[line 443]
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.fs.TestHdfsNativeCodeLoader
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.fs.TestHdfsNativeCodeLoader
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793495/HADOOP-12911-v1.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux e87a8fdc7c7f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5de848c
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-auth.txt
          mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.8.0_74.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.8.0_74.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-checkstyle-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-auth.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
          javadoc hadoop-common-project_hadoop-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          javadoc hadoop-common-project_hadoop-common-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-auth-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-auth-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 22s Maven dependency ordering for branch +1 mvninstall 9m 32s trunk passed +1 compile 10m 59s trunk passed with JDK v1.8.0_74 +1 compile 9m 48s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 29s trunk passed +1 mvnsite 4m 50s trunk passed +1 mvneclipse 2m 0s trunk passed 0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project +1 findbugs 7m 32s trunk passed +1 javadoc 4m 37s trunk passed with JDK v1.8.0_74 +1 javadoc 5m 11s trunk passed with JDK v1.7.0_95 0 mvndep 0m 32s Maven dependency ordering for patch -1 mvninstall 0m 15s hadoop-auth in the patch failed. -1 mvninstall 0m 25s hadoop-common in the patch failed. -1 compile 0m 31s root in the patch failed with JDK v1.8.0_74. -1 javac 0m 31s root in the patch failed with JDK v1.8.0_74. -1 compile 0m 26s root in the patch failed with JDK v1.7.0_95. -1 javac 0m 26s root in the patch failed with JDK v1.7.0_95. -1 checkstyle 1m 25s root: patch generated 14 new + 105 unchanged - 9 fixed = 119 total (was 114) -1 mvnsite 0m 17s hadoop-auth in the patch failed. -1 mvnsite 0m 35s hadoop-common in the patch failed. +1 mvneclipse 1m 34s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project -1 findbugs 0m 44s hadoop-common-project/hadoop-minikdc generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) -1 findbugs 0m 13s hadoop-auth in the patch failed. -1 findbugs 0m 25s hadoop-common in the patch failed. -1 javadoc 6m 33s hadoop-common-project_hadoop-common-jdk1.8.0_74 with JDK v1.8.0_74 generated 4 new + 1 unchanged - 0 fixed = 5 total (was 1) +1 javadoc 4m 29s the patch passed with JDK v1.8.0_74 -1 javadoc 12m 19s hadoop-common-project_hadoop-common-jdk1.7.0_95 with JDK v1.7.0_95 generated 4 new + 13 unchanged - 0 fixed = 17 total (was 13) +1 javadoc 5m 21s the patch passed with JDK v1.7.0_95 +1 unit 0m 6s hadoop-project in the patch passed with JDK v1.8.0_74. +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74. -1 unit 0m 16s hadoop-auth in the patch failed with JDK v1.8.0_74. -1 unit 0m 26s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_74. -1 unit 83m 25s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 71m 47s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 15s hadoop-minikdc in the patch passed with JDK v1.7.0_95. -1 unit 0m 15s hadoop-auth in the patch failed with JDK v1.7.0_95. -1 unit 0m 24s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.7.0_95. -1 unit 77m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 unit 72m 55s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 398m 22s Reason Tests FindBugs module:hadoop-common-project/hadoop-minikdc   Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java: [line 442]   Exceptional return value of java.io.File.delete() ignored in org.apache.hadoop.minikdc.MiniKdc.createPrincipal(File, String[]) At MiniKdc.java:ignored in org.apache.hadoop.minikdc.MiniKdc.createPrincipal(File, String[]) At MiniKdc.java: [line 443] JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.fs.TestHdfsNativeCodeLoader   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.fs.TestHdfsNativeCodeLoader   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793495/HADOOP-12911-v1.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux e87a8fdc7c7f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5de848c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-auth.txt mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.8.0_74.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.8.0_74.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-checkstyle-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-auth.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt javadoc hadoop-common-project_hadoop-common-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt javadoc hadoop-common-project_hadoop-common-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-auth-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-auth-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8850/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jiajia Jiajia Li added a comment -

          Thanks for your suggest, I will update the patch and add the links.

          Show
          jiajia Jiajia Li added a comment - Thanks for your suggest, I will update the patch and add the links.
          Hide
          jiajia Jiajia Li added a comment -

          Upload the v2.
          1. Refine the imports according Steve's advice.
          2. Fix the TestClientRMTokens failure.
          3. Fix checkstyles.

          Show
          jiajia Jiajia Li added a comment - Upload the v2. 1. Refine the imports according Steve's advice. 2. Fix the TestClientRMTokens failure. 3. Fix checkstyles.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 10 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 25s trunk passed
          +1 compile 5m 34s trunk passed with JDK v1.8.0_74
          +1 compile 6m 32s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 3m 27s trunk passed
          +1 mvneclipse 1m 33s trunk passed
          0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project
          +1 findbugs 5m 48s trunk passed
          +1 javadoc 2m 57s trunk passed with JDK v1.8.0_74
          +1 javadoc 4m 2s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 22s Maven dependency ordering for patch
          +1 mvninstall 2m 42s the patch passed
          +1 compile 5m 31s the patch passed with JDK v1.8.0_74
          -1 javac 8m 35s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 1 new + 738 unchanged - 0 fixed = 739 total (was 738)
          +1 javac 5m 31s the patch passed
          +1 compile 6m 32s the patch passed with JDK v1.7.0_95
          -1 javac 15m 7s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 734 unchanged - 0 fixed = 735 total (was 734)
          +1 javac 6m 32s the patch passed
          +1 checkstyle 1m 4s root: patch generated 0 new + 88 unchanged - 14 fixed = 88 total (was 102)
          +1 mvnsite 3m 22s the patch passed
          +1 mvneclipse 1m 24s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project
          -1 findbugs 0m 37s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 2m 56s the patch passed with JDK v1.8.0_74
          +1 javadoc 3m 59s the patch passed with JDK v1.7.0_95
          +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74.
          +1 unit 2m 31s hadoop-auth in the patch passed with JDK v1.8.0_74.
          +1 unit 6m 32s hadoop-common in the patch passed with JDK v1.8.0_74.
          +1 unit 1m 24s hadoop-kms in the patch passed with JDK v1.8.0_74.
          -1 unit 57m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 65m 41s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 41s hadoop-auth in the patch passed with JDK v1.7.0_95.
          -1 unit 6m 28s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.7.0_95.
          +1 unit 52m 18s hadoop-hdfs in the patch passed with JDK v1.7.0_95.
          -1 unit 66m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          339m 29s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-minikdc
            Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java:[line 441]
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.TestDeleteRace
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.TestHFlush
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793738/HADOOP-12911-v2.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 293ae35e6391 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 605fdcb
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 10 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 25s trunk passed +1 compile 5m 34s trunk passed with JDK v1.8.0_74 +1 compile 6m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 4s trunk passed +1 mvnsite 3m 27s trunk passed +1 mvneclipse 1m 33s trunk passed 0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project +1 findbugs 5m 48s trunk passed +1 javadoc 2m 57s trunk passed with JDK v1.8.0_74 +1 javadoc 4m 2s trunk passed with JDK v1.7.0_95 0 mvndep 0m 22s Maven dependency ordering for patch +1 mvninstall 2m 42s the patch passed +1 compile 5m 31s the patch passed with JDK v1.8.0_74 -1 javac 8m 35s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 1 new + 738 unchanged - 0 fixed = 739 total (was 738) +1 javac 5m 31s the patch passed +1 compile 6m 32s the patch passed with JDK v1.7.0_95 -1 javac 15m 7s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 734 unchanged - 0 fixed = 735 total (was 734) +1 javac 6m 32s the patch passed +1 checkstyle 1m 4s root: patch generated 0 new + 88 unchanged - 14 fixed = 88 total (was 102) +1 mvnsite 3m 22s the patch passed +1 mvneclipse 1m 24s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project -1 findbugs 0m 37s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 2m 56s the patch passed with JDK v1.8.0_74 +1 javadoc 3m 59s the patch passed with JDK v1.7.0_95 +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_74. +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74. +1 unit 2m 31s hadoop-auth in the patch passed with JDK v1.8.0_74. +1 unit 6m 32s hadoop-common in the patch passed with JDK v1.8.0_74. +1 unit 1m 24s hadoop-kms in the patch passed with JDK v1.8.0_74. -1 unit 57m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 65m 41s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 2m 41s hadoop-auth in the patch passed with JDK v1.7.0_95. -1 unit 6m 28s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.7.0_95. +1 unit 52m 18s hadoop-hdfs in the patch passed with JDK v1.7.0_95. -1 unit 66m 56s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 339m 29s Reason Tests FindBugs module:hadoop-common-project/hadoop-minikdc   Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java: [line 441] JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.TestDeleteRace   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.TestHFlush   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793738/HADOOP-12911-v2.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 293ae35e6391 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 605fdcb Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8861/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jiajia Jiajia Li added a comment -

          Hi Steve, do you mean the JIRAs created to upgrade minikdc in HDFS and YARN? We have just started this work, and no patches in other JIRAs.

          Show
          jiajia Jiajia Li added a comment - Hi Steve, do you mean the JIRAs created to upgrade minikdc in HDFS and YARN? We have just started this work, and no patches in other JIRAs.
          Hide
          drankye Kai Zheng added a comment -

          I guess Steve meant that you create new JIRAs in HDFS and YARN and upload this patch to them to trigger the more complete Jenkins building. Steve please help clarify if I misunderstood. Thanks.

          Show
          drankye Kai Zheng added a comment - I guess Steve meant that you create new JIRAs in HDFS and YARN and upload this patch to them to trigger the more complete Jenkins building. Steve please help clarify if I misunderstood. Thanks.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          that's right ... as they use the minikdc, it's good to see if they cope with the change

          I should warn that I'm without the time and skill to really review this; you're going to need to find someone with both to really work this through

          Show
          stevel@apache.org Steve Loughran added a comment - that's right ... as they use the minikdc, it's good to see if they cope with the change I should warn that I'm without the time and skill to really review this; you're going to need to find someone with both to really work this through
          Hide
          jiajia Jiajia Li added a comment -

          Thanks Kai and Steve make it clear, I will create the jiras in HDFS and YARN after updating the patch in this jira.

          Show
          jiajia Jiajia Li added a comment - Thanks Kai and Steve make it clear, I will create the jiras in HDFS and YARN after updating the patch in this jira.
          Hide
          jiajia Jiajia Li added a comment -

          Upload the v3.
          Fix findbugs.
          Junit tests failure can not be reproduced in local.

          Show
          jiajia Jiajia Li added a comment - Upload the v3. Fix findbugs. Junit tests failure can not be reproduced in local.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 10 new or modified test files.
          0 mvndep 0m 25s Maven dependency ordering for branch
          +1 mvninstall 6m 58s trunk passed
          +1 compile 6m 49s trunk passed with JDK v1.8.0_74
          +1 compile 6m 51s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 3m 31s trunk passed
          +1 mvneclipse 1m 32s trunk passed
          0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project
          +1 findbugs 5m 52s trunk passed
          +1 javadoc 3m 17s trunk passed with JDK v1.8.0_74
          +1 javadoc 4m 10s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 47s the patch passed
          +1 compile 6m 13s the patch passed with JDK v1.8.0_74
          -1 javac 9m 14s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 1 new + 738 unchanged - 0 fixed = 739 total (was 738)
          +1 javac 6m 13s the patch passed
          +1 compile 6m 52s the patch passed with JDK v1.7.0_95
          -1 javac 16m 7s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 734 unchanged - 0 fixed = 735 total (was 734)
          +1 javac 6m 52s the patch passed
          +1 checkstyle 1m 4s root: patch generated 0 new + 88 unchanged - 14 fixed = 88 total (was 102)
          +1 mvnsite 3m 25s the patch passed
          +1 mvneclipse 1m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project
          -1 findbugs 0m 41s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 3m 20s the patch passed with JDK v1.8.0_74
          +1 javadoc 4m 26s the patch passed with JDK v1.7.0_95
          +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74.
          +1 unit 2m 34s hadoop-auth in the patch passed with JDK v1.8.0_74.
          -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.8.0_74.
          +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_74.
          -1 unit 61m 38s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 67m 43s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74.
          +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 15s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 40s hadoop-auth in the patch passed with JDK v1.7.0_95.
          -1 unit 6m 56s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 35s hadoop-kms in the patch passed with JDK v1.7.0_95.
          -1 unit 58m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 unit 69m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 29s Patch generated 2 ASF License warnings.
          361m 11s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-minikdc
            Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java:[line 441]
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795147/HADOOP-12911-v3.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 1cafc429be20 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 19b645c
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 10 new or modified test files. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 6m 58s trunk passed +1 compile 6m 49s trunk passed with JDK v1.8.0_74 +1 compile 6m 51s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 3m 31s trunk passed +1 mvneclipse 1m 32s trunk passed 0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project +1 findbugs 5m 52s trunk passed +1 javadoc 3m 17s trunk passed with JDK v1.8.0_74 +1 javadoc 4m 10s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 47s the patch passed +1 compile 6m 13s the patch passed with JDK v1.8.0_74 -1 javac 9m 14s root-jdk1.8.0_74 with JDK v1.8.0_74 generated 1 new + 738 unchanged - 0 fixed = 739 total (was 738) +1 javac 6m 13s the patch passed +1 compile 6m 52s the patch passed with JDK v1.7.0_95 -1 javac 16m 7s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 734 unchanged - 0 fixed = 735 total (was 734) +1 javac 6m 52s the patch passed +1 checkstyle 1m 4s root: patch generated 0 new + 88 unchanged - 14 fixed = 88 total (was 102) +1 mvnsite 3m 25s the patch passed +1 mvneclipse 1m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project -1 findbugs 0m 41s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 3m 20s the patch passed with JDK v1.8.0_74 +1 javadoc 4m 26s the patch passed with JDK v1.7.0_95 +1 unit 0m 9s hadoop-project in the patch passed with JDK v1.8.0_74. +1 unit 0m 13s hadoop-minikdc in the patch passed with JDK v1.8.0_74. +1 unit 2m 34s hadoop-auth in the patch passed with JDK v1.8.0_74. -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.8.0_74. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_74. -1 unit 61m 38s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 67m 43s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_74. +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 15s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 2m 40s hadoop-auth in the patch passed with JDK v1.7.0_95. -1 unit 6m 56s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 35s hadoop-kms in the patch passed with JDK v1.7.0_95. -1 unit 58m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 unit 69m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 29s Patch generated 2 ASF License warnings. 361m 11s Reason Tests FindBugs module:hadoop-common-project/hadoop-minikdc   Inconsistent synchronization of org.apache.hadoop.minikdc.MiniKdc.simpleKdc; locked 85% of time Unsynchronized access at MiniKdc.java:85% of time Unsynchronized access at MiniKdc.java: [line 441] JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork   hadoop.hdfs.server.datanode.TestDataNodeMetrics   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_74 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795147/HADOOP-12911-v3.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 1cafc429be20 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 19b645c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_74: https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_74.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8910/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jiajia Jiajia Li added a comment -

          Upload v4.
          1.Fix findbugs.
          2.javac warning "KeyStoreTestUtil.java:[52,28] [deprecation] X509V1CertificateGenerator in org.bouncycastle.x509 has been deprecated"
          It is because after removing Apache Directory dependencies, will use the "bcprov-jdk16" rather than "bcprov-jdk15". So the warning can be ignored.
          3. Test failures are spurious.

          Show
          jiajia Jiajia Li added a comment - Upload v4. 1.Fix findbugs. 2.javac warning "KeyStoreTestUtil.java: [52,28] [deprecation] X509V1CertificateGenerator in org.bouncycastle.x509 has been deprecated" It is because after removing Apache Directory dependencies, will use the "bcprov-jdk16" rather than "bcprov-jdk15". So the warning can be ignored. 3. Test failures are spurious.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I really like this code; I like how much simpler the POMs and the internals are getting. We will need people who understand Kerberos to review it though; I don't count myself as one.

          MiniKdc.java

          1. This could be a good time to make MiniKDC a subclass of AbstractService, though it may (will?) break external users. Perhaps we could have a MiniKDC service, which the existing MiniKDC code instantiated on its existing lifecycle.

          2. If not done already, can the miniconf prefer TCP over UDP? It fails faster.

          3. Why does /MiniKdc reset "sun.security.krb5.debug"? If set, I'd like it to stay that way. You could use is current value as the default when reading
          the configuration .

          4.

          InputStream is = getResourceAsStream("minikdc-krb5.conf");
          

          Add a Preconditions check that this isn't null.

          5. need a check after transport = conf.getProperty(TRANSPORT); for a null value?

          6. Teardown. Again, I'd like the JVM properties to be left unaltered. At the very least: remember their originals. KDiag manipulates these in its own production code.

          in TestClientRMTokens.java the realm is changed. does that make the comment on line 122 incorrect?

          Show
          stevel@apache.org Steve Loughran added a comment - I really like this code; I like how much simpler the POMs and the internals are getting. We will need people who understand Kerberos to review it though; I don't count myself as one. MiniKdc.java 1. This could be a good time to make MiniKDC a subclass of AbstractService, though it may (will?) break external users. Perhaps we could have a MiniKDC service, which the existing MiniKDC code instantiated on its existing lifecycle. 2. If not done already, can the miniconf prefer TCP over UDP? It fails faster. 3. Why does /MiniKdc reset "sun.security.krb5.debug"? If set, I'd like it to stay that way. You could use is current value as the default when reading the configuration . 4. InputStream is = getResourceAsStream( "minikdc-krb5.conf" ); Add a Preconditions check that this isn't null. 5. need a check after transport = conf.getProperty(TRANSPORT); for a null value? 6. Teardown. Again, I'd like the JVM properties to be left unaltered. At the very least: remember their originals. KDiag manipulates these in its own production code. in TestClientRMTokens.java the realm is changed. does that make the comment on line 122 incorrect?
          Hide
          drankye Kai Zheng added a comment -

          We will need people who understand Kerberos to review it though

          I understand something about Kerberos and am a bit familiar with Kerby, will take some time to review this and hope it helps.

          Show
          drankye Kai Zheng added a comment - We will need people who understand Kerberos to review it though I understand something about Kerberos and am a bit familiar with Kerby, will take some time to review this and hope it helps.
          Hide
          jiajia Jiajia Li added a comment -

          Hi Steve, thanks for your review and comments.
          1. It's a great idea to make MiniKDC the subclass of AbstractService.
          2. I think the TCP is set as the defaut transport. line 179:

          DEFAULT_CONFIG.setProperty(TRANSPORT, "TCP");
          

          Did I get your point?
          3.

          System.setProperty(SUN_SECURITY_KRB5_DEBUG, conf.getProperty(DEBUG, "false"));
          

          The same as the the original MiniKDC, but I agree you point to use the current value as the default.
          4,5,6. Yes, I will update the patch.
          Thanks again.

          Show
          jiajia Jiajia Li added a comment - Hi Steve, thanks for your review and comments. 1. It's a great idea to make MiniKDC the subclass of AbstractService. 2. I think the TCP is set as the defaut transport. line 179: DEFAULT_CONFIG.setProperty(TRANSPORT, "TCP" ); Did I get your point? 3. System .setProperty(SUN_SECURITY_KRB5_DEBUG, conf.getProperty(DEBUG, " false " )); The same as the the original MiniKDC, but I agree you point to use the current value as the default. 4,5,6. Yes, I will update the patch. Thanks again.
          Hide
          drankye Kai Zheng added a comment -

          This could be a good time to make MiniKDC a subclass of AbstractService, though it may (will?) break external users. Perhaps we could have a MiniKDC service, which the existing MiniKDC code instantiated on its existing lifecycle.

          Sounds good to have a MiniKDC service that extends AbstractService, in addition to the MiniKDC construct. The both are valid for respective environments. Should we worry about the breaking of external users, if we target this for Hadoop 3.0, marked as incompatible changes? If acceptable, it will allow to clean up all the unnecessary configurations (they're exposed publicly and may be used) and interfaces, making it more easy to use.

          Show
          drankye Kai Zheng added a comment - This could be a good time to make MiniKDC a subclass of AbstractService, though it may (will?) break external users. Perhaps we could have a MiniKDC service, which the existing MiniKDC code instantiated on its existing lifecycle. Sounds good to have a MiniKDC service that extends AbstractService, in addition to the MiniKDC construct. The both are valid for respective environments. Should we worry about the breaking of external users, if we target this for Hadoop 3.0, marked as incompatible changes? If acceptable, it will allow to clean up all the unnecessary configurations (they're exposed publicly and may be used) and interfaces, making it more easy to use.
          Hide
          jiajia Jiajia Li added a comment -

          Upload v5.
          Changes are according to Steve's comments except "making MiniKDC a subclass of AbstractService" and I will implemented it in new JIRA.

          Show
          jiajia Jiajia Li added a comment - Upload v5. Changes are according to Steve's comments except "making MiniKDC a subclass of AbstractService" and I will implemented it in new JIRA.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 7m 18s trunk passed
          +1 compile 7m 5s trunk passed with JDK v1.8.0_77
          +1 compile 7m 4s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 3m 28s trunk passed
          +1 mvneclipse 1m 33s trunk passed
          0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project
          +1 findbugs 5m 51s trunk passed
          +1 javadoc 3m 8s trunk passed with JDK v1.8.0_77
          +1 javadoc 4m 8s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 29s Maven dependency ordering for patch
          +1 mvninstall 2m 40s the patch passed
          -1 compile 3m 44s root in the patch failed with JDK v1.8.0_77.
          -1 javac 3m 44s root in the patch failed with JDK v1.8.0_77.
          -1 compile 4m 5s root in the patch failed with JDK v1.7.0_95.
          -1 javac 4m 5s root in the patch failed with JDK v1.7.0_95.
          -1 checkstyle 1m 6s root: patch generated 2 new + 86 unchanged - 15 fixed = 88 total (was 101)
          +1 mvnsite 3m 20s the patch passed
          +1 mvneclipse 1m 20s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project
          +1 findbugs 7m 1s the patch passed
          +1 javadoc 2m 57s the patch passed with JDK v1.8.0_77
          +1 javadoc 3m 56s the patch passed with JDK v1.7.0_95
          +1 unit 0m 7s hadoop-project in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 12s hadoop-minikdc in the patch passed with JDK v1.8.0_77.
          +1 unit 2m 31s hadoop-auth in the patch passed with JDK v1.8.0_77.
          -1 unit 7m 19s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 1m 24s hadoop-kms in the patch passed with JDK v1.8.0_77.
          +1 unit 56m 24s hadoop-hdfs in the patch passed with JDK v1.8.0_77.
          -1 unit 59m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 38s hadoop-auth in the patch passed with JDK v1.7.0_95.
          -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.7.0_95.
          -1 unit 53m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 unit 60m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          -1 asflicense 0m 23s Patch generated 2 ASF License warnings.
          327m 1s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796249/HADOOP-12911-v5.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux d2ebdb1d558a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 32c0c3e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.8.0_77.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.8.0_77.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 7m 18s trunk passed +1 compile 7m 5s trunk passed with JDK v1.8.0_77 +1 compile 7m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 4s trunk passed +1 mvnsite 3m 28s trunk passed +1 mvneclipse 1m 33s trunk passed 0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project +1 findbugs 5m 51s trunk passed +1 javadoc 3m 8s trunk passed with JDK v1.8.0_77 +1 javadoc 4m 8s trunk passed with JDK v1.7.0_95 0 mvndep 0m 29s Maven dependency ordering for patch +1 mvninstall 2m 40s the patch passed -1 compile 3m 44s root in the patch failed with JDK v1.8.0_77. -1 javac 3m 44s root in the patch failed with JDK v1.8.0_77. -1 compile 4m 5s root in the patch failed with JDK v1.7.0_95. -1 javac 4m 5s root in the patch failed with JDK v1.7.0_95. -1 checkstyle 1m 6s root: patch generated 2 new + 86 unchanged - 15 fixed = 88 total (was 101) +1 mvnsite 3m 20s the patch passed +1 mvneclipse 1m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project +1 findbugs 7m 1s the patch passed +1 javadoc 2m 57s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 56s the patch passed with JDK v1.7.0_95 +1 unit 0m 7s hadoop-project in the patch passed with JDK v1.8.0_77. +1 unit 0m 12s hadoop-minikdc in the patch passed with JDK v1.8.0_77. +1 unit 2m 31s hadoop-auth in the patch passed with JDK v1.8.0_77. -1 unit 7m 19s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 1m 24s hadoop-kms in the patch passed with JDK v1.8.0_77. +1 unit 56m 24s hadoop-hdfs in the patch passed with JDK v1.8.0_77. -1 unit 59m 5s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 2m 38s hadoop-auth in the patch passed with JDK v1.7.0_95. -1 unit 7m 12s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.7.0_95. -1 unit 53m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 unit 60m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 23s Patch generated 2 ASF License warnings. 327m 1s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.util.TestNativeLibraryChecker Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12796249/HADOOP-12911-v5.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux d2ebdb1d558a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 32c0c3e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.8.0_77.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.8.0_77.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-compile-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8984/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          drankye Kai Zheng added a comment -

          Some comments:
          1. Much cleaned now in pom.xml files. An issue was noted:

          +    <dependency>
          +      <groupId>org.apache.kerby</groupId>
          +      <artifactId>kerb-simplekdc</artifactId>
          +      <version>1.0.0-RC2</version>
          +      <exclusions>
          +        <exclusion>
          +          <groupId>org.bouncycastle</groupId>
          +          <artifactId>bcprov-jdk15on</artifactId>
          +        </exclusion>
          +      </exclusions>
          +    </dependency>
          

          The RC2 release of Kerby relied on bouncycastle in mistake, and the dependency will be cleaned up in the next release. Could you fire an issue to mark this as a follow-on, once the new Kerby release is available, then this can be cleaned up accordingly?
          2. In TestKMS, looks like the following change isn't relevant.

          -        conf.setInt(KeyProvider.DEFAULT_BITLENGTH_NAME, 64);
          +        conf.setInt(KeyProvider.DEFAULT_BITLENGTH_NAME, 128);
          

          3. Not sure why the following are removed. Are you sure the two configurations are not usable? The krb5.conf item can be set in case tests will rely on the env variable; the debug one can be set by tests to allow verbose output or not.

          - * MiniKdc sets 2 System properties when started and un-sets them when stopped:
          - * <ul>
          - *   <li>java.security.krb5.conf: set to the MiniKDC real/host/port</li>
          - *   <li>sun.security.krb5.debug: set to the debug value provided in the
          - *   configuration</li>
          - * </ul>
          

          4. In MiniKDC, why it needs resetDefaultRealm? I thought the desired realm can be set previously, then SimpleKDC will take care of it.
          5. I think Kerby SimpleKDC can be improved to do the synchronized in itself, rather than let MiniKDC bother to do it.

          +    synchronized (this) {
          +      simpleKdc.createPrincipals(principals);
          +    }
          
          Show
          drankye Kai Zheng added a comment - Some comments: 1. Much cleaned now in pom.xml files. An issue was noted: + <dependency> + <groupId>org.apache.kerby</groupId> + <artifactId>kerb-simplekdc</artifactId> + <version>1.0.0-RC2</version> + <exclusions> + <exclusion> + <groupId>org.bouncycastle</groupId> + <artifactId>bcprov-jdk15on</artifactId> + </exclusion> + </exclusions> + </dependency> The RC2 release of Kerby relied on bouncycastle in mistake, and the dependency will be cleaned up in the next release. Could you fire an issue to mark this as a follow-on, once the new Kerby release is available, then this can be cleaned up accordingly? 2. In TestKMS , looks like the following change isn't relevant. - conf.setInt(KeyProvider.DEFAULT_BITLENGTH_NAME, 64); + conf.setInt(KeyProvider.DEFAULT_BITLENGTH_NAME, 128); 3. Not sure why the following are removed. Are you sure the two configurations are not usable? The krb5.conf item can be set in case tests will rely on the env variable; the debug one can be set by tests to allow verbose output or not. - * MiniKdc sets 2 System properties when started and un-sets them when stopped: - * <ul> - * <li>java.security.krb5.conf: set to the MiniKDC real/host/port</li> - * <li>sun.security.krb5.debug: set to the debug value provided in the - * configuration</li> - * </ul> 4. In MiniKDC, why it needs resetDefaultRealm? I thought the desired realm can be set previously, then SimpleKDC will take care of it. 5. I think Kerby SimpleKDC can be improved to do the synchronized in itself, rather than let MiniKDC bother to do it. + synchronized ( this ) { + simpleKdc.createPrincipals(principals); + }
          Hide
          jiajia Jiajia Li added a comment -

          Hi Kai,
          Thanks for your review.
          1. Yes, the dependency can be cleaned up in the next release
          2. If the KeyProvider.DEFAULT_BITLENGTH_NAME be 64, will with some exception: java.security.InvalidParameterException: Wrong keysize: must be equal to 128, 192 or 256
          3. I will revert the removing because of the latest patch with some compile error. They will not be removed in the next patch.
          4. The krb5.conf will used by other tests(which not use the MiniKDC), such as "TestClientRMTokens", so "default_realm" can not be set by SimpleKDC. But after changing the code of SimpleKDC, I think resetDefaultRealm can be removed.
          5. You are right, this can be improved in Kerby SimpleKDC.

          Show
          jiajia Jiajia Li added a comment - Hi Kai, Thanks for your review. 1. Yes, the dependency can be cleaned up in the next release 2. If the KeyProvider.DEFAULT_BITLENGTH_NAME be 64, will with some exception: java.security.InvalidParameterException: Wrong keysize: must be equal to 128, 192 or 256 3. I will revert the removing because of the latest patch with some compile error. They will not be removed in the next patch. 4. The krb5.conf will used by other tests(which not use the MiniKDC), such as "TestClientRMTokens", so "default_realm" can not be set by SimpleKDC. But after changing the code of SimpleKDC, I think resetDefaultRealm can be removed. 5. You are right, this can be improved in Kerby SimpleKDC.
          Hide
          jiajia Jiajia Li added a comment -

          Upload v6.
          Fix the compile error.

          Show
          jiajia Jiajia Li added a comment - Upload v6. Fix the compile error.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          BTW, will this code work on JVMs without the full JCE extensions? Some of the ASF build VMs are like that

          Show
          stevel@apache.org Steve Loughran added a comment - BTW, will this code work on JVMs without the full JCE extensions? Some of the ASF build VMs are like that
          Hide
          drankye Kai Zheng added a comment -

          I'm not sure about what's the full JCE extensions are like, but guess basically DES, RC, and AES ciphers should be available. If it's totally lack any encryption cipher, then camellia related kerberos encryption types can be used in the setups for MiniKDC since Kerby equips itself with camellia ciphers, not relying on JRE.

          Show
          drankye Kai Zheng added a comment - I'm not sure about what's the full JCE extensions are like, but guess basically DES, RC, and AES ciphers should be available. If it's totally lack any encryption cipher, then camellia related kerberos encryption types can be used in the setups for MiniKDC since Kerby equips itself with camellia ciphers, not relying on JRE.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 1m 4s Maven dependency ordering for branch
          +1 mvninstall 7m 3s trunk passed
          +1 compile 6m 36s trunk passed with JDK v1.8.0_77
          +1 compile 7m 24s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 11s trunk passed
          +1 mvnsite 3m 42s trunk passed
          +1 mvneclipse 1m 36s trunk passed
          0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project
          +1 findbugs 6m 10s trunk passed
          +1 javadoc 3m 2s trunk passed with JDK v1.8.0_77
          +1 javadoc 4m 6s trunk passed with JDK v1.7.0_95
          0 mvndep 1m 56s Maven dependency ordering for patch
          +1 mvninstall 2m 44s the patch passed
          +1 compile 6m 2s the patch passed with JDK v1.8.0_77
          -1 javac 10m 42s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 1 new + 739 unchanged - 0 fixed = 740 total (was 739)
          +1 javac 6m 2s the patch passed
          +1 compile 6m 44s the patch passed with JDK v1.7.0_95
          -1 javac 17m 26s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 735 unchanged - 0 fixed = 736 total (was 735)
          +1 javac 6m 44s the patch passed
          +1 checkstyle 1m 4s root: patch generated 0 new + 86 unchanged - 15 fixed = 86 total (was 101)
          +1 mvnsite 3m 22s the patch passed
          +1 mvneclipse 1m 25s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project
          +1 findbugs 6m 58s the patch passed
          +1 javadoc 2m 58s the patch passed with JDK v1.8.0_77
          +1 javadoc 3m 59s the patch passed with JDK v1.7.0_95
          +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_77.
          +1 unit 0m 12s hadoop-minikdc in the patch passed with JDK v1.8.0_77.
          +1 unit 2m 32s hadoop-auth in the patch passed with JDK v1.8.0_77.
          -1 unit 16m 46s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 1m 25s hadoop-kms in the patch passed with JDK v1.8.0_77.
          -1 unit 56m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          -1 unit 58m 55s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 2m 41s hadoop-auth in the patch passed with JDK v1.7.0_95.
          -1 unit 16m 56s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 unit 1m 27s hadoop-kms in the patch passed with JDK v1.7.0_95.
          +1 unit 51m 20s hadoop-hdfs in the patch passed with JDK v1.7.0_95.
          -1 unit 60m 8s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          351m 21s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797025/HADOOP-12911-v6.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 595015ef52f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 818d6b7
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac root-jdk1.8.0_77: https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_77.txt
          javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 1m 4s Maven dependency ordering for branch +1 mvninstall 7m 3s trunk passed +1 compile 6m 36s trunk passed with JDK v1.8.0_77 +1 compile 7m 24s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 11s trunk passed +1 mvnsite 3m 42s trunk passed +1 mvneclipse 1m 36s trunk passed 0 findbugs 0m 0s Skipped branch modules with no Java source: hadoop-project +1 findbugs 6m 10s trunk passed +1 javadoc 3m 2s trunk passed with JDK v1.8.0_77 +1 javadoc 4m 6s trunk passed with JDK v1.7.0_95 0 mvndep 1m 56s Maven dependency ordering for patch +1 mvninstall 2m 44s the patch passed +1 compile 6m 2s the patch passed with JDK v1.8.0_77 -1 javac 10m 42s root-jdk1.8.0_77 with JDK v1.8.0_77 generated 1 new + 739 unchanged - 0 fixed = 740 total (was 739) +1 javac 6m 2s the patch passed +1 compile 6m 44s the patch passed with JDK v1.7.0_95 -1 javac 17m 26s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 735 unchanged - 0 fixed = 736 total (was 735) +1 javac 6m 44s the patch passed +1 checkstyle 1m 4s root: patch generated 0 new + 86 unchanged - 15 fixed = 86 total (was 101) +1 mvnsite 3m 22s the patch passed +1 mvneclipse 1m 25s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patch modules with no Java source: hadoop-project +1 findbugs 6m 58s the patch passed +1 javadoc 2m 58s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 59s the patch passed with JDK v1.7.0_95 +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_77. +1 unit 0m 12s hadoop-minikdc in the patch passed with JDK v1.8.0_77. +1 unit 2m 32s hadoop-auth in the patch passed with JDK v1.8.0_77. -1 unit 16m 46s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 1m 25s hadoop-kms in the patch passed with JDK v1.8.0_77. -1 unit 56m 37s hadoop-hdfs in the patch failed with JDK v1.8.0_77. -1 unit 58m 55s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 16s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 2m 41s hadoop-auth in the patch passed with JDK v1.7.0_95. -1 unit 16m 56s hadoop-common in the patch failed with JDK v1.7.0_95. +1 unit 1m 27s hadoop-kms in the patch passed with JDK v1.7.0_95. +1 unit 51m 20s hadoop-hdfs in the patch passed with JDK v1.7.0_95. -1 unit 60m 8s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 351m 21s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797025/HADOOP-12911-v6.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 595015ef52f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 818d6b7 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac root-jdk1.8.0_77: https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_77.txt javac root-jdk1.7.0_95: https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9025/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          drankye Kai Zheng added a comment -

          Hi Andrew Wang, this seems to simplify the implementation and usage, and may involve incompatible interface changes, for example, some configurations are just not necessary and removed. IMO, it's ok because it's only for test systems. Do you think it's good to mark this incompatible and target for the 3.0 release? Thanks.

          Show
          drankye Kai Zheng added a comment - Hi Andrew Wang , this seems to simplify the implementation and usage, and may involve incompatible interface changes, for example, some configurations are just not necessary and removed. IMO, it's ok because it's only for test systems. Do you think it's good to mark this incompatible and target for the 3.0 release? Thanks.
          Hide
          andrew.wang Andrew Wang added a comment -

          What's the downstream impact if we do these incompatible changes? I know a lot of downstreams use MiniCluster for instance, and thus perhaps MiniKdc for secure cluster testing.

          One review comment, I think we should use "test.build.dir" rather than "test.dir" to find a local dir to put files.

          Show
          andrew.wang Andrew Wang added a comment - What's the downstream impact if we do these incompatible changes? I know a lot of downstreams use MiniCluster for instance, and thus perhaps MiniKdc for secure cluster testing. One review comment, I think we should use "test.build.dir" rather than "test.dir" to find a local dir to put files.
          Hide
          drankye Kai Zheng added a comment -

          It's a good question. I thought they may need to do some simple clean up in codes (I assume it's good) to build. If we want to avoid this, we can remain and deprecate interface configuration and methods. Jiajia Li, would you do a survey on this and see how the change will affect downstream projects? Thanks.

          Show
          drankye Kai Zheng added a comment - It's a good question. I thought they may need to do some simple clean up in codes (I assume it's good) to build. If we want to avoid this, we can remain and deprecate interface configuration and methods. Jiajia Li , would you do a survey on this and see how the change will affect downstream projects? Thanks.
          Hide
          jiajia Jiajia Li added a comment -

          Thanks for Andrew and Kai's advises, I will survery how the MiniKDC to be used in downstream projects(such as: HBase).

          Show
          jiajia Jiajia Li added a comment - Thanks for Andrew and Kai's advises, I will survery how the MiniKDC to be used in downstream projects(such as: HBase).
          Hide
          stevel@apache.org Steve Loughran added a comment -

          MiniKDC does sometimes get used, but it's been weak and not ideal. Limited support for protocols, a pain to set up, and I could never get it to issue tickets for >1 person in the same JVM (though that was probably UGI's static initializers there).

          I don't know how much use of MiniKDC there is outside; mvnrepo says: Kafka. HBase, Accumulo ... these are all people we can talk to.

          This is the evolution of classic MiniKDC: it will have to move on, what needs to be done is do it carefully and with users of the module happy.

          Show
          stevel@apache.org Steve Loughran added a comment - MiniKDC does sometimes get used, but it's been weak and not ideal. Limited support for protocols, a pain to set up, and I could never get it to issue tickets for >1 person in the same JVM (though that was probably UGI's static initializers there). I don't know how much use of MiniKDC there is outside; mvnrepo says: Kafka. HBase, Accumulo ... these are all people we can talk to. This is the evolution of classic MiniKDC: it will have to move on, what needs to be done is do it carefully and with users of the module happy.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Quick patch review

          1. KerberosSecurityTestcase should be retained/updated; tests shouldn't need to move off it
          2. does hadoop-auth/pom.xml need changing for this patch?

          Overally, I'm thinking this is significant enough it has to be a 3.0 change; not so much for the minikdc itself but the whole migration off the existing code and changing dependencies.

          Show
          stevel@apache.org Steve Loughran added a comment - Quick patch review KerberosSecurityTestcase should be retained/updated; tests shouldn't need to move off it does hadoop-auth/pom.xml need changing for this patch? Overally, I'm thinking this is significant enough it has to be a 3.0 change; not so much for the minikdc itself but the whole migration off the existing code and changing dependencies.
          Hide
          jiajia Jiajia Li added a comment -

          Thanks for your review.
          1. Do you mean in the patch the TestKerberosAuthenticator class not extends from KerberosSecurityTestcase? In the new patch, I will keep it as original.
          2. All the ApacheDS dependency will be removed, and we will use the Kerby's Keytab class. So the haddop-auth/pom.xml be changed.

          Show
          jiajia Jiajia Li added a comment - Thanks for your review. 1. Do you mean in the patch the TestKerberosAuthenticator class not extends from KerberosSecurityTestcase? In the new patch, I will keep it as original. 2. All the ApacheDS dependency will be removed, and we will use the Kerby's Keytab class. So the haddop-auth/pom.xml be changed.
          Hide
          jiajia Jiajia Li added a comment -

          Agree, the "test.build.dir" will be better.

          Show
          jiajia Jiajia Li added a comment - Agree, the "test.build.dir" will be better.
          Hide
          drankye Kai Zheng added a comment -

          Thanks Steve Loughran.

          I could never get it to issue tickets for >1 person in the same JVM (though that was probably UGI's static initializers there).

          It's a good idea to support tickets > 1, I believe Kerby SimpleKDC can do so and the new MiniKDC will do. It would be good to add tests to verify this. Jiajia Li, how would you think of this?
          In your previous experience, I guess you're right it's because of UGI. So it's good not to couple this with UGI (so inherent the bad), as it is and will be.

          This is the evolution of classic MiniKDC: it will have to move on, what needs to be done is do it carefully and with users of the module happy.

          I agree. Based on the survey, let's see how to make all the clients happy. On the other hand, people already suffered with the current, I do wish the evolving will make their future life easier. Jiajia Li, IIRC, there were some issues related to MiniKDC in the current codebase, maybe you would check them out and see how this new implementation will solve them or not? Thanks.

          Show
          drankye Kai Zheng added a comment - Thanks Steve Loughran . I could never get it to issue tickets for >1 person in the same JVM (though that was probably UGI's static initializers there). It's a good idea to support tickets > 1, I believe Kerby SimpleKDC can do so and the new MiniKDC will do. It would be good to add tests to verify this. Jiajia Li , how would you think of this? In your previous experience, I guess you're right it's because of UGI. So it's good not to couple this with UGI (so inherent the bad), as it is and will be. This is the evolution of classic MiniKDC: it will have to move on, what needs to be done is do it carefully and with users of the module happy. I agree. Based on the survey, let's see how to make all the clients happy. On the other hand, people already suffered with the current, I do wish the evolving will make their future life easier. Jiajia Li , IIRC, there were some issues related to MiniKDC in the current codebase, maybe you would check them out and see how this new implementation will solve them or not? Thanks.
          Hide
          jiajia Jiajia Li added a comment -

          It's a good idea to support tickets > 1, I believe Kerby SimpleKDC can do so and the new MiniKDC will do. It would be good to add tests to verify this.

          Agree, I will add the test to verify this.

          there were some issues related to MiniKDC in the current codebase, maybe you would check them out and see how this new implementation will solve them or not?

          Yes, I think it will make MiniKDC better, I will do it.

          Show
          jiajia Jiajia Li added a comment - It's a good idea to support tickets > 1, I believe Kerby SimpleKDC can do so and the new MiniKDC will do. It would be good to add tests to verify this. Agree, I will add the test to verify this. there were some issues related to MiniKDC in the current codebase, maybe you would check them out and see how this new implementation will solve them or not? Yes, I think it will make MiniKDC better, I will do it.
          Hide
          jiajia Jiajia Li added a comment -

          What's the downstream impact if we do these incompatible changes? I know a lot of downstreams use MiniCluster for instance, and thus perhaps MiniKdc for secure cluster testing.

          I checked HBase and Ooize, they use these interface methods: createConf(), start(), createPrincipal(), getRealm(), stop().
          In this patch, only the conf file in tests is changed, and it incurred no changes of the interface.
          So I think it will not impact the downstream projects.

          Show
          jiajia Jiajia Li added a comment - What's the downstream impact if we do these incompatible changes? I know a lot of downstreams use MiniCluster for instance, and thus perhaps MiniKdc for secure cluster testing. I checked HBase and Ooize, they use these interface methods: createConf(), start(), createPrincipal(), getRealm(), stop(). In this patch, only the conf file in tests is changed, and it incurred no changes of the interface. So I think it will not impact the downstream projects.
          Hide
          drankye Kai Zheng added a comment -

          Thanks Jiajia for looking for the impacts, it sounds good so we probably don't need to mark this as incompatible, as most downstream projects use MiniKDC in the most simpler approach for their tests.

          Show
          drankye Kai Zheng added a comment - Thanks Jiajia for looking for the impacts, it sounds good so we probably don't need to mark this as incompatible, as most downstream projects use MiniKDC in the most simpler approach for their tests.
          Hide
          drankye Kai Zheng added a comment -

          As Jiajia Li clarified it's still keeping the existing configuration interfaces and surveyed existing downstream clients are using the interface methods still kept, we don't have to mark it as incompatible. As Steve Loughran suggested previously, I target this to 3.0 release. Andrew Wang could you please comment if you think otherwise. Thanks!

          Show
          drankye Kai Zheng added a comment - As Jiajia Li clarified it's still keeping the existing configuration interfaces and surveyed existing downstream clients are using the interface methods still kept, we don't have to mark it as incompatible. As Steve Loughran suggested previously, I target this to 3.0 release. Andrew Wang could you please comment if you think otherwise. Thanks!
          Hide
          andrew.wang Andrew Wang added a comment -

          Sounds good to me, thanks Kai. I unset "Fix Version" and set the "Target Version" to 3.0.0, we normally only set "Fix Version" when the patch is committed.

          Show
          andrew.wang Andrew Wang added a comment - Sounds good to me, thanks Kai. I unset "Fix Version" and set the "Target Version" to 3.0.0, we normally only set "Fix Version" when the patch is committed.
          Hide
          drankye Kai Zheng added a comment -

          Oh see. Thanks Andrew for the correction and clarifying for me.

          Show
          drankye Kai Zheng added a comment - Oh see. Thanks Andrew for the correction and clarifying for me.
          Hide
          jiajia Jiajia Li added a comment -

          Upload v7.
          1. Remove the exclusion of bcprov in pom.
          2. Keep TestKerberosAuthenticator class extending from KerberosSecurityTestcase.

          Show
          jiajia Jiajia Li added a comment - Upload v7. 1. Remove the exclusion of bcprov in pom. 2. Keep TestKerberosAuthenticator class extending from KerberosSecurityTestcase.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 30s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 1m 23s Maven dependency ordering for branch
          +1 mvninstall 8m 11s trunk passed
          +1 compile 7m 4s trunk passed with JDK v1.8.0_91
          +1 compile 7m 58s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 15s trunk passed
          +1 mvnsite 4m 1s trunk passed
          +1 mvneclipse 1m 47s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 6m 25s trunk passed
          +1 javadoc 3m 32s trunk passed with JDK v1.8.0_91
          +1 javadoc 4m 40s trunk passed with JDK v1.7.0_95
          0 mvndep 1m 27s Maven dependency ordering for patch
          +1 mvninstall 3m 16s the patch passed
          +1 compile 7m 20s the patch passed with JDK v1.8.0_91
          -1 javac 7m 20s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 723 unchanged - 0 fixed = 724 total (was 723)
          +1 compile 7m 53s the patch passed with JDK v1.7.0_95
          -1 javac 7m 53s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 719 unchanged - 0 fixed = 720 total (was 719)
          -1 checkstyle 1m 20s root: The patch generated 2 new + 86 unchanged - 15 fixed = 88 total (was 101)
          +1 mvnsite 3m 51s the patch passed
          +1 mvneclipse 1m 31s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 39s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 3m 25s the patch passed with JDK v1.8.0_91
          +1 javadoc 4m 40s the patch passed with JDK v1.7.0_95
          +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 25s hadoop-minikdc in the patch passed with JDK v1.8.0_91.
          +1 unit 3m 24s hadoop-auth in the patch passed with JDK v1.8.0_91.
          +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_91.
          -1 unit 75m 55s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          -1 unit 31m 33s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 23s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 3m 22s hadoop-auth in the patch passed with JDK v1.7.0_95.
          +1 unit 7m 12s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 34s hadoop-kms in the patch passed with JDK v1.7.0_95.
          -1 unit 68m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 unit 33m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          327m 2s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-minikdc
            org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java:[line 345]
          JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.TestRollingUpgrade
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802145/HADOOP-12911-v7.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux a731567b37fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 355325b
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_91.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-checkstyle-root.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 1m 23s Maven dependency ordering for branch +1 mvninstall 8m 11s trunk passed +1 compile 7m 4s trunk passed with JDK v1.8.0_91 +1 compile 7m 58s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 15s trunk passed +1 mvnsite 4m 1s trunk passed +1 mvneclipse 1m 47s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 6m 25s trunk passed +1 javadoc 3m 32s trunk passed with JDK v1.8.0_91 +1 javadoc 4m 40s trunk passed with JDK v1.7.0_95 0 mvndep 1m 27s Maven dependency ordering for patch +1 mvninstall 3m 16s the patch passed +1 compile 7m 20s the patch passed with JDK v1.8.0_91 -1 javac 7m 20s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 723 unchanged - 0 fixed = 724 total (was 723) +1 compile 7m 53s the patch passed with JDK v1.7.0_95 -1 javac 7m 53s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 719 unchanged - 0 fixed = 720 total (was 719) -1 checkstyle 1m 20s root: The patch generated 2 new + 86 unchanged - 15 fixed = 88 total (was 101) +1 mvnsite 3m 51s the patch passed +1 mvneclipse 1m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 39s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 3m 25s the patch passed with JDK v1.8.0_91 +1 javadoc 4m 40s the patch passed with JDK v1.7.0_95 +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.8.0_91. +1 unit 0m 25s hadoop-minikdc in the patch passed with JDK v1.8.0_91. +1 unit 3m 24s hadoop-auth in the patch passed with JDK v1.8.0_91. +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.8.0_91. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_91. -1 unit 75m 55s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 31m 33s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 23s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 3m 22s hadoop-auth in the patch passed with JDK v1.7.0_95. +1 unit 7m 12s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 1m 34s hadoop-kms in the patch passed with JDK v1.7.0_95. -1 unit 68m 27s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 unit 33m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 327m 2s Reason Tests FindBugs module:hadoop-common-project/hadoop-minikdc   org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java: [line 345] JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.server.balancer.TestBalancer   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.TestRollingUpgrade   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802145/HADOOP-12911-v7.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux a731567b37fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 355325b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9272/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jiajia Jiajia Li added a comment -

          Upload v8.
          Fix checkstyle issue.

          Show
          jiajia Jiajia Li added a comment - Upload v8. Fix checkstyle issue.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 28s trunk passed
          +1 compile 5m 41s trunk passed with JDK v1.8.0_91
          +1 compile 6m 36s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 3m 32s trunk passed
          +1 mvneclipse 1m 33s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          +1 findbugs 5m 46s trunk passed
          +1 javadoc 3m 4s trunk passed with JDK v1.8.0_91
          +1 javadoc 4m 7s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 32s Maven dependency ordering for patch
          +1 mvninstall 2m 43s the patch passed
          +1 compile 5m 38s the patch passed with JDK v1.8.0_91
          -1 javac 5m 38s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 663 unchanged - 0 fixed = 664 total (was 663)
          +1 compile 6m 36s the patch passed with JDK v1.7.0_95
          -1 javac 6m 36s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 672 unchanged - 0 fixed = 673 total (was 672)
          +1 checkstyle 1m 5s root: The patch generated 0 new + 86 unchanged - 15 fixed = 86 total (was 101)
          +1 mvnsite 3m 24s the patch passed
          +1 mvneclipse 1m 27s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project
          -1 findbugs 0m 35s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 3m 5s the patch passed with JDK v1.8.0_91
          +1 javadoc 4m 6s the patch passed with JDK v1.7.0_95
          +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 19s hadoop-minikdc in the patch passed with JDK v1.8.0_91.
          +1 unit 3m 16s hadoop-auth in the patch passed with JDK v1.8.0_91.
          +1 unit 8m 12s hadoop-common in the patch passed with JDK v1.8.0_91.
          +1 unit 1m 26s hadoop-kms in the patch passed with JDK v1.8.0_91.
          -1 unit 60m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
          -1 unit 28m 12s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95.
          +1 unit 0m 23s hadoop-minikdc in the patch passed with JDK v1.7.0_95.
          +1 unit 3m 19s hadoop-auth in the patch passed with JDK v1.7.0_95.
          +1 unit 7m 56s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 unit 1m 34s hadoop-kms in the patch passed with JDK v1.7.0_95.
          -1 unit 53m 13s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 unit 29m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          274m 1s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-minikdc
            org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java:[line 345]
          JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeLifeline
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802396/HaDOOP-12911-v8.patch
          JIRA Issue HADOOP-12911
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux caa4bc6a476b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 72b0477
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_91.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/testReport/
          modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 28s trunk passed +1 compile 5m 41s trunk passed with JDK v1.8.0_91 +1 compile 6m 36s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 4s trunk passed +1 mvnsite 3m 32s trunk passed +1 mvneclipse 1m 33s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project +1 findbugs 5m 46s trunk passed +1 javadoc 3m 4s trunk passed with JDK v1.8.0_91 +1 javadoc 4m 7s trunk passed with JDK v1.7.0_95 0 mvndep 0m 32s Maven dependency ordering for patch +1 mvninstall 2m 43s the patch passed +1 compile 5m 38s the patch passed with JDK v1.8.0_91 -1 javac 5m 38s root-jdk1.8.0_91 with JDK v1.8.0_91 generated 1 new + 663 unchanged - 0 fixed = 664 total (was 663) +1 compile 6m 36s the patch passed with JDK v1.7.0_95 -1 javac 6m 36s root-jdk1.7.0_95 with JDK v1.7.0_95 generated 1 new + 672 unchanged - 0 fixed = 673 total (was 672) +1 checkstyle 1m 5s root: The patch generated 0 new + 86 unchanged - 15 fixed = 86 total (was 101) +1 mvnsite 3m 24s the patch passed +1 mvneclipse 1m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project -1 findbugs 0m 35s hadoop-common-project/hadoop-minikdc generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 3m 5s the patch passed with JDK v1.8.0_91 +1 javadoc 4m 6s the patch passed with JDK v1.7.0_95 +1 unit 0m 8s hadoop-project in the patch passed with JDK v1.8.0_91. +1 unit 0m 19s hadoop-minikdc in the patch passed with JDK v1.8.0_91. +1 unit 3m 16s hadoop-auth in the patch passed with JDK v1.8.0_91. +1 unit 8m 12s hadoop-common in the patch passed with JDK v1.8.0_91. +1 unit 1m 26s hadoop-kms in the patch passed with JDK v1.8.0_91. -1 unit 60m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 28m 12s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. +1 unit 0m 11s hadoop-project in the patch passed with JDK v1.7.0_95. +1 unit 0m 23s hadoop-minikdc in the patch passed with JDK v1.7.0_95. +1 unit 3m 19s hadoop-auth in the patch passed with JDK v1.7.0_95. +1 unit 7m 56s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 1m 34s hadoop-kms in the patch passed with JDK v1.7.0_95. -1 unit 53m 13s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 unit 29m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 274m 1s Reason Tests FindBugs module:hadoop-common-project/hadoop-minikdc   org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java: [line 345] JDK v1.8.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeLifeline   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802396/HaDOOP-12911-v8.patch JIRA Issue HADOOP-12911 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux caa4bc6a476b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 72b0477 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_95.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-minikdc.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/testReport/ modules C: hadoop-project hadoop-common-project/hadoop-minikdc hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9288/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          drankye Kai Zheng added a comment -

          Hi Jiajia Li,

          Looks like the reported findbug issue needs to be addressed. Could you check? Thanks.

          org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java:[line 345]
          

          It's very close and would be good to speed up to allow this to be included in the alpha1 release cut of Hadoop 3, per an off-line sync up with Andrew Wang.

          Show
          drankye Kai Zheng added a comment - Hi Jiajia Li , Looks like the reported findbug issue needs to be addressed. Could you check? Thanks. org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held At MiniKdc.java:lock held At MiniKdc.java:[line 345] It's very close and would be good to speed up to allow this to be included in the alpha1 release cut of Hadoop 3, per an off-line sync up with Andrew Wang .
          Hide
          jiajia Jiajia Li added a comment -

          Looks like the reported findbug issue needs to be addressed.

          This is the issue in Kerby, when stop the kdcserver, it did not wait until the network pool has terminated. If without the code "Thread.sleep()" in stop() function, will fail to restart MiniKdc. And I've fixed it in DIRKRB-552.

          Show
          jiajia Jiajia Li added a comment - Looks like the reported findbug issue needs to be addressed. This is the issue in Kerby, when stop the kdcserver, it did not wait until the network pool has terminated. If without the code "Thread.sleep()" in stop() function, will fail to restart MiniKdc. And I've fixed it in DIRKRB-552 .
          Hide
          drankye Kai Zheng added a comment -

          Thanks Jiajia. Yes I see we have a comment about this and we can get rid of the work around when given next Kerby revision.

          +1 on latest patch and will commit it shortly.

          Show
          drankye Kai Zheng added a comment - Thanks Jiajia. Yes I see we have a comment about this and we can get rid of the work around when given next Kerby revision. +1 on latest patch and will commit it shortly.
          Hide
          drankye Kai Zheng added a comment -

          Committed to trunk. Thanks Jiajia Li for this great contribution, Steve Loughran and Andrew Wang for the reviewing!

          Show
          drankye Kai Zheng added a comment - Committed to trunk. Thanks Jiajia Li for this great contribution, Steve Loughran and Andrew Wang for the reviewing!
          Hide
          jiajia Jiajia Li added a comment -

          Thanks all the reviewers for the great comments and suggestions, that really help me a lot.

          Show
          jiajia Jiajia Li added a comment - Thanks all the reviewers for the great comments and suggestions, that really help me a lot.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9874 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9874/)
          HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia (kai.zheng: rev 916140604ffef59466ba30832478311d3e6249bd)

          • hadoop-common-project/hadoop-common/pom.xml
          • hadoop-common-project/hadoop-minikdc/src/test/java/org/apache/hadoop/minikdc/TestMiniKdc.java
          • hadoop-common-project/hadoop-auth/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/krb5.conf
          • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
          • hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc.ldiff
          • hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestKerberosAuthenticationHandler.java
          • hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosUtil.java
          • hadoop-common-project/hadoop-common/src/test/resources/krb5.conf
          • hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java
          • hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosUtil.java
          • hadoop-project/pom.xml
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/KDiag.java
          • hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc-krb5.conf
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/krb5.conf
          • hadoop-common-project/hadoop-minikdc/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9874 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9874/ ) HADOOP-12911 . Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia (kai.zheng: rev 916140604ffef59466ba30832478311d3e6249bd) hadoop-common-project/hadoop-common/pom.xml hadoop-common-project/hadoop-minikdc/src/test/java/org/apache/hadoop/minikdc/TestMiniKdc.java hadoop-common-project/hadoop-auth/pom.xml hadoop-hdfs-project/hadoop-hdfs/src/test/resources/krb5.conf hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc.ldiff hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/server/TestKerberosAuthenticationHandler.java hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosUtil.java hadoop-common-project/hadoop-common/src/test/resources/krb5.conf hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/util/TestKerberosUtil.java hadoop-project/pom.xml hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/KDiag.java hadoop-common-project/hadoop-minikdc/src/main/resources/minikdc-krb5.conf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/krb5.conf hadoop-common-project/hadoop-minikdc/pom.xml
          Hide
          aw Allen Wittenauer added a comment -

          Why was this committed with a findbugs errors rather than adding the necessary plumbing in pom.xml to make it go away? Why wasn't the fix version set in jira? Do we actually need to package all of these extra bits into the distribution tar ball?

          Show
          aw Allen Wittenauer added a comment - Why was this committed with a findbugs errors rather than adding the necessary plumbing in pom.xml to make it go away? Why wasn't the fix version set in jira? Do we actually need to package all of these extra bits into the distribution tar ball?
          Hide
          drankye Kai Zheng added a comment -

          Hi Allen Wittenauer, thanks for your questions!

          Why was this committed with a findbugs errors rather than adding the necessary plumbing in pom.xml to make it go away?

          Sorry I'm not aware of that it's a must-to-do instead of better-to-do to make findbug issues go away, as we do for check style issues. I planned to get rid of this given kerby-1.0.0-rc3 release, by removing the related work around codes totally.

          Why wasn't the fix version set in jira?

          Yes it's my mistake, I should have been careful about it. Done!

          Do we actually need to package all of these extra bits into the distribution tar ball?

          I'm not sure I understood this. Did you see any related change affecting this? What we wanted to change is the involved dependencies, indicated in the pom.xml file.

          Show
          drankye Kai Zheng added a comment - Hi Allen Wittenauer , thanks for your questions! Why was this committed with a findbugs errors rather than adding the necessary plumbing in pom.xml to make it go away? Sorry I'm not aware of that it's a must-to-do instead of better-to-do to make findbug issues go away, as we do for check style issues. I planned to get rid of this given kerby-1.0.0-rc3 release, by removing the related work around codes totally. Why wasn't the fix version set in jira? Yes it's my mistake, I should have been careful about it. Done! Do we actually need to package all of these extra bits into the distribution tar ball? I'm not sure I understood this. Did you see any related change affecting this? What we wanted to change is the involved dependencies, indicated in the pom.xml file.
          Hide
          aw Allen Wittenauer added a comment -

          Sorry I'm not aware of that it's a must-to-do instead of better-to-do to make findbug issues go away, as we do for check style issues.

          Well, now you know.

          Did you see any related change affecting this? What we wanted to change is the involved dependencies, indicated in the pom.xml file.

          If we're changing this, it seems like an ideal time to strip dependencies out of the dist since this is clearly going to change what gets bundled.

          Show
          aw Allen Wittenauer added a comment - Sorry I'm not aware of that it's a must-to-do instead of better-to-do to make findbug issues go away, as we do for check style issues. Well, now you know. Did you see any related change affecting this? What we wanted to change is the involved dependencies, indicated in the pom.xml file. If we're changing this, it seems like an ideal time to strip dependencies out of the dist since this is clearly going to change what gets bundled.
          Hide
          vinayrpet Vinayakumar B added a comment -

          I am sorry for coming in late for this,
          I have one quick comment/suggestion, can be fixed in follow up jira.

          <groupId>org.apache.kerby</groupId>
          <artifactId>kerb-simplekdc</artifactId>
          <version>1.0.0-RC2</version>

          hadoop-project/pom.xml contains the dependencies of all libraries used in all modules of hadoop, under dependencyManagement. Only here version will be mentioned. All other Hadoop Modules will inherit hadoop-project, so all submodules will use the same version. In submodule, version need not be mentioned in pom.xml. This will make version management easier.

          Show
          vinayrpet Vinayakumar B added a comment - I am sorry for coming in late for this, I have one quick comment/suggestion, can be fixed in follow up jira. <groupId>org.apache.kerby</groupId> <artifactId>kerb-simplekdc</artifactId> <version>1.0.0-RC2</version> hadoop-project/pom.xml contains the dependencies of all libraries used in all modules of hadoop, under dependencyManagement . Only here version will be mentioned. All other Hadoop Modules will inherit hadoop-project , so all submodules will use the same version. In submodule, version need not be mentioned in pom.xml. This will make version management easier.
          Hide
          drankye Kai Zheng added a comment -

          Thanks Vinayakumar B for the suggestion! Sure we can do it in a follow-on issue as well as other suggestion like raised above by Allen Wittenauer.

          Show
          drankye Kai Zheng added a comment - Thanks Vinayakumar B for the suggestion! Sure we can do it in a follow-on issue as well as other suggestion like raised above by Allen Wittenauer .
          Hide
          jiajia Jiajia Li added a comment -

          If we're changing this, it seems like an ideal time to strip dependencies out of the dist since this is clearly going to change what gets bundled.

          Hi Allen Wittenauer, I'd like to take this work, but I don't know how to do it? Do you mean to remove the kerby dependency from hadoop-dist pom.xml? Can you provide more detail info? Thanks in advance.

          Show
          jiajia Jiajia Li added a comment - If we're changing this, it seems like an ideal time to strip dependencies out of the dist since this is clearly going to change what gets bundled. Hi Allen Wittenauer , I'd like to take this work, but I don't know how to do it? Do you mean to remove the kerby dependency from hadoop-dist pom.xml? Can you provide more detail info? Thanks in advance.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Allen is talking about removing the apache-ds and similar, everything needed for the existing mini kdc doe.

          Bear in mind that anything stripping out dependencies needs to be tested carefully. Don't just try and rebuild hadoop; install it locally via a "mvn install', then try to rebuild: Hive, Hbase, Tez & maybe Spark locally. It does take time, but it finds those transient problems before Hadoop ships.

          Show
          stevel@apache.org Steve Loughran added a comment - Allen is talking about removing the apache-ds and similar, everything needed for the existing mini kdc doe. Bear in mind that anything stripping out dependencies needs to be tested carefully. Don't just try and rebuild hadoop; install it locally via a "mvn install', then try to rebuild: Hive, Hbase, Tez & maybe Spark locally. It does take time, but it finds those transient problems before Hadoop ships.
          Hide
          andrew.wang Andrew Wang added a comment -

          Given this is targeted at 3.0 and not 2.x, I think we should start by removing all the related dependencies. As it is, I'm not sure any of those apps are going to compile against 3.0.

          Show
          andrew.wang Andrew Wang added a comment - Given this is targeted at 3.0 and not 2.x, I think we should start by removing all the related dependencies. As it is, I'm not sure any of those apps are going to compile against 3.0.
          Hide
          jiajia Jiajia Li added a comment -

          Thanks Steve for making it clear, I will try to rebuild these projects.

          Show
          jiajia Jiajia Li added a comment - Thanks Steve for making it clear, I will try to rebuild these projects.
          Hide
          jiajia Jiajia Li added a comment -

          Thanks Vinayakumar B for the suggestion! Sure we can do it in a follow-on issue as well as other suggestion like raised above by Allen Wittenauer.

          I've created the follow-on issue HADOOP-13220.

          Show
          jiajia Jiajia Li added a comment - Thanks Vinayakumar B for the suggestion! Sure we can do it in a follow-on issue as well as other suggestion like raised above by Allen Wittenauer. I've created the follow-on issue HADOOP-13220 .
          Hide
          jiajia Jiajia Li added a comment -

          Thanks for Andrew's explanation. We will choose some of those apps to rebuild although maybe they are not going to compile against 3.0, I think that can help to find some issues.

          Show
          jiajia Jiajia Li added a comment - Thanks for Andrew's explanation. We will choose some of those apps to rebuild although maybe they are not going to compile against 3.0, I think that can help to find some issues.

            People

            • Assignee:
              jiajia Jiajia Li
              Reporter:
              jiajia Jiajia Li
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development