Details

    • Hadoop Flags:
      Reviewed

      Description

      Sometimes, KMS resource file can not be loaded. When this happens, an InputStream variable will be a null pointer which will subsequently throw NPE.

      This is a supportability JIRA that makes the error message more explicit, and explain why NPE is thrown. Ultimately, leads us to understand why the resource files can not be loaded.

      1. HADOOP-12615.001.patch
        7 kB
        Wei-Chiu Chuang
      2. HADOOP-12615.002.patch
        8 kB
        Wei-Chiu Chuang
      3. HADOOP-12615.003.patch
        8 kB
        Wei-Chiu Chuang
      4. HADOOP-12615.004.patch
        8 kB
        Wei-Chiu Chuang
      5. HADOOP-12615.005.patch
        10 kB
        Wei-Chiu Chuang
      6. HADOOP-12615.006.patch
        11 kB
        Wei-Chiu Chuang
      7. HADOOP-12615.007.patch
        9 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        There are a few other classes that also use ClassLoader.getResourceAsStream()

        Show
        jojochuang Wei-Chiu Chuang added a comment - There are a few other classes that also use ClassLoader.getResourceAsStream()
        Hide
        stevel@apache.org Steve Loughran added a comment -

        hi. Can you tag your issues with the hadoop version this is failing against. Thx

        Show
        stevel@apache.org Steve Loughran added a comment - hi. Can you tag your issues with the hadoop version this is failing against. Thx
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        I saw this in one of yesterday's jenkins job for trunk. Does 3.0.0 sound reasonable? Thank you.

        Show
        jojochuang Wei-Chiu Chuang added a comment - I saw this in one of yesterday's jenkins job for trunk. Does 3.0.0 sound reasonable? Thank you.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        I saw this in one of yesterday's jenkins job for trunk. Does 3.0.0 sound reasonable? Thank you.

        Show
        jojochuang Wei-Chiu Chuang added a comment - I saw this in one of yesterday's jenkins job for trunk. Does 3.0.0 sound reasonable? Thank you.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        yes -anything that breaks against trunk affects 3.0; breaks branch-2 == 2.9.0

        Show
        stevel@apache.org Steve Loughran added a comment - yes -anything that breaks against trunk affects 3.0; breaks branch-2 == 2.9.0
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev02. updated error message.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev02. updated error message.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 9s trunk passed
        +1 compile 9m 3s trunk passed with JDK v1.8.0_66
        +1 compile 9m 26s trunk passed with JDK v1.7.0_85
        +1 checkstyle 1m 4s trunk passed
        +1 mvnsite 2m 8s trunk passed
        +1 mvneclipse 0m 57s trunk passed
        +1 findbugs 3m 24s trunk passed
        +1 javadoc 1m 36s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 53s trunk passed with JDK v1.7.0_85
        +1 mvninstall 2m 39s the patch passed
        +1 compile 8m 59s the patch passed with JDK v1.8.0_66
        +1 javac 8m 59s the patch passed
        +1 compile 9m 25s the patch passed with JDK v1.7.0_85
        +1 javac 9m 25s the patch passed
        +1 checkstyle 1m 3s the patch passed
        +1 mvnsite 2m 7s the patch passed
        +1 mvneclipse 0m 55s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 55s the patch passed
        +1 javadoc 1m 37s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 52s the patch passed with JDK v1.7.0_85
        +1 unit 0m 28s hadoop-minikdc in the patch passed with JDK v1.8.0_66.
        -1 unit 8m 5s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 35s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 34s hadoop-minikdc in the patch passed with JDK v1.7.0_85.
        -1 unit 10m 29s hadoop-common in the patch failed with JDK v1.7.0_85.
        +1 unit 1m 40s hadoop-kms in the patch passed with JDK v1.7.0_85.
        +1 unit 1m 41s hadoop-kms in the patch passed with JDK v1.7.0_85.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        98m 21s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.security.TestShellBasedIdMapping
        JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776076/HADOOP-12615.002.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b4f5079c05a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 01a641b
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
        JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
        Max memory used 75MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 9s trunk passed +1 compile 9m 3s trunk passed with JDK v1.8.0_66 +1 compile 9m 26s trunk passed with JDK v1.7.0_85 +1 checkstyle 1m 4s trunk passed +1 mvnsite 2m 8s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 3m 24s trunk passed +1 javadoc 1m 36s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 53s trunk passed with JDK v1.7.0_85 +1 mvninstall 2m 39s the patch passed +1 compile 8m 59s the patch passed with JDK v1.8.0_66 +1 javac 8m 59s the patch passed +1 compile 9m 25s the patch passed with JDK v1.7.0_85 +1 javac 9m 25s the patch passed +1 checkstyle 1m 3s the patch passed +1 mvnsite 2m 7s the patch passed +1 mvneclipse 0m 55s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 55s the patch passed +1 javadoc 1m 37s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 52s the patch passed with JDK v1.7.0_85 +1 unit 0m 28s hadoop-minikdc in the patch passed with JDK v1.8.0_66. -1 unit 8m 5s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 35s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 1m 32s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 0m 34s hadoop-minikdc in the patch passed with JDK v1.7.0_85. -1 unit 10m 29s hadoop-common in the patch failed with JDK v1.7.0_85. +1 unit 1m 40s hadoop-kms in the patch passed with JDK v1.7.0_85. +1 unit 1m 41s hadoop-kms in the patch passed with JDK v1.7.0_85. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 98m 21s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.security.TestShellBasedIdMapping JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776076/HADOOP-12615.002.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b4f5079c05a1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 01a641b findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8193/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Test failures are unrelated to this patch.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Test failures are unrelated to this patch.
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Wei-Chiu for the fix! A few questions/comments

        1. We should avoid wildcard imports: import java.io.*;
        2. Could you add to the description the failed test you saw? I think we should only throw exception for the NPE that showed up. For some other places, I suspect some logics could handle a null inputstream already. For example, looks like PropertyConfigurator.configure(in) does nothing if in is null.
        Show
        zhz Zhe Zhang added a comment - Thanks Wei-Chiu for the fix! A few questions/comments We should avoid wildcard imports: import java.io.*; Could you add to the description the failed test you saw? I think we should only throw exception for the NPE that showed up. For some other places, I suspect some logics could handle a null inputstream already. For example, looks like PropertyConfigurator.configure(in) does nothing if in is null.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Hi Zhe Zhang Thanks for the review and comments.

        IntelliJ was beling too cleve to use wildcard import. I have removed these wildcards in rev03.

        I unfortunately can not find where the NPE was any more. It was found in one of the daily jenkins build and was deleted subsequently. I have verified locally that each of the objects that use the null InputStream will subsequently throw a NullPointerException.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Hi Zhe Zhang Thanks for the review and comments. IntelliJ was beling too cleve to use wildcard import. I have removed these wildcards in rev03. I unfortunately can not find where the NPE was any more. It was found in one of the daily jenkins build and was deleted subsequently. I have verified locally that each of the objects that use the null InputStream will subsequently throw a NullPointerException.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 10m 49s trunk passed
        +1 compile 13m 31s trunk passed with JDK v1.8.0_66
        +1 compile 12m 17s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 21s trunk passed
        +1 mvnsite 2m 52s trunk passed
        +1 mvneclipse 1m 19s trunk passed
        +1 findbugs 4m 19s trunk passed
        +1 javadoc 2m 26s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 28s trunk passed with JDK v1.7.0_91
        +1 mvninstall 3m 44s the patch passed
        +1 compile 13m 37s the patch passed with JDK v1.8.0_66
        +1 javac 13m 37s the patch passed
        +1 compile 13m 30s the patch passed with JDK v1.7.0_91
        +1 javac 13m 30s the patch passed
        +1 checkstyle 1m 30s the patch passed
        +1 mvnsite 2m 54s the patch passed
        +1 mvneclipse 1m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 5m 2s the patch passed
        +1 javadoc 2m 25s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 30s the patch passed with JDK v1.7.0_91
        -1 unit 0m 41s hadoop-minikdc in the patch failed with JDK v1.8.0_66.
        -1 unit 9m 50s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 0m 52s hadoop-kms in the patch failed with JDK v1.8.0_66.
        -1 unit 0m 53s hadoop-kms in the patch failed with JDK v1.8.0_66.
        -1 unit 0m 41s hadoop-minikdc in the patch failed with JDK v1.7.0_91.
        -1 unit 16m 49s hadoop-common in the patch failed with JDK v1.7.0_91.
        -1 unit 0m 50s hadoop-kms in the patch failed with JDK v1.7.0_91.
        -1 unit 0m 49s hadoop-kms in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 35s Patch does not generate ASF License warnings.
        132m 8s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.minikdc.TestMiniKdc
          hadoop.minikdc.TestChangeOrgNameAndDomain
          hadoop.security.TestUGILoginFromKeytab
          hadoop.security.token.delegation.web.TestWebDelegationToken
          hadoop.fs.shell.find.TestFind
          hadoop.metrics2.impl.TestMetricsSystemImpl
          hadoop.test.TestTimedOutTestsListener
          hadoop.crypto.key.kms.server.TestKMS
          hadoop.crypto.key.kms.server.TestKMS
        JDK v1.7.0_91 Failed junit tests hadoop.minikdc.TestMiniKdc
          hadoop.minikdc.TestChangeOrgNameAndDomain
          hadoop.security.TestUGILoginFromKeytab
          hadoop.security.token.delegation.web.TestWebDelegationToken
          hadoop.test.TestTimedOutTestsListener
          hadoop.crypto.key.kms.server.TestKMS
          hadoop.crypto.key.kms.server.TestKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777624/HADOOP-12615.003.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 63e1b45d2f78 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d8a4542
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.1.0 http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 10m 49s trunk passed +1 compile 13m 31s trunk passed with JDK v1.8.0_66 +1 compile 12m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 21s trunk passed +1 mvnsite 2m 52s trunk passed +1 mvneclipse 1m 19s trunk passed +1 findbugs 4m 19s trunk passed +1 javadoc 2m 26s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 28s trunk passed with JDK v1.7.0_91 +1 mvninstall 3m 44s the patch passed +1 compile 13m 37s the patch passed with JDK v1.8.0_66 +1 javac 13m 37s the patch passed +1 compile 13m 30s the patch passed with JDK v1.7.0_91 +1 javac 13m 30s the patch passed +1 checkstyle 1m 30s the patch passed +1 mvnsite 2m 54s the patch passed +1 mvneclipse 1m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 2s the patch passed +1 javadoc 2m 25s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 30s the patch passed with JDK v1.7.0_91 -1 unit 0m 41s hadoop-minikdc in the patch failed with JDK v1.8.0_66. -1 unit 9m 50s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 0m 52s hadoop-kms in the patch failed with JDK v1.8.0_66. -1 unit 0m 53s hadoop-kms in the patch failed with JDK v1.8.0_66. -1 unit 0m 41s hadoop-minikdc in the patch failed with JDK v1.7.0_91. -1 unit 16m 49s hadoop-common in the patch failed with JDK v1.7.0_91. -1 unit 0m 50s hadoop-kms in the patch failed with JDK v1.7.0_91. -1 unit 0m 49s hadoop-kms in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 132m 8s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.minikdc.TestMiniKdc   hadoop.minikdc.TestChangeOrgNameAndDomain   hadoop.security.TestUGILoginFromKeytab   hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.fs.shell.find.TestFind   hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.test.TestTimedOutTestsListener   hadoop.crypto.key.kms.server.TestKMS   hadoop.crypto.key.kms.server.TestKMS JDK v1.7.0_91 Failed junit tests hadoop.minikdc.TestMiniKdc   hadoop.minikdc.TestChangeOrgNameAndDomain   hadoop.security.TestUGILoginFromKeytab   hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.test.TestTimedOutTestsListener   hadoop.crypto.key.kms.server.TestKMS   hadoop.crypto.key.kms.server.TestKMS Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777624/HADOOP-12615.003.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 63e1b45d2f78 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d8a4542 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-minikdc-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Max memory used 76MB Powered by Apache Yetus 0.1.0 http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8238/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev04. I accidentally reset an InputStream to null in rev03. Rev04 fixed the bug.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev04. I accidentally reset an InputStream to null in rev03. Rev04 fixed the bug.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 10m 42s trunk passed
        +1 compile 14m 2s trunk passed with JDK v1.8.0_66
        +1 compile 12m 21s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 19s trunk passed
        +1 mvnsite 2m 52s trunk passed
        +1 mvneclipse 1m 18s trunk passed
        +1 findbugs 4m 23s trunk passed
        +1 javadoc 2m 25s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 30s trunk passed with JDK v1.7.0_91
        +1 mvninstall 3m 43s the patch passed
        +1 compile 13m 27s the patch passed with JDK v1.8.0_66
        +1 javac 13m 27s the patch passed
        +1 compile 12m 11s the patch passed with JDK v1.7.0_91
        +1 javac 12m 11s the patch passed
        +1 checkstyle 1m 20s the patch passed
        +1 mvnsite 2m 54s the patch passed
        +1 mvneclipse 1m 23s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 5m 4s the patch passed
        +1 javadoc 2m 30s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 36s the patch passed with JDK v1.7.0_91
        +1 unit 0m 43s hadoop-minikdc in the patch passed with JDK v1.8.0_66.
        -1 unit 9m 45s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 57s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 58s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 44s hadoop-minikdc in the patch passed with JDK v1.7.0_91.
        -1 unit 9m 9s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 1m 54s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 unit 1m 56s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 32s Patch does not generate ASF License warnings.
        128m 1s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics
          hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.ipc.TestIPC
        JDK v1.7.0_91 Failed junit tests hadoop.http.TestHttpServer
          hadoop.metrics2.impl.TestMetricsSystemImpl



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777656/HADOOP-12615.004.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2fe6aab20959 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d8a4542
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.1.0 http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 10m 42s trunk passed +1 compile 14m 2s trunk passed with JDK v1.8.0_66 +1 compile 12m 21s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 19s trunk passed +1 mvnsite 2m 52s trunk passed +1 mvneclipse 1m 18s trunk passed +1 findbugs 4m 23s trunk passed +1 javadoc 2m 25s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 30s trunk passed with JDK v1.7.0_91 +1 mvninstall 3m 43s the patch passed +1 compile 13m 27s the patch passed with JDK v1.8.0_66 +1 javac 13m 27s the patch passed +1 compile 12m 11s the patch passed with JDK v1.7.0_91 +1 javac 12m 11s the patch passed +1 checkstyle 1m 20s the patch passed +1 mvnsite 2m 54s the patch passed +1 mvneclipse 1m 23s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 4s the patch passed +1 javadoc 2m 30s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 36s the patch passed with JDK v1.7.0_91 +1 unit 0m 43s hadoop-minikdc in the patch passed with JDK v1.8.0_66. -1 unit 9m 45s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 57s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 1m 58s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 0m 44s hadoop-minikdc in the patch passed with JDK v1.7.0_91. -1 unit 9m 9s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 1m 54s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 unit 1m 56s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 32s Patch does not generate ASF License warnings. 128m 1s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.metrics2.impl.TestGangliaMetrics   hadoop.ipc.TestIPC JDK v1.7.0_91 Failed junit tests hadoop.http.TestHttpServer   hadoop.metrics2.impl.TestMetricsSystemImpl Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777656/HADOOP-12615.004.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2fe6aab20959 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d8a4542 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Max memory used 76MB Powered by Apache Yetus 0.1.0 http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8242/console This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        there's enough repetition of the if is == null throw IOE(missing $path) that it could be teased out into is own verifyStream(is, source) method

        Show
        stevel@apache.org Steve Loughran added a comment - there's enough repetition of the if is == null throw IOE(missing $path) that it could be teased out into is own verifyStream(is, source) method
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for the new comments.
        How about creating a convenient method ThreadUtil.getResourceAsStream() and use it instead? Rev05 does it.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for the new comments. How about creating a convenient method ThreadUtil.getResourceAsStream() and use it instead? Rev05 does it.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 14s trunk passed
        +1 compile 9m 14s trunk passed with JDK v1.8.0_66
        +1 compile 8m 41s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 58s trunk passed
        +1 mvnsite 2m 2s trunk passed
        +1 mvneclipse 0m 53s trunk passed
        +1 findbugs 3m 5s trunk passed
        +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 35s the patch passed
        +1 compile 8m 14s the patch passed with JDK v1.8.0_66
        +1 javac 8m 14s the patch passed
        +1 compile 8m 48s the patch passed with JDK v1.7.0_91
        +1 javac 8m 48s the patch passed
        +1 checkstyle 0m 58s the patch passed
        +1 mvnsite 1m 59s the patch passed
        +1 mvneclipse 0m 54s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 37s the patch passed
        +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 44s the patch passed with JDK v1.7.0_91
        +1 unit 0m 26s hadoop-minikdc in the patch passed with JDK v1.8.0_66.
        -1 unit 6m 51s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 29s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 33s hadoop-minikdc in the patch passed with JDK v1.7.0_91.
        +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 unit 1m 36s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        90m 20s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777793/HADOOP-12615.005.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 930d25e85028 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8602692
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
        Max memory used 75MB
        Powered by Apache Yetus 0.1.0 http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 14s trunk passed +1 compile 9m 14s trunk passed with JDK v1.8.0_66 +1 compile 8m 41s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 58s trunk passed +1 mvnsite 2m 2s trunk passed +1 mvneclipse 0m 53s trunk passed +1 findbugs 3m 5s trunk passed +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 35s the patch passed +1 compile 8m 14s the patch passed with JDK v1.8.0_66 +1 javac 8m 14s the patch passed +1 compile 8m 48s the patch passed with JDK v1.7.0_91 +1 javac 8m 48s the patch passed +1 checkstyle 0m 58s the patch passed +1 mvnsite 1m 59s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 37s the patch passed +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 44s the patch passed with JDK v1.7.0_91 +1 unit 0m 26s hadoop-minikdc in the patch passed with JDK v1.8.0_66. -1 unit 6m 51s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 1m 29s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 0m 33s hadoop-minikdc in the patch passed with JDK v1.7.0_91. +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 unit 1m 36s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 90m 20s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777793/HADOOP-12615.005.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 930d25e85028 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8602692 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Max memory used 75MB Powered by Apache Yetus 0.1.0 http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8248/console This message was automatically generated.
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Steve for the suggestion; good point.

        Thank you for the update Wei-Chiu. I have a few questions/comments.

        1. I'm not sure if cl = ThreadUtil.class.getClassLoader() is the right behavior. Could you explain why we can use the CL of ThreadUtil to replace the CL of the current thread?
        2. In MiniKMS, better make the following 2 segments consistent. They should both use a try-finally structure, or neither. Actually I don't think we need the try-finally: in case of IOException, is remains null anyway.
              if (!aclsFile.exists()) {
                InputStream is =
                    ThreadUtil.getResourceAsStream("mini-kms-acls-default.xml");
          
                InputStream is = null;
                OutputStream os;
                try {
                  is = ThreadUtil.getResourceAsStream("kms-webapp/WEB-INF/web.xml");
                  os = new FileOutputStream(new File(webInf, "web.xml"));
                  IOUtils.copy(is, os);
                } finally {
                  IOUtils.closeQuietly(is);
                  IOUtils.closeQuietly(is);
                }
          
        3. Also, is is closed twice in the above. A typo with os?
        4. The local variable is in TestKMSAudit.java doesn't look necessary? How about keeping the original structure but use the new getResourceAsStream method?
        5. MiniKdc still uses the old getResourceAsStream
        Show
        zhz Zhe Zhang added a comment - Thanks Steve for the suggestion; good point. Thank you for the update Wei-Chiu. I have a few questions/comments. I'm not sure if cl = ThreadUtil.class.getClassLoader() is the right behavior. Could you explain why we can use the CL of ThreadUtil to replace the CL of the current thread? In MiniKMS , better make the following 2 segments consistent. They should both use a try-finally structure, or neither. Actually I don't think we need the try-finally: in case of IOException, is remains null anyway. if (!aclsFile.exists()) { InputStream is = ThreadUtil.getResourceAsStream( "mini-kms-acls- default .xml" ); InputStream is = null ; OutputStream os; try { is = ThreadUtil.getResourceAsStream( "kms-webapp/WEB-INF/web.xml" ); os = new FileOutputStream( new File(webInf, "web.xml" )); IOUtils.copy(is, os); } finally { IOUtils.closeQuietly(is); IOUtils.closeQuietly(is); } Also, is is closed twice in the above. A typo with os ? The local variable is in TestKMSAudit.java doesn't look necessary? How about keeping the original structure but use the new getResourceAsStream method? MiniKdc still uses the old getResourceAsStream
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for the details reviews. I am taking this opportunity to learn.

        (1) It is only used if the current thread's class loader is for some reason set as null. Do you think it's better to throw an exception in this case?
        (2) Will update it
        (3) It is a typo. Good catch!
        (4) Ok.
        (5) MiniKdc can not import ThreadUtil.java, and it does not seem worth it to update pom.xml for this.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for the details reviews. I am taking this opportunity to learn. (1) It is only used if the current thread's class loader is for some reason set as null. Do you think it's better to throw an exception in this case? (2) Will update it (3) It is a typo. Good catch! (4) Ok. (5) MiniKdc can not import ThreadUtil.java, and it does not seem worth it to update pom.xml for this.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev06:
        Fixed the bug discovered by Zhe. Also, did a refactoring in MiniKMS which uses a private helper method to error-handle copying resource input stream in a consistent manner.

        Similarly, created a convenient method getResourceAsStream() in MiniKdc to get resource as stream (basically, replicate ThreadUtil.getResourceAsStream(), because MiniKdc does not depend on Hadoop)

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev06: Fixed the bug discovered by Zhe. Also, did a refactoring in MiniKMS which uses a private helper method to error-handle copying resource input stream in a consistent manner. Similarly, created a convenient method getResourceAsStream() in MiniKdc to get resource as stream (basically, replicate ThreadUtil.getResourceAsStream(), because MiniKdc does not depend on Hadoop)
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 34s trunk passed
        +1 compile 7m 56s trunk passed with JDK v1.8.0_66
        +1 compile 8m 42s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 0s trunk passed
        +1 mvnsite 2m 1s trunk passed
        +1 mvneclipse 0m 53s trunk passed
        +1 findbugs 3m 4s trunk passed
        +1 javadoc 1m 29s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 44s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 37s the patch passed
        +1 compile 7m 55s the patch passed with JDK v1.8.0_66
        +1 javac 7m 55s the patch passed
        +1 compile 8m 43s the patch passed with JDK v1.7.0_91
        +1 javac 8m 43s the patch passed
        +1 checkstyle 0m 58s the patch passed
        +1 mvnsite 2m 2s the patch passed
        +1 mvneclipse 0m 54s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 38s the patch passed
        +1 javadoc 1m 30s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_91
        +1 unit 0m 26s hadoop-minikdc in the patch passed with JDK v1.8.0_66.
        +1 unit 6m 40s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 29s hadoop-kms in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 33s hadoop-minikdc in the patch passed with JDK v1.7.0_91.
        -1 unit 6m 32s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        86m 52s



        Reason Tests
        JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestRPCWaitForProxy



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778084/HADOOP-12615.006.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 25b1a8e2e5dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0f708d4
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 34s trunk passed +1 compile 7m 56s trunk passed with JDK v1.8.0_66 +1 compile 8m 42s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 0s trunk passed +1 mvnsite 2m 1s trunk passed +1 mvneclipse 0m 53s trunk passed +1 findbugs 3m 4s trunk passed +1 javadoc 1m 29s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 44s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 37s the patch passed +1 compile 7m 55s the patch passed with JDK v1.8.0_66 +1 javac 7m 55s the patch passed +1 compile 8m 43s the patch passed with JDK v1.7.0_91 +1 javac 8m 43s the patch passed +1 checkstyle 0m 58s the patch passed +1 mvnsite 2m 2s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 38s the patch passed +1 javadoc 1m 30s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_91 +1 unit 0m 26s hadoop-minikdc in the patch passed with JDK v1.8.0_66. +1 unit 6m 40s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 1m 30s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 1m 29s hadoop-kms in the patch passed with JDK v1.8.0_66. +1 unit 0m 33s hadoop-minikdc in the patch passed with JDK v1.7.0_91. -1 unit 6m 32s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 unit 1m 37s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 86m 52s Reason Tests JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestRPCWaitForProxy Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778084/HADOOP-12615.006.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 25b1a8e2e5dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0f708d4 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc hadoop-hdfs-project/hadoop-hdfs-httpfs U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8257/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        test failure is unrelated.

        Show
        jojochuang Wei-Chiu Chuang added a comment - test failure is unrelated.
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Wei-Chiu. +1 on the latest patch pending a couple of minor suggestions:

        1. In ThreadUtil#getResourceAsStream I suggest we throw an exception when current thread's CL is null. Pls let me know you have an explanation that the same input stream will be returned from ThreadUtil's CL.
        2. Similarly, I suggest we remove the change in Server, or change it to throw an exception. Otherwise it will possibly use the CL of ThreadUtil.
          	    if (is == null) {
          	      throw new IOException("Can not read resource file '" +
          	          resourceName + "'");
          	    }
          
        Show
        zhz Zhe Zhang added a comment - Thanks Wei-Chiu. +1 on the latest patch pending a couple of minor suggestions: In ThreadUtil#getResourceAsStream I suggest we throw an exception when current thread's CL is null. Pls let me know you have an explanation that the same input stream will be returned from ThreadUtil's CL. Similarly, I suggest we remove the change in Server , or change it to throw an exception. Otherwise it will possibly use the CL of ThreadUtil. if (is == null ) { throw new IOException( "Can not read resource file '" + resourceName + "'" ); }
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev07: removed the change in Server.java. Throw exception if class loader of the current thread is null.

        Thanks @Zhe for the review!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev07: removed the change in Server.java. Throw exception if class loader of the current thread is null. Thanks @Zhe for the review!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 26s trunk passed
        +1 compile 7m 50s trunk passed with JDK v1.8.0_66
        +1 compile 8m 43s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 1m 40s trunk passed
        +1 mvneclipse 0m 39s trunk passed
        +1 findbugs 2m 39s trunk passed
        +1 javadoc 1m 17s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 31s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 14s the patch passed
        +1 compile 8m 33s the patch passed with JDK v1.8.0_66
        +1 javac 8m 33s the patch passed
        +1 compile 9m 45s the patch passed with JDK v1.7.0_91
        +1 javac 9m 45s the patch passed
        +1 checkstyle 0m 23s the patch passed
        +1 mvnsite 1m 49s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 25s the patch passed
        +1 javadoc 1m 27s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_91
        +1 unit 0m 29s hadoop-minikdc in the patch passed with JDK v1.8.0_66.
        -1 unit 7m 38s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 1m 36s hadoop-kms in the patch failed with JDK v1.8.0_66.
        +1 unit 0m 34s hadoop-minikdc in the patch passed with JDK v1.7.0_91.
        +1 unit 8m 5s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 unit 1m 41s hadoop-kms in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        83m 56s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag
          hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.crypto.key.kms.server.TestKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778146/HADOOP-12615.007.patch
        JIRA Issue HADOOP-12615
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux cd9c4de124d6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f741476
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc U: hadoop-common-project
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 26s trunk passed +1 compile 7m 50s trunk passed with JDK v1.8.0_66 +1 compile 8m 43s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 20s trunk passed +1 mvnsite 1m 40s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 2m 39s trunk passed +1 javadoc 1m 17s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 31s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 14s the patch passed +1 compile 8m 33s the patch passed with JDK v1.8.0_66 +1 javac 8m 33s the patch passed +1 compile 9m 45s the patch passed with JDK v1.7.0_91 +1 javac 9m 45s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 49s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 25s the patch passed +1 javadoc 1m 27s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_91 +1 unit 0m 29s hadoop-minikdc in the patch passed with JDK v1.8.0_66. -1 unit 7m 38s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 1m 36s hadoop-kms in the patch failed with JDK v1.8.0_66. +1 unit 0m 34s hadoop-minikdc in the patch passed with JDK v1.7.0_91. +1 unit 8m 5s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 1m 41s hadoop-kms in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 83m 56s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag   hadoop.metrics2.impl.TestGangliaMetrics   hadoop.crypto.key.kms.server.TestKMS Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778146/HADOOP-12615.007.patch JIRA Issue HADOOP-12615 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cd9c4de124d6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f741476 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-kms-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-common-project/hadoop-minikdc U: hadoop-common-project Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8263/console This message was automatically generated.
        Hide
        zhz Zhe Zhang added a comment -

        Thanks Wei-Chiu! +1 on the latest patch. I just committed to trunk.

        Show
        zhz Zhe Zhang added a comment - Thanks Wei-Chiu! +1 on the latest patch. I just committed to trunk.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks! Zhe Zhang for the commit, and Steve Loughran for the review!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks! Zhe Zhang for the commit, and Steve Loughran for the review!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8988 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8988/)
        HADOOP-12615. Fix NPE in MiniKMS.start(). Contributed by Wei-Chiu (zhz: rev f5756a2038cdacc6faf590dcab0aa62d56f5bcaf)

        • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSAudit.java
        • hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java
        • hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/VersionInfo.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ThreadUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8988 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8988/ ) HADOOP-12615 . Fix NPE in MiniKMS.start(). Contributed by Wei-Chiu (zhz: rev f5756a2038cdacc6faf590dcab0aa62d56f5bcaf) hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSAudit.java hadoop-common-project/hadoop-minikdc/src/main/java/org/apache/hadoop/minikdc/MiniKdc.java hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/MiniKMS.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/VersionInfo.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/ThreadUtil.java

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development