Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3873

Hftp assumes security is disabled if token fetch fails

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3, 3.0.0, 2.0.2-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      Hftp ignores all exceptions generated while trying to get a token, based on the assumption that it means security is disabled. Debugging problems is excruciatingly difficult when security is enabled but something goes wrong. Job submissions succeed, but tasks fail because the NN rejects the user as unauthenticated.

      1. HDFS-3873.branch-23.patch
        4 kB
        Daryn Sharp
      2. HDFS-3873.patch
        4 kB
        Daryn Sharp

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #410 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/410/)
          svn merge -c 1393777 FIXES: HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1400216)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1400216
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #410 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/410/ ) svn merge -c 1393777 FIXES: HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1400216) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1400216 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/)
          HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/ ) HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/)
          HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/ ) HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2830 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2830/)
          HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2830 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2830/ ) HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2869 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2869/)
          HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2869 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2869/ ) HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2807 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2807/)
          HDFS-3996. Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2807 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2807/ ) HDFS-3996 . Add debug log removed in HDFS-3873 back. Contributed by Eli Collins (Revision 1393777) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393777 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          Hide
          Eli Collins added a comment -

          Filed HDFS-3996 to add the log back.

          Show
          Eli Collins added a comment - Filed HDFS-3996 to add the log back.
          Hide
          Eli Collins added a comment -

          I'm saying we should re-introduce the "LOG.debug("error was ", e);" that was removed below. This debug statement is what gives us the full backtrace (only if debug is enabled). Make sense?

          -            if(LOG.isDebugEnabled()) {
          -              LOG.debug("error was ", e);
          +          } catch (IOException e) {
          +            if (e.getCause() instanceof ConnectException) {
          +              LOG.warn("Couldn't connect to " + nnHttpUrl +
          +                  ", assuming security is disabled");
          +              return null;
                       }
          
          Show
          Eli Collins added a comment - I'm saying we should re-introduce the "LOG.debug("error was ", e);" that was removed below. This debug statement is what gives us the full backtrace (only if debug is enabled). Make sense? - if (LOG.isDebugEnabled()) { - LOG.debug( "error was " , e); + } catch (IOException e) { + if (e.getCause() instanceof ConnectException) { + LOG.warn( "Couldn't connect to " + nnHttpUrl + + ", assuming security is disabled" ); + return null ; }
          Hide
          Daryn Sharp added a comment -

          I think it was being double logged, so it should be getting logged at a higher level... I'm not sure why you aren't seeing the cause because the applicable line is throw new IOException("Unable to obtain remote token", e). It's rather disconcerting that a NPE is being thrown?

          Show
          Daryn Sharp added a comment - I think it was being double logged, so it should be getting logged at a higher level... I'm not sure why you aren't seeing the cause because the applicable line is throw new IOException("Unable to obtain remote token", e) . It's rather disconcerting that a NPE is being thrown?
          Hide
          Eli Collins added a comment -

          Hey Daryn,

          With this change we no longer log the exception that caused us to fail to obtain a token. Intentional?

          We now get "cause:java.io.IOException: Unable to obtain remote token" but used to get..

          12/09/26 18:33:10 DEBUG fs.FileSystem: error was 
          java.io.IOException: Unable to obtain remote token
          ..
          aused by: java.lang.RuntimeException: java.lang.NullPointerException
          	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1137)
          ...
          
          Show
          Eli Collins added a comment - Hey Daryn, With this change we no longer log the exception that caused us to fail to obtain a token. Intentional? We now get "cause:java.io.IOException: Unable to obtain remote token" but used to get.. 12/09/26 18:33:10 DEBUG fs.FileSystem: error was java.io.IOException: Unable to obtain remote token .. aused by: java.lang.RuntimeException: java.lang.NullPointerException at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1137) ...
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615)

          Result = SUCCESS
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615)

          Result = SUCCESS
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379620)

          Result = UNSTABLE
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379620
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379620) Result = UNSTABLE daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379620 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2692 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2692/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615)

          Result = FAILURE
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2692 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2692/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615) Result = FAILURE daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Suresh Srinivas added a comment -

          Daryn, the change looks good. +1.

          Show
          Suresh Srinivas added a comment - Daryn, the change looks good. +1.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2730 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2730/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615)

          Result = SUCCESS
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2730 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2730/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2667 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2667/)
          HDFS-3873. Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615)

          Result = SUCCESS
          daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2667 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2667/ ) HDFS-3873 . Hftp assumes security is disabled if token fetch fails (daryn) (Revision 1379615) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379615 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HftpFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpDelegationToken.java
          Hide
          Robert Joseph Evans added a comment -

          It looks good to me. I am not really an expert on HFTP, but this is a simple enough change that I feel OK to give it a +1, but please use your discretion before checking it in.

          I am not sure why Jenkins ran the tests again and they failed, but when I run them with your patch they pass. except for TestHftpDelegationToken which is a known issue.

          Show
          Robert Joseph Evans added a comment - It looks good to me. I am not really an expert on HFTP, but this is a simple enough change that I feel OK to give it a +1, but please use your discretion before checking it in. I am not sure why Jenkins ran the tests again and they failed, but when I run them with your patch they pass. except for TestHftpDelegationToken which is a known issue.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543197/HDFS-3873.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestHftpDelegationToken
          org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
          org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          org.apache.hadoop.hdfs.TestReplication
          org.apache.hadoop.hdfs.TestPersistBlocks

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3130//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3130//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543197/HDFS-3873.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestHftpDelegationToken org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints org.apache.hadoop.hdfs.TestReplication org.apache.hadoop.hdfs.TestPersistBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3130//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3130//console This message is automatically generated.
          Hide
          Daryn Sharp added a comment -

          Failed test precedes this patch, it's fixed by HDFS-3852.

          Show
          Daryn Sharp added a comment - Failed test precedes this patch, it's fixed by HDFS-3852 .
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543175/HDFS-3873.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestHftpDelegationToken

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3128//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3128//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543175/HDFS-3873.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestHftpDelegationToken +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3128//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3128//console This message is automatically generated.
          Hide
          Daryn Sharp added a comment -

          Re-attaching trunk patch since build tried to use 23 patch.

          Show
          Daryn Sharp added a comment - Re-attaching trunk patch since build tried to use 23 patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543176/HDFS-3873.branch-23.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3129//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543176/HDFS-3873.branch-23.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3129//console This message is automatically generated.
          Hide
          Daryn Sharp added a comment -

          Update test to expect different exception from 23.

          Show
          Daryn Sharp added a comment - Update test to expect different exception from 23.
          Hide
          Daryn Sharp added a comment -

          Only considers a connection refused exception as "security disabled" since an insecure cluster does not listen on the secure port. Note this prevents jobs from launching w/o tokens.

          I spent the better part of the day debugging why an oozie launcher task was trying to get a hftp token. Turns out AES was specified in krb5.conf which caused a SSL exception that was silently swallowed during job submission. The job launched and the tasks failed with user not authenticated messages from the NN. This patch evolved from the debugging effort.

          Show
          Daryn Sharp added a comment - Only considers a connection refused exception as "security disabled" since an insecure cluster does not listen on the secure port. Note this prevents jobs from launching w/o tokens. I spent the better part of the day debugging why an oozie launcher task was trying to get a hftp token. Turns out AES was specified in krb5.conf which caused a SSL exception that was silently swallowed during job submission. The job launched and the tasks failed with user not authenticated messages from the NN. This patch evolved from the debugging effort.

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development