Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4951

FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.1.0-beta
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Target Version/s:

      Description

      It looks like there isn't a TokenRenewer for HttpFS delegation tokens (HTTPFS_DELEGATION_TOKENS tokens, so when it goes to cancel the token, it throws an exception:

      $ hadoop fs -ls webhdfs://host:14000
      // File listing omitted
      13/06/21 13:09:04 WARN token.Token: No TokenRenewer defined for token kind HTTPFS_DELEGATION_TOKEN
      13/06/21 13:09:04 WARN util.ShutdownHookManager: ShutdownHook 'ClientFinalizer' failed, java.lang.UnsupportedOperationException: Token cancel is not supported  for HTTPFS_DELEGATION_TOKEN tokens
      java.lang.UnsupportedOperationException: Token cancel is not supported  for HTTPFS_DELEGATION_TOKEN tokens
      	at org.apache.hadoop.security.token.Token$TrivialRenewer.cancel(Token.java:417)
      	at org.apache.hadoop.security.token.Token.cancel(Token.java:382)
      	at org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.cancel(DelegationTokenRenewer.java:146)
      	at org.apache.hadoop.fs.DelegationTokenRenewer$RenewAction.access$200(DelegationTokenRenewer.java:58)
      	at org.apache.hadoop.fs.DelegationTokenRenewer.removeRenewAction(DelegationTokenRenewer.java:233)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.close(WebHdfsFileSystem.java:790)
      	at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2398)
      	at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2414)
      	at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
      

      WebHDFS doesn't have this problem because it has a TokenRenewer for its delegation tokens ("WEBHDFS delegation" tokens).

      1. HDFS-4951.patch
        2 kB
        Robert Kanter
      2. HDFS-4951.patch
        1 kB
        Robert Kanter

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1483 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1483/)
        HDFS-4951. FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1483 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1483/ ) HDFS-4951 . FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1456 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1456/)
        HDFS-4951. FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1456 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1456/ ) HDFS-4951 . FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #266 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/266/)
        HDFS-4951. FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #266 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/266/ ) HDFS-4951 . FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #4055 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4055/)
        HDFS-4951. FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #4055 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4055/ ) HDFS-4951 . FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer. (rknater via tucu) (Revision 1501451) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1501451 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSKerberosAuthenticator.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/lib/service/DelegationTokenIdentifier.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Robert. Committed to trunk, branch-2, branch-2.1.

        Show
        Alejandro Abdelnur added a comment - Thanks Robert. Committed to trunk, branch-2, branch-2.1.
        Hide
        Robert Kanter added a comment -

        Again, the test failure is unrelated as it happens without the patch and is to be addressed in HDFS-4969.

        Show
        Robert Kanter added a comment - Again, the test failure is unrelated as it happens without the patch and is to be addressed in HDFS-4969 .
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12591473/HDFS-4951.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs-httpfs:

        org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4612//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4612//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591473/HDFS-4951.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4612//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4612//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        +1 pending test-patch report

        Show
        Alejandro Abdelnur added a comment - +1 pending test-patch report
        Hide
        Robert Kanter added a comment -

        That's a good idea; it will prevent anyone from accidentally using it. The new patch also removes the HttpFSKerberosAuthenticator.TOKEN_KIND.

        Show
        Robert Kanter added a comment - That's a good idea; it will prevent anyone from accidentally using it. The new patch also removes the HttpFSKerberosAuthenticator.TOKEN_KIND.
        Hide
        Alejandro Abdelnur added a comment -

        shouldn't we remove the HttpFSKerberosAuthenticator.TOKEN_KIND constant then?

        Show
        Alejandro Abdelnur added a comment - shouldn't we remove the HttpFSKerberosAuthenticator.TOKEN_KIND constant then?
        Hide
        Robert Kanter added a comment -

        Test failure is unreleated; TestHttpFSFWithWebhdfsFileSystem fails even without the patch due to a NPE. I've created HDFS-4969 for that.

        Show
        Robert Kanter added a comment - Test failure is unreleated; TestHttpFSFWithWebhdfsFileSystem fails even without the patch due to a NPE. I've created HDFS-4969 for that.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12591321/HDFS-4951.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs-httpfs:

        org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4608//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4608//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591321/HDFS-4951.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4608//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4608//console This message is automatically generated.
        Hide
        Robert Kanter added a comment -

        Instead of essentially redoing the delegation token renewer (and related) code from WebHDFS for HttpFS, and because the server is using WebHDFSFileSystem anyway, I think the best and simplest solution is to make HttpFS use the same delegation token kind as WebHDFS is using.

        The patch simply changes the token kind that HttpFS's DelegationTokenIdentifier is using from "HTTPFS_DELEGATION_TOKEN" to "WEBHDFS delegation". I manually verified that using the FsShell with HttpFS now works properly.

        Show
        Robert Kanter added a comment - Instead of essentially redoing the delegation token renewer (and related) code from WebHDFS for HttpFS, and because the server is using WebHDFSFileSystem anyway, I think the best and simplest solution is to make HttpFS use the same delegation token kind as WebHDFS is using. The patch simply changes the token kind that HttpFS's DelegationTokenIdentifier is using from "HTTPFS_DELEGATION_TOKEN" to "WEBHDFS delegation". I manually verified that using the FsShell with HttpFS now works properly.

          People

          • Assignee:
            Robert Kanter
            Reporter:
            Robert Kanter
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development