Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4576 Webhdfs authentication issues
  3. HDFS-3367

WebHDFS doesn't use the logged in user when opening connections

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 1.0.2, 2.0.0-alpha, 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: webhdfs
    • Labels:
      None

      Description

      Something along the lines of

      UserGroupInformation.loginUserFromKeytab(<blah blah>)
      Filesystem fs = FileSystem.get(new URI("webhdfs://blah"), conf)
      

      doesn't work as webhdfs doesn't use the correct context and the user shows up to the spnego filter without kerberos credentials:

      Exception in thread "main" java.io.IOException: Authentication failed, url=http://<NN>:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&user.name=<USER>
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:337)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.httpConnect(WebHdfsFileSystem.java:347)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:403)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:675)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initDelegationToken(WebHdfsFileSystem.java:176)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initialize(WebHdfsFileSystem.java:160)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
      ...
      Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:232)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:141)
      	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:217)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:332)
      	... 16 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:130)
      ...

      Explicitly getting the current user's context via a doAs block works, but this should be done by webhdfs.

        Attachments

        1. HDFS-3367.patch
          3 kB
          Daryn Sharp
        2. HDFS-3367.branch-23.patch
          3 kB
          Daryn Sharp
        3. HDFS-3367.patch
          3 kB
          Daryn Sharp
        4. HDFS-3367.patch
          3 kB
          Daryn Sharp
        5. HDFS-3367.branch-23.patch
          3 kB
          Daryn Sharp

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                jghoman Jakob Homan
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: