Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4576 Webhdfs authentication issues
  3. HDFS-3367

WebHDFS doesn't use the logged in user when opening connections

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 0.23.0, 1.0.2, 2.0.0-alpha, 3.0.0-alpha1
    • None
    • webhdfs
    • None

    Description

      Something along the lines of

      UserGroupInformation.loginUserFromKeytab(<blah blah>)
      Filesystem fs = FileSystem.get(new URI("webhdfs://blah"), conf)
      

      doesn't work as webhdfs doesn't use the correct context and the user shows up to the spnego filter without kerberos credentials:

      Exception in thread "main" java.io.IOException: Authentication failed, url=http://<NN>:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&user.name=<USER>
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:337)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.httpConnect(WebHdfsFileSystem.java:347)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:403)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:675)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initDelegationToken(WebHdfsFileSystem.java:176)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initialize(WebHdfsFileSystem.java:160)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
      ...
      Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:232)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:141)
      	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:217)
      	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:332)
      	... 16 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:130)
      ...

      Explicitly getting the current user's context via a doAs block works, but this should be done by webhdfs.

      Attachments

        1. HDFS-3367.branch-23.patch
          3 kB
          Daryn Sharp
        2. HDFS-3367.branch-23.patch
          3 kB
          Daryn Sharp
        3. HDFS-3367.patch
          3 kB
          Daryn Sharp
        4. HDFS-3367.patch
          3 kB
          Daryn Sharp
        5. HDFS-3367.patch
          3 kB
          Daryn Sharp

        Issue Links

          Activity

            People

              daryn Daryn Sharp
              jghoman Jakob Homan
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: