[HDFS-3367] WebHDFS doesn't use the logged in user when opening connections

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.23.0, 1.0.2, 2.0.0-alpha, 3.0.0-alpha1
• Component/s: webhdfs
• Target Version/s: 1.2.0, 2.0.3-alpha, 0.23.7
• Hadoop Flags: Reviewed
• Assignee: Daryn Sharp
• Reporter: Jakob Homan
• Created: 04/May/12 01:21
• Updated: 12/May/16 18:17
• Resolved: 14/Mar/13 15:20

Description

Something along the lines of

UserGroupInformation.loginUserFromKeytab(<blah blah>)
Filesystem fs = FileSystem.get(new URI("webhdfs://blah"), conf)

doesn't work as webhdfs doesn't use the correct context and the user shows up to the spnego filter without kerberos credentials:

Exception in thread "main" java.io.IOException: Authentication failed, url=http://<NN>:50070/webhdfs/v1/?op=GETDELEGATIONTOKEN&user.name=<USER>
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:337)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.httpConnect(WebHdfsFileSystem.java:347)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:403)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:675)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initDelegationToken(WebHdfsFileSystem.java:176)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.initialize(WebHdfsFileSystem.java:160)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
...
Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:232)
	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:141)
	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:217)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getHttpUrlConnection(WebHdfsFileSystem.java:332)
	... 16 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:130)
...

Explicitly getting the current user's context via a doAs block works, but this should be done by webhdfs.

Issue Links

• requires: HDFS-4542 Webhdfs doesn't support secure proxy users