[HDFS-15765] Add support for Kerberos and Basic Auth in webhdfs

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: hadoop-client
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Pushpendra Singh
• Created: 06/Jan/21 10:50
• Updated: 02/Apr/23 04:03

Description

webhdfs's HTTP operation like get ( GetOpParam.java) operation and other HTTP operation has 'requireAuth' set to false and expected to work with Delegation token only. However, when working with webhdfs over Apache Knox, delegation token authentication is not supported, we should support Kerberos authentication (SPNEGO) or Basic authentication for WebHdfsFileSystem if user turns on a configuration.

Further webhdfs (WebHDFSFileSystem.java)  is calling 'public URLConnection openConnection(URL url)' and providing no way to use the kerberos authentication, if configured.

Even after setting the UserGroupInformation with user name and keytab, openConnection is not using the keytab for authentication.

Also WebHdfsFileSystem doesn't provide any support for HTTP BASIC authentication (username/password). Provide support to read the password via environment variable.

Issue Links

• links to: GitHub Pull Request #5447