[HDFS-15545] (S)Webhdfs will not use updated delegation tokens available in the ugi after the old ones expire

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: webhdfs
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.2
• Hadoop Flags: Reviewed
• Assignee: Issac Buenrostro
• Reporter: Issac Buenrostro
• Created: 27/Aug/20 23:23
• Updated: 27/Jan/24 03:11
• Resolved: 12/Nov/20 22:58

Description

WebHdfsFileSystem can select a delegation token to use from the current user UGI. The token selection is sticky, and WebHdfsFileSystem will re-use it every time without searching the UGI again.

If the previous token expires, WebHdfsFileSystem will catch the exception and attempt to get a new token. However, the mechanism to get a new token bypasses searching for one on the UGI, so even if there is external logic that has retrieved a new token, it is not possible to make the FileSystem use the new, valid token, rendering the FileSystem object unusable.

A simple fix would allow WebHdfsFileSystem to re-search the UGI, and if it finds a different token than the cached one try to use it.

Issue Links

• is related to: HADOOP-17345 Work with externally managed user credentials
• links to: GitHub Pull Request #2255