[HADOOP-18623] S3A delegation token implementations to be able to update tokens from the user credentials

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.3.5
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 07/Feb/23 17:06
• Updated: 09/Feb/23 17:35

Description

Spark never renews tokens, instead it can create new ones and attach them to the current users credentials.

This means long-running S3A instances which can pick up new tokens/credentials need a way to look for new tokens in the credential chain.

Proposed

• class AbstractDelegationTokenBinding adds a CallableRaisingIOE field which can be updated with a callback
• S3ADelegationTokens to add method boolean maybeUpdateTokenFromOwner() to look for any new token and switch to it if new
• S3ADelegationTokens serviceInit() to pass the method down to the instantiated DT binding as the callback

It is up to the token binding implementation to decide what to do about it; the standard implementations will do: nothing.

Issue Links

• links to: GitHub Pull Request #5365