That seems pretty convoluted.
In the future, would appreciate you explaining why a proposal seems convoluted. It seems quite straightforward to me, so not sure how to address this comment. This proposal is an attempt to compromise and address your concern, which I understood to be not wanting to have this method baked into the KeyProvider interface, and thus allow some implementations to use it and others to not.
A. For #1 can't we have a standalone DelegationTokenClient component - especially since there is another jira for refactoring delegation token support out into common to be more reusable? Such a client could then potentially be used inside the KMSClientKeyProvider.
That JIRA seems to me to be orthogonal to this one, so I don't think we should couple the two. How the KmsClientKeyProvider gets tokens under the hood shouldn't have anything to do with the API. Also, as you point out later in question C, it will still be necessary for the submitting code to somehow call/interact with the tokens/credentials of the KeyProvider at submission time, so I don't think it's actually possible to entirely encapsulate the delegation token fetching/storage within the KeyProvider implementation.
B. Wouldn't it be better if providers that know they need delegation tokens were able to handle #2 themselves?
How about changing the proposal to mimic what's done in FileSystem today and add a method like "public Token<?> addDelegationTokens(final String renewer, Credentials credentials)" to the KeyProvider API? The default behavior would be to add no tokens to the provided Credentials object, but the KmsClientKeyProvider could instead fetch and stash away the tokens in the provided Credentials object.
C. How is #3 above going to be handled using the current interfaces - I don't see how it is being added to the interaction currently?
I believe this will happen transparently, because the tokens contained in the Credentials object will be added to the UGI object which will then be used to authenticate all the RPCs. The KeyProvider shouldn't need access to the tokens in the tasks.
D. If the KMSClientKeyProvider had access to the credentials object ( already have access to UserKeyProvider) or some other execution context itself then could that be a way that #3 could be addressed?
If I'm understanding you correctly, I think this basically the same as what I'm proposing above in response to your question B. Am I right about that? Will this work for you?