Apologies for not being thorough enough earlier. But I think the API needs a little bit more of tweaking.
I think in TokenCache, all APIs should be static (the addDelegationToken is not).
The TokenStorage class should be annotated with InterfaceAudience Private. The TokenCache should be annotated with InterfaceStability Evolving.
As far as possible, TokenCache should be the one that should be used by the framework and the applications. TokenStorage should be a helper class that is used internally by the TokenCache. In that sense, some of the APIs like TokenCache.getTokenStorage could be package private (so that you can still use it in your new tests to inspect the fields). TokenCache.setStorage should be annotated with InterfaceAudience Private (since you use it in LocalJobRunner, and hence the method cannot be made package private).
Can we move the loadJobToken calls to TokenStorage. The TokenCache internally checks for whether the tokenStorage field is null or not on every API invocation. If the field is null, it invokes the static method TokenStorage.loadJobToken, and gets an initialized TokenStorage object.
The setJobToken/getJobToken could be moved to TokenCache but annotated with InterfaceAudience Private.
Doing the above would make the TokenCache as the single point of entry to the token storage and make the APIs more clear IMO.
Does these make sense? Again, apologies for not commenting on these earlier.