When obtaining the tokens for a FileSystem, the TokenCache will read the binary token file if a token is not already in the Credentials. However, it will overwrite any existing tokens in the Credentials with the contents of the binary token file if a single token is missing. This may cause new tokens to be replaced with invalid/cancelled tokens from the binary file. The new tokens will not be canceled, and thus "leak" in the namenode until they expire.
The binary tokens should be merged with, but not replace, existing tokens in the Credentials.
The code that reads the binary token file is prefaced with:
Also, the loading of the binary token file is the only reason that the TokenCache has to use getCanonicalService. If this linkage can be broken, then the 1-to-1 filesystem to token service coupling may be removed. And use of getCanonicalService can be removed in a subsequent jira.