Attaching a preliminary patch to facilitate discussions on how code should be structured and what interfaces should be adopted. It's incomplete, but should suffice for a high-level discussion.
A few new classes are introduced in the patch. DelegationTokenHandler is the main class. It is responsible for choosing master keys and updating them. It also uses master keys to generate, re-generate or verify tokens. Master keys are not exposed. One must invoke methods on a DelegationTokenHandler for operations involving master keys. A DelegationTokenHandler is only needed on the server-side. Client uses tokens generated by a handler and does not require access to the handler.
DelegationToken is the token object generated by the handler and sent to the client over the wire. It supports serialization and should contain methods that are necessary for the client to use the token. A token has two fields, tokenID and tokenAuth. When used for authentication, tokenID is effectively used as username and tokenAuth is the password. A client doesn't need to understand what is encoded in the tokenID. Both fields are opaque to the user. TokenID can be used to encode as many data items as necessary and server needs to be able to encode and decode those items. That logic is captured in the handler. We may add different types of tokens in the future and each type will have its own handler. The tokens generated by different handlers may have different types but otherwise look similar to the client.
Note that client needs to be able to find out who the tokenIssuer is so that it can pick the right token to use from its cache (a token can only be used against its issuing server). A tokenIssuer field or getTokenIssuer() method needs to be added to DelegationToken or its superclass Token. Let's ignore this issue for now.
An alternative design is to break tokenID into individual data items and store those items as separate fields inside the token. TokenID can be computed on demand from those fields. The benefit is that users are able to retrieve those fields without de-serializing. However, apart from tokenIssuer mentioned above and possibly expirationDate, it's unclear the client needs to know anything more about the token. Moreover, different types of tokens may require different fields and it would be harder to abstract a common Token class than the opaque model. Maybe we should just separate out tokenIssuer and expirationDate, and put everything else into an opaque field, which only the handler understands.
Another alternative is to move operations that use master keys from DelegationTokenHandler to DelegationToken, making the handler a key storage facility and doesn't care about how keys are used. However, the keys need to be used to initialize Mac objects, which are then used to compute tokenAuth. It's better to share a single Mac object for each key. The handler provides a convenient place to synchronize accesses to the shared Mac object. Conceptually, a token is a credential and once it's generated it doesn't change. The client that uses a token never needs to re-compute tokenAuth or needs access to the key. The server does need access to the key and it seems a good idea to hide the key in a single place - the handler, where all the key management is happening.