Critical Description
Kerberized solr nodes accept negotiate/spnego/kerberos requests and processes them. It also passes back to the client a cookie called "hadoop.auth" (which is currently unused, but will eventually be used for delegation tokens).
If two or more nodes are on the same machine, they all send out the cookie which have the same domain (hostname) and same path, but different cookie values.
Upon receipt at the client, if a cookie is rejected (which in this case will be), the client gets a TGT from the KDC. This is causing the heavy traffic at the KDC, plus intermittent "Request is a replay" (which indicates race condition at KDC while handing out the TGT for the same principal). I think having a (well configured) ticket cache is a potential solution, but having cookies get rejected is bad enough.