When a cluster has been Kerberized, alerts use the curl_krb_request module in order to make requests using SPNEGO negotiation.
Normally this would involve calling kinit and then invoking the curl command to use the acquired ticket. However, because alerts run often on fixed intervals, this would mean that the KDC would be flooded with requests every minute.
To alleviate this problem, curl_krb_request uses klist to inspect the KRB5CCNAME cache. Only if an invalid ticket is found is kinit invoked. Additionally, kinit is invoked with a fixed ticket lifetime of 5 minutes. Since many alerts run on 5-minute intervals, this causes boundary issues.
To workaround these problems while continuing to leverage the cache, curl_krb_request should be changed to:
- Use the default ticket expiry configured for Kerberos in krb5.conf
- Employ in-memory tracking of the last time kinit was called so that it can be invoked before hitting the boundary of the ticket's expiration time