Description
There was an issue with one of our Kafka consumers no longer sending a valid OAuth token. Looking at the logs, there seems to be an error in some of the math in the timestamp calculation:
14 Feb 2019 06:42:45,694 Expiring credential expires at 2019-02-14T06:48:21.000+0000, so buffer times of 60 and 300 seconds at the front and back, respectively, cannot be accommodated. We will refresh at 2019-02-14T06:01:39.078+0000. 14 Feb 2019 06:42:45,694 org.apache.kafka.common.utils.KafkaThread: Uncaught exception in thread java.lang.IllegalArgumentException: timeout value is negative at java.lang.Thread.sleep(Native Method) ~[?:1.8.0_202] at org.apache.kafka.common.utils.SystemTime.sleep(SystemTime.java:45) ~[kafka-clients-2.x.jar:?] at org.apache.kafka.common.security.oauthbearer.internals.expiring.ExpiringCredentialRefreshingLogin$Refresher.run(ExpiringCredentialRefreshingLogin.java:86) ~[kafka-clients-2.x.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_202]
At this point the refresh logic would never recover and so the producer couldn't consume until we restarted the process.
Attachments
Issue Links
- links to