[KAFKA-10375] Restore consumer fails with SSL handshake fail exception

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.0
• Component/s: streams
• Assignee: Satyawati Tripathi
• Reporter: Levani Kokhreidze
• Created: 08/Aug/20 12:02
• Updated: 08/Sep/20 04:35

Description

After upgrading to 2.6, we started getting "SSL handshake fail" exceptions. Curios thing is that it seems to affect only restore consumers. For mTLS, we use dynamic certificates that are being reloaded automatically every X minutes.

We didn't have any issues with it, up until upgrading 2.6 and other stream processing jobs running Kafka 2.4 don't have similar problems.

After restarting the Kafka Streams instance, issue goes away.

 

From the stacktrace, it's visible that problem is:

Aug 07 10:36:12.478 | Caused by: java.security.cert.CertificateExpiredException: NotAfter: Fri Aug 07 07:45:16 GMT 2020 

Seems like somehow restore consumer gets stuck with old certificate and it's not refreshed.