[KAFKA-5945] Improve handling of authentication failures when credentials are removed

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0.0
• Component/s: clients
• Assignee: Rajini Sivaram
• Reporter: Rajini Sivaram
• Created: 20/Sep/17 18:42
• Updated: 17/Feb/19 18:57

Description

KAFKA-5854 improves the handling of authentication failures. The scope of KAFKA-5854 was limited to a specific scenario - provide better feedback to applications when security is misconfigured. The PR improves diagnostics for this scenario by throwing an AuthenticationException and also avoids retries. To enable this, the first request initiated by any public API was updated to throw authentication exceptions.

This JIRA is for a more extensive handling of authentication exceptions which also includes proper handling of credential updates at any time. If a credential is removed, then we could see authentication exception from any request and we want to propagate this properly. This needs quite extensive testing and is less likely to be hit by users, so it will be done later under this JIRA.

The gaps that need covering are:
1. Ensure authentication failures are processed in the Network client for any request
2. Ensure metadata refresh failures are handled properly at any time
3. Heartbeat threads and other background threads should handle authentication failures. Threads should not terminate on failure, but should avoid retries until application performs a new operation.