[ATLAS-2047] NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8-incubating
• Fix Version/s: 0.8.2, 1.0.0
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 18/Aug/17
• Created: 15/Aug/17 16:49
• Updated: 17/May/18 05:00
• Resolved: 11/Sep/17 20:46

Description

Background

KafkaConsumer is abstracted by AtlasKafkaConsumer. This is run using HookConsumer which is derived from kafka.utils.ShutdownableThread.

The ShutdownableThread manages the thread. It handles exceptions thrown in the doWork method and logs them.

Problem
Exception reported in the bug is thrown by KafkaConsumer is handed by HookConsumer. Exception is logged but the thread keeps running. In cases where Kafa is in irrecoverable state, this behavior ends up filling up logs and providing no value in return.

Solution
Let ShutdownableThread hand exceptions thrown by KafkaConsumer. Stop the thread if KafkaConsumer suffers from irrecoverable error. This will avoid the situation described in the problem section.