[SPARK-7621] Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.3.1
• Fix Version/s: 1.3.2, 1.4.0
• Component/s: DStreams
• Flags: Patch
• Assignee: Jeremy A. Lucas
• Reporter: Jeremy A. Lucas
• Created: 14/May/15 00:38
• Updated: 19/May/15 01:22
• Resolved: 19/May/15 01:22

Description

Currently, when a MessageHandler (for any of the Kafka Receiver implementations) encounters an error handling a message, the error is only logged with:

case e: Exception => logError("Error handling message", e)

It would be incredibly useful to be able to notify any registered StreamingListener of this receiver error (especially since this try...catch block masks more fatal Kafka connection exceptions).

Issue Links

• links to: [Github] Pull Request #6204 (jerluc)