Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7621

Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 1.3.1
    • Fix Version/s: 1.3.2, 1.4.0
    • Component/s: DStreams
    • Labels:
      None
    • Flags:
      Patch

      Description

      Currently, when a MessageHandler (for any of the Kafka Receiver implementations) encounters an error handling a message, the error is only logged with:

      case e: Exception => logError("Error handling message", e)
      

      It would be incredibly useful to be able to notify any registered StreamingListener of this receiver error (especially since this try...catch block masks more fatal Kafka connection exceptions).

        Attachments

        1. SPARK-7621.patch
          2 kB
          Jeremy A. Lucas

          Activity

            People

            • Assignee:
              jerluc Jeremy A. Lucas
              Reporter:
              jerluc Jeremy A. Lucas
            • Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified