Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43482

Expand QueryTerminatedEvent to contain error class if it exists in exception

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • Structured Streaming
    • None

    Description

      As of now, the event QueryTerminatedEvent has a field about exception which is just a string representation of underlying exception, mostly stack trace. That said, people relying on streaming query listener can't get any benefit from the movement for employing error class framework. This is a bad observability once we migrate exceptions in streaming query to error class framework where exceptions are well categorized.

      We can expand the event of QueryTerminatedEvent - if the exception contains error class information, we can ship this along with existing string version of exception (message).

      Attachments

        Activity

          People

            kabhwan Jungtaek Lim
            kabhwan Jungtaek Lim
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: