Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18908

It's hard for the user to see the failure if StreamExecution fails to create the logical plan

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: Structured Streaming
    • Labels:
      None
    • Target Version/s:

      Description

      If the logical plan fails to create, e.g., some Source options are invalid, the user cannot use the code to detect the failure. The only place receiving this error is Thread's UncaughtExceptionHandler.

      This bug is because logicalPlan is lazy, and when we try to create StreamingQueryException to wrap the exception thrown by creating logicalPlan, it calls logicalPlan agains.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zsxwing Shixiong Zhu
                Reporter:
                zsxwing Shixiong Zhu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: