Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27391

deadlock in ContinuousExecution unit tests

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 3.0.0
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      ContinuousExecution (in the final query execution phrase) holds the lazy val lock of its IncrementalExecution for the entire duration of the (indefinite length) job. This can cause deadlocks in unit tests, which hook into internal APIs and try to instantiate other lazy vals.

       

      (Note that this should not be able to affect production.)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                joseph.torres Jose Torres
                Reporter:
                joseph.torres Jose Torres
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: