Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11761

Prevent the call to StreamingContext#stop() in the listener bus's thread

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • DStreams
    • None

    Description

      Quoting Shixiong's comment from https://github.com/apache/spark/pull/9723 :

      The user should not call stop or other long-time work in a listener since it will block the listener thread, and prevent from stopping SparkContext/StreamingContext.
      
      I cannot see an approach since we need to stop the listener bus's thread before stopping SparkContext/StreamingContext totally.
      

      Proposed solution is to prevent the call to StreamingContext#stop() in the listener bus's thread.

      Attachments

        Activity

          People

            tedyu@apache.org Ted Yu
            yuzhihong@gmail.com Ted Yu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: