Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6404

Call broadcast() in each interval for spark streaming programs.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • None
    • None
    • Spark Core
    • None

    Description

      If I understand it correctly, Spark’s broadcast() function will be called only once at the beginning of the batch. For streaming applications that need to run for 24/7, it is often needed to update variables that shared by broadcast() dynamically. It would be ideal if broadcast() could be called at the beginning of each interval.

      Attachments

        Activity

          People

            Unassigned Unassigned
            heavenstar@gmail.com Yifan Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: