Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Invalid
-
None
-
None
-
None
Description
If I understand it correctly, Spark’s broadcast() function will be called only once at the beginning of the batch. For streaming applications that need to run for 24/7, it is often needed to update variables that shared by broadcast() dynamically. It would be ideal if broadcast() could be called at the beginning of each interval.