[SPARK-21908] Checkpoint broadcast variable in spark streaming job

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 1.6.0
• Component/s: DStreams
• Assignee: Unassigned
• Reporter: Venkat Gurukrishna
• Created: 04/Sep/17 14:42
• Updated: 04/Sep/17 15:07
• Resolved: 04/Sep/17 14:44

Description

In our Spark 1.6 CDH 5.8.3, job application, we are using the broadcast variables and when we checkpoint them and restart the spark job getting error.
Even tried with making the broadcast variable as transient. But we are getting different exception.
I have checked this JIRA link: https://issues.apache.org/jira/browse/SPARK-5206
which had mentioned to use singleton reference to broadcast variable and also to use the transient.
Whether this needs to be done in the driver side or at the executor side?