[SPARK-21765] Ensure all leaf nodes that are derived from streaming sources have isStreaming=true

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: SQL, Structured Streaming
• Assignee: Jose Torres
• Reporter: Jose Torres
• Created: 17/Aug/17 17:09
• Updated: 13/Jul/18 05:32
• Resolved: 06/Sep/17 18:27

Description

LogicalPlan has an isStreaming bit, but it's incompletely implemented. Some streaming sources don't set the bit, and the bit can sometimes be lost in rewriting. Setting the bit for all plans that are logically streaming will help us simplify the logic around checking query plan validity.

Issue Links

• contains: SPARK-20672 Keep the `isStreaming` property in triggerLogicalPlan in Structured Streaming
• links to: [Github] Pull Request #18973 (joseph-torres)
• links to: [Github] Pull Request #19056 (joseph-torres)