[SPARK-20441] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0, 2.1.1, 2.1.2
• Fix Version/s: 2.2.0
• Component/s: Structured Streaming
• Assignee: Unassigned
• Reporter: Liwei Lin
• Created: 23/Apr/17 10:11
• Updated: 25/Dec/18 02:21
• Resolved: 03/May/17 15:57

Description

Within the same streaming query, when one StreamingRelation is referred multiple times – e.g. df.union(df) – we should transform it only to one StreamingExecutionRelation, instead of two or more different StreamingExecutionRelations.

Issue Links

• is related to: SPARK-20432 Unioning two identical Streaming DataFrames fails during attribute resolution
• links to: [Github] Pull Request #17735 (lw-lin)
• links to: GitHub Pull Request #17735