[SPARK-15812] Allow sorting on aggregated streaming dataframe when the output mode is Complete

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Structured Streaming
• Assignee: Tathagata Das
• Reporter: Tathagata Das
• Created: 08/Jun/16 01:56
• Updated: 01/Nov/16 22:15
• Resolved: 10/Jun/16 17:48

Description

When the output mode is complete, then the output of a streaming aggregation essentially will contain the complete aggregates every time. So this is not different from a batch dataset within an incremental execution. Other non-streaming operations should be supported on this dataset. In this JIRA, we are just adding support for sorting, as it is a common useful functionality. Support for other operations will come later.

Issue Links

• links to: [Github] Pull Request #13549 (tdas)