[SPARK-14160] Windowing for structured streaming

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Structured Streaming
• Assignee: Burak Yavuz
• Reporter: Burak Yavuz
• Created: 25/Mar/16 17:47
• Updated: 01/Nov/16 22:15
• Resolved: 01/Apr/16 20:19

Description

This JIRA is to track the status regarding event time windowing operations for Continuous queries.

The proposition is to add a

window(timeColumn, windowDuration, slideDuration, startTime)

expression that will bucket time columns into time windows. This expression will be useful in both batch analysis and streaming. With streaming, it will open up the use case for event-time window aggregations.

For R, and Python interoperability, we will take windowDuration, slideDuration, startTime as strings and parse interval lengths.

Issue Links

• is related to: SPARK-14353 Dateset Time Windowing API for Python, R, and SQL
• links to: [Github] Pull Request #12008 (brkyvz)