[SPARK-17893] Window functions should also allow looking back in time

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.0.1
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Raviteja Lokineni
• Created: 12/Oct/16 20:14
• Updated: 13/Oct/16 12:08
• Resolved: 13/Oct/16 09:25

Description

This function should allow looking back. The current window(timestamp, duration) seems to be for looking forward in time.

Example:

dataFrame.groupBy(window("date", "7 days ago")).agg(min("col1"), max("col1"))

For example, if date: 2013-01-07 then the window should be 2013-01-01 - 2013-01-07