Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.0
-
None
Description
See parent ticket for more information. Two proposals with sketches:
Proposal 1. create a version of rangeBetween that accepts Strings, i.e. rangeBetween(String, String). This is obviously very flexible, but less type safe.
Proposal 2. creates a new type called WindowFrameBoundary:
trait WindowFrameBoundary object WindowFrameBoundary { def unboundedPreceding: WindowFrameBoundary def unboundedFollowing: WindowFrameBoundary def currentRow: WindowFrameBoundary def at(value: Long) def interval(interval: String) }
And create a new rangeBetween that accepts WindowFrameBoundary's, i.e.
def rangeBetween(start: WindowFrameBoundary, end: WindowFrameBoundary)
This is also very flexible and type safe at the same time.
Note the two are not mutually exclusive, and we can also deprecate the existing confusing APIs.
Attachments
Issue Links
- blocks
-
SPARK-25844 Implement Python API once we have a new API
- Open