[HIVE-15520] Improve the sum performance for Range based window - ASF JIRA

XML

Word

Printable

JSON

Release Note:

Hide
This is to improve the performance of sum function over range based windowing.

One issue related to sum(lag(x)) over (partition by c1 order by c2 range between ...) and sum(lead(x)) over (partition by c1 order by c2 range between ...) has been fixed which would produce different result. Without the patch, lag(x)/lead(x) would only consider the previous/next row in the windowing, not within the partition, which doesn't match other databases, also doesn't match rows based windowing.

Show
This is to improve the performance of sum function over range based windowing. One issue related to sum(lag(x)) over (partition by c1 order by c2 range between ...) and sum(lead(x)) over (partition by c1 order by c2 range between ...) has been fixed which would produce different result. Without the patch, lag(x)/lead(x) would only consider the previous/next row in the windowing, not within the partition, which doesn't match other databases, also doesn't match rows based windowing.

Currently streaming process is not supported for range based windowing. Thus sum( x ) over (partition by y order by z) is O(n^2) running time.

Investigate the possibility of streaming support.

links to

Code Review