[HIVE-24710] Optimise PTF iteration for count(*) to reduce CPU and IO cost

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: HiveServer2
• Labels: performance pull-request-available
• Assignee: Rajesh Balamohan
• Reporter: Rajesh Balamohan
• Created: 01/Feb/21 05:37
• Updated: 17/Nov/22 08:47
• Resolved: 18/Feb/21 00:50

Description

E.g query

select x, y, count(*) over (partition by x order by y range between 86400 PRECEDING and CURRENT ROW) r0 from foo

1. In such cases, there is no need to iterate over the rowcontainers often (internally it does O(n^2) operations taking forever when window frame is really large). This can be optimised to reduce CPU burn and IO.
2. BasePartitionEvaluator::calcFunctionValue need not materialize ROW when parameters are empty. This codepath can also be optimised.

Issue Links

• links to: GitHub Pull Request #1940