[HIVE-24761] Vectorization: Support PTF - bounded start windows

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: László Bodor
• Reporter: László Bodor
• Created: 10/Feb/21 06:15
• Updated: 17/Nov/22 08:48
• Resolved: 25/May/21 21:59

Description

 notVectorizedReason: PTF operator: *** only UNBOUNDED start frame is supported

Currently, bounded windows are not supported in VectorPTFOperator. If we simply remove the check compile-time:
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java#L2911

      if (!windowFrameDef.isStartUnbounded()) {
        setOperatorIssue(functionName + " only UNBOUNDED start frame is supported");
        return false;
      }

We get incorrect results, that's because vectorized codepath completely ignores boundaries, and simply iterates through all the input batches in VectorPTFGroupBatches:

    for (VectorPTFEvaluatorBase evaluator : evaluators) {
      evaluator.evaluateGroupBatch(batch);
      if (isLastGroupBatch) {
        evaluator.doLastBatchWork();
      }
    }

Issue Links

• causes: HIVE-26074 PTF Vectorization: BoundaryScanner for varchar
• Dependent: HIVE-24945 PTF: Support vectorization for lead/lag functions
• is related to: HIVE-17164 Vectorization: Support PTF (Part 2: Unbounded Support-- Turn ON by default)
• is related to: HIVE-16369 Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
• relates to: HIVE-24905 only CURRENT ROW end frame is supported for RANGE
• links to: GitHub Pull Request #2099