[FLINK-28247] Exception will be thrown when over window contains grouping in Hive Dialect

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.16.0
• Labels: pull-request-available
• Assignee: luoyuxia
• Reporter: luoyuxia
• Created: 24/Jun/22 12:10
• Updated: 08/Jul/22 04:11
• Resolved: 08/Jul/22 04:11

Description

The exception will be reprodued by the following sql when using Hive Dialect:

create table t(category int, live int, comments int)
SELECT grouping(category), lag(live) over(partition by grouping(category)) FROM t GROUP BY category, live; 

The reson is it will first call `HiveParserCalcitePlanner#genSelectForWindowing` to generate the window, which will then call `HiveParserUtils#rewriteGroupingFunctionAST` to rewrite the group function in the over window :

 

// rewrite grouping function
if (current.getType() == HiveASTParser.TOK_FUNCTION
        && current.getChildCount() >= 2) {
    HiveParserASTNode func = (HiveParserASTNode) current.getChild(0);
    if (func.getText().equals("grouping")) {
        visited.setValue(true);
        convertGrouping(
                current, grpByAstExprs, noneSet, legacyGrouping, found);
    }
} 

 

So `grouping(category)` will be converted to `grouping(0, 1)`. 

After `HiveParserCalcitePlanner#genSelectForWindowing`, it will try to rewrite it again:

 

if (!qbp.getDestToGroupBy().isEmpty()) {
    // Special handling of grouping function
    expr =
            rewriteGroupingFunctionAST(
                    getGroupByForClause(qbp, selClauseName),
                    expr,
                    !cubeRollupGrpSetPresent);
} 

And it will also fall back to `convertGrouping` again as `current.getChildCount() >= 2` will be true. But then, it can't find any field 

presented  in group by  for it's `grouping(0, 1)` now.

 

Issue Links

• links to: GitHub Pull Request #20071