[IMPALA-9620] Predicates in the SELECT and GROUP-BY cause failure with CNF rewrite enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.4.0
• Fix Version/s: Impala 4.0.0
• Component/s: Frontend
• Epic Color: ghx-label-2
• Assignee: Aman Sinha
• Reporter: Aman Sinha
• Created: 08/Apr/20 00:55
• Updated: 01/Dec/22 17:37
• Resolved: 10/Apr/20 15:04

Description

Predicates can appear in the SELECT and GROUP BY list as part of IF(), CASE() clauses.
When enable_cnf_rewrites is set to true, such queries encounter failure during planning as shown below. Queries run successfully when the flag is disabled.

Note that the predicate does not have to be disjunctive predicate for this failure to occur..even other types of predicates repro the issue.

set enable_cnf_rewrites = true;

select l_quantity, if(l_quantity < 5 or l_quantity > 45, 'invalid', 'valid') from lineitem group by l_quantity, if(l_quantity < 5 or l_quantity > 45, 'invalid', 'valid') limit 5

ERROR: IllegalStateException: null

Stack trace:

I0407 17:40:40.306650 31240 jni-util.cc:288] 2741e90d2edac592:c625a35f00000000] java.lang.IllegalStateException
        at com.google.common.base.Preconditions.checkState(Preconditions.java:492)
        at org.apache.impala.analysis.SlotRef.getIdsHelper(SlotRef.java:229)
        at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
        at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
        at org.apache.impala.analysis.Expr.getIds(Expr.java:1279)
        at org.apache.impala.rewrite.ConvertToCNFRule.convertToCNF(ConvertToCNFRule.java:111)
        at org.apache.impala.rewrite.ConvertToCNFRule.apply(ConvertToCNFRule.java:86)
        at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
        at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:83)
        at org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
        at org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
        at org.apache.impala.analysis.SelectStmt.rewriteCheckOrdinalResult(SelectStmt.java:1043)
        at org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:1068)
        at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:472)
        at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1530)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1497)

Other variations of the same pattern that also encounter similar failure:

explain select case when not (l_quantity = 5) then 0 else 1 end from lineitem group by case when not (l_quantity = 5) then 0 else 1 end
ERROR: IllegalStateException: null

Issue Links

• causes: IMPALA-10096 May throw exception after expr rewritten , if the group by ordinal reference is still a numeric literal
• is related to: IMPALA-11770 Review Predicate arguments for impact of CNF rewrites
• is related to: IMPALA-10865 Query with select alias the same as column name rewrite error for grouping exprs
• relates to: IMPALA-9183 TPC-DS query 13 - customer_address predicates not propagated to scan
• links to: Gerrit review