[HIVE-12666] PCRExprProcFactory.GenericFuncExprProcessor.process() aggressively removes dynamic partition pruner generated synthetic join predicates.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Assignee: Hari Sankar Sivarama Subramaniyan
• Reporter: Hari Sankar Sivarama Subramaniyan
• Created: 14/Dec/15 20:56
• Updated: 02/Jun/16 10:17
• Resolved: 15/Dec/15 19:33

Description

Introduced by HIVE-11634. The original idea in HIVE-11634 was to remove the IN partition conditions from the predicate list since the static dynamic partitioning would kick in and push these predicates down to metastore. However, the check is too aggressive and removes events such as below :

-                    Select Operator
-                      expressions: UDFToDouble(UDFToInteger((hr / 2))) (type: double)
-                      outputColumnNames: _col0
-                      Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
-                      Group By Operator
-                        keys: _col0 (type: double)
-                        mode: hash
-                        outputColumnNames: _col0
-                        Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
-                        Dynamic Partitioning Event Operator
-                          Target Input: srcpart
-                          Partition key expr: UDFToDouble(hr)
-                          Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
-                          Target column: hr
-                          Target Vertex: Map 1

Issue Links

• relates to: HIVE-12462 DPP: DPP optimizers need to run on the TS predicate not FIL
• relates to: HIVE-11634 Support partition pruning for IN(STRUCT(partcol, nonpartcol..)...)
• links to: Review board