[HIVE-24334] SyntheticJoinPredicate creation may be missed when ReduceSink has Join input

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Physical Optimizer
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 30/Oct/20 07:08
• Updated: 06/Jan/21 01:09

Description

Let assume we have a plan where some Reduce Sink operators has a Join operator as input:

TS[33]-FIL[34]-SEL[35]-RS[42]-JOIN[44]-RS[45]-JOIN[47]
TS[36]-FIL[37]-SEL[38]-RS[43]-JOIN[44]
TS[39]-FIL[40]-SEL[41]-RS[46]-JOIN[47]

RS[45] inputs is JOIN[44].
When searching for additional opportunities to create additional SyntheticJoinPredicates ExprNodeDescUtils.backtrack does not return the input expression of the expression in the join operator but the expression itself.
This is caused by

• if the operator is a join operator where we create the join predicate derivatives the expression is not resolved
  https://github.com/apache/hive/blob/375433510b73c5a22bde4e13485dfc16eaa24706/ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java#L400
• later the backtrack algorithm does the resolution of the expression doesn't do any iterations since it is already in a terminal state when it is called
  https://github.com/apache/hive/blob/375433510b73c5a22bde4e13485dfc16eaa24706/ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java#L414

Issue Links

• links to: GitHub Pull Request #1631