[HIVE-23882] Compiler extensions for MJ probe optimization

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Panagiotis Garefalakis
• Reporter: Panagiotis Garefalakis
• Created: 20/Jul/20 12:28
• Updated: 17/Nov/22 08:48
• Resolved: 19/Feb/21 21:52

Description

In probe we cannot currently support Key expressions (on the big table Side) as ORC CVs Probe directly the smalltable HT (there is no expr evaluation at that level).

TezCompiler should take this into account when picking MJs to push probe details, In this ticket we extend probedecode compiler logic to:

• Validate that MJ is a single Key join, where the bigTable keyCol is only a ExprNodeColumnDesc
• For the selected MJ use backtracking logic to find the orignal TS key it maps to (on the same vertex) - this actually revealed several missed or wrong optimizations (updated q outs)
• Finally, extend the optimization logic to check if there a type Cast between src and destination (as the types have to match in the probe case) and only use it on the LLAP mode for now

Issue Links

• links to: GitHub Pull Request #1286