Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22731 MJ probe decode with row-level filtering
  3. HIVE-23882

Compiler extensions for MJ probe optimization

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: None

      Description

      In probe we cannot currently support Key expressions (on the big table Side) as ORC CVs Probe directly the smalltable HT (there is no expr evaluation at that level).

      TezCompiler should take this into account when picking MJs to push probe details, In this ticket we extend probedecode compiler logic to:

      • Validate that MJ is a single Key join, where the bigTable keyCol is only a ExprNodeColumnDesc
      • For the selected MJ use backtracking logic to find the orignal TS key it maps to (on the same vertex) - this actually revealed several missed or wrong optimizations (updated q outs)
      • Finally, extend the optimization logic to check if there a type Cast between src and destination (as the types have to match in the probe case) and only use it on the LLAP mode for now

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pgaref Panagiotis Garefalakis
                Reporter:
                pgaref Panagiotis Garefalakis
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 20m
                  3h 20m