Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12666

PCRExprProcFactory.GenericFuncExprProcessor.process() aggressively removes dynamic partition pruner generated synthetic join predicates.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • None

    Description

      Introduced by HIVE-11634. The original idea in HIVE-11634 was to remove the IN partition conditions from the predicate list since the static dynamic partitioning would kick in and push these predicates down to metastore. However, the check is too aggressive and removes events such as below :

      -                    Select Operator
      -                      expressions: UDFToDouble(UDFToInteger((hr / 2))) (type: double)
      -                      outputColumnNames: _col0
      -                      Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
      -                      Group By Operator
      -                        keys: _col0 (type: double)
      -                        mode: hash
      -                        outputColumnNames: _col0
      -                        Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
      -                        Dynamic Partitioning Event Operator
      -                          Target Input: srcpart
      -                          Partition key expr: UDFToDouble(hr)
      -                          Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
      -                          Target column: hr
      -                          Target Vertex: Map 1
      

      Attachments

        1. HIVE-12666.1.patch
          15 kB
          Hari Sankar Sivarama Subramaniyan
        2. HIVE-12666.2.patch
          23 kB
          Hari Sankar Sivarama Subramaniyan

        Issue Links

          Activity

            People

              hsubramaniyan Hari Sankar Sivarama Subramaniyan
              hsubramaniyan Hari Sankar Sivarama Subramaniyan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: