Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8740

Sorted dynamic partition does not work correctly with constant folding

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      Sorted dynamic partition optimization looks for partition columns from the operator above FileSinkOperator. As per hive convention it expects partition columns at the last. But with HIVE-8585 equality filters on partition columns gets folded to constant. The column pruner then prunes the constant expression as they don't reference any columns. This in some cases will yield unexpected results (throw ArrayIndexOutOfBounds exception) with sorted dynamic partition insert optimization. In such cases we don't really need sorted dynamic partition optimization.

        Attachments

        1. HIVE-8740.1.patch
          54 kB
          Prasanth Jayachandran
        2. HIVE-8740.2.patch
          54 kB
          Prasanth Jayachandran
        3. HIVE-8740.3.patch
          70 kB
          Prasanth Jayachandran
        4. HIVE-8740.4.patch
          69 kB
          Prasanth Jayachandran

          Issue Links

            Activity

              People

              • Assignee:
                prasanth_j Prasanth Jayachandran
                Reporter:
                prasanth_j Prasanth Jayachandran
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: