Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23406

SharedWorkOptimizer should check nullSortOrders when comparing ReduceSink operators

    XMLWordPrintableJSON

Details

    Description

      SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when compares ReduceSink operators:
      https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444

            ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
            ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();
      
            if (StringUtils.equals(op1Conf.getKeyColString(), op2Conf.getKeyColString()) &&
              StringUtils.equals(op1Conf.getValueColsString(), op2Conf.getValueColsString()) &&
              StringUtils.equals(op1Conf.getParitionColsString(), op2Conf.getParitionColsString()) &&
              op1Conf.getTag() == op2Conf.getTag() &&
              StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
              op1Conf.getTopN() == op2Conf.getTopN() &&
              canDeduplicateReduceTraits(op1Conf, op2Conf)) {
              return true;
            } else {
              return false;
            }
      

      An expression like

              StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
      

      should be added.

       

      Attachments

        Issue Links

          Activity

            People

              kkasa Krisztian Kasa
              kkasa Krisztian Kasa
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m