[HIVE-23406] SharedWorkOptimizer should check nullSortOrders when comparing ReduceSink operators

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Physical Optimizer
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 07/May/20 19:04
• Updated: 04/Sep/20 08:08
• Resolved: 04/Sep/20 08:08

Description

SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when compares ReduceSink operators:
https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444

      ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
      ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();

      if (StringUtils.equals(op1Conf.getKeyColString(), op2Conf.getKeyColString()) &&
        StringUtils.equals(op1Conf.getValueColsString(), op2Conf.getValueColsString()) &&
        StringUtils.equals(op1Conf.getParitionColsString(), op2Conf.getParitionColsString()) &&
        op1Conf.getTag() == op2Conf.getTag() &&
        StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
        op1Conf.getTopN() == op2Conf.getTopN() &&
        canDeduplicateReduceTraits(op1Conf, op2Conf)) {
        return true;
      } else {
        return false;
      }

An expression like

        StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&

should be added.

 

Issue Links

• links to: GitHub Pull Request #1464