Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24073

Execution exception in sort-merge semijoin

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Operators

      Description

      Working on HIVE-24041, we trigger an additional SJ conversion that leads to this exception at execution time:

      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1063)
      	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:685)
      	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
      	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
      	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
      	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
      	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.close(MapRecordProcessor.java:462)
      	... 16 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1037)
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1060)
      	... 22 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
      	at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processKey(CommonMergeJoinOperator.java:564)
      	at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:243)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
      	at org.apache.hadoop.hive.ql.exec.TezDummyStoreOperator.process(TezDummyStoreOperator.java:49)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1003)
      	at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1020)
      	... 23 more
      

      To reproduce, just set hive.auto.convert.sortmerge.join to true in the last query in auto_sortmerge_join_10.q after HIVE-24041 has been merged.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                maheshk114 mahesh kumar behera
                Reporter:
                jcamachorodriguez Jesus Camacho Rodriguez
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h