Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Manifests as

      Caused by: java.lang.NullPointerException
      	at org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115)
      	at org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114)
      	at org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:340)
      	at org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:114)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
      	at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
      	... 29 more
      

      It's actually a somewhat subtle ordering problem in sortmerge - as it stands, it calls different branches of the tree in closeOp after they themselves have already been closed. Other operators that clean stuff up in close may result in different errors. The common pattern is

         1125         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
         1126         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
         1127         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:404)
      ...
         1131         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:428)
         1132         at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:388)
         1133         at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
      ...
         1139         at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:294)
      

        Attachments

        1. HIVE-15278.patch
          2 kB
          Sergey Shelukhin

          Activity

            People

            • Assignee:
              sershe Sergey Shelukhin
              Reporter:
              sershe Sergey Shelukhin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: