Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-134

DecimalColumnStatistics methods throws NPE when all column values are null

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.1, 1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      FastDecimal changes introduced a regression that when all column column values in stripe or file is null, the min, max values are also null. getMinimum() and getMaximum() methods in ColumnStatisticsImpl is throwing NPE. Following is the stacktrace

      Caused by: java.lang.NullPointerException
      	at org.apache.orc.impl.ColumnStatisticsImpl$DecimalStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:673)
      	at org.apache.orc.impl.RecordReaderImpl.getMin(RecordReaderImpl.java:322)
      	at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:368)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2088)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2067)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2058)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:149)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1376)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1164)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1345)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1342)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1342)
      	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1164)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                prasanth_j Prasanth Jayachandran
                Reporter:
                prasanth_j Prasanth Jayachandran
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: