Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12815

column stats NPE for a query w/o a table

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      I was running something like create table as select 1;

      First it logs why it cannot get stats:

      2016-01-08T21:46:31,876 ERROR [0883a32c-c789-4695-aec2-ed73bb1cc9ce 0883a32c-c789-4695-aec2-ed73bb1cc9ce main]: stats.StatsUtils (StatsUtils.java:getTableColumnStats(756)) - Failed to retrieve table statistics:
      org.apache.hadoop.hive.ql.metadata.HiveException: NoSuchObjectException(message:Specified database/table does not exist : _dummy_database._dummy_table)
              at org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:3195) ~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:752) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:198) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:144) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:132) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
      

      and returns null, then it fails with NPE:

      2016-01-08T21:46:31,885 ERROR [0883a32c-c789-4695-aec2-ed73bb1cc9ce 0883a32c-c789-4695-aec2-ed73bb1cc9ce main]: ql.Driver (SessionState.java:printError(1010)) - FAILED: NullPointerException null
      java.lang.NullPointerException
              at org.apache.hadoop.hive.ql.stats.StatsUtils.getDataSizeFromColumnStats(StatsUtils.java:1450)
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:199)
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:144)
              at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:132)
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:114)
              at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
              at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
              at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
              at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
              at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
      

      Only "NullPointerException null" is logged to CLI...

        Attachments

        1. HIVE-12815.1.patch
          5 kB
          Prasanth Jayachandran

          Activity

            People

            • Assignee:
              prasanth_j Prasanth Jayachandran
              Reporter:
              sershe Sergey Shelukhin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: