[HIVE-12815] column stats NPE for a query w/o a table

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 1.3.0, 2.0.0
• Assignee: Prasanth Jayachandran
• Reporter: Sergey Shelukhin
• Created: 08/Jan/16 21:49
• Updated: 16/Feb/16 23:50
• Resolved: 13/Jan/16 21:09

Description

I was running something like create table as select 1;

First it logs why it cannot get stats:

2016-01-08T21:46:31,876 ERROR [0883a32c-c789-4695-aec2-ed73bb1cc9ce 0883a32c-c789-4695-aec2-ed73bb1cc9ce main]: stats.StatsUtils (StatsUtils.java:getTableColumnStats(756)) - Failed to retrieve table statistics:
org.apache.hadoop.hive.ql.metadata.HiveException: NoSuchObjectException(message:Specified database/table does not exist : _dummy_database._dummy_table)
        at org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:3195) ~[hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:752) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:198) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:144) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:132) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]

and returns null, then it fails with NPE:

2016-01-08T21:46:31,885 ERROR [0883a32c-c789-4695-aec2-ed73bb1cc9ce 0883a32c-c789-4695-aec2-ed73bb1cc9ce main]: ql.Driver (SessionState.java:printError(1010)) - FAILED: NullPointerException null
java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.stats.StatsUtils.getDataSizeFromColumnStats(StatsUtils.java:1450)
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:199)
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:144)
        at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:132)
        at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:114)
        at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
        at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
        at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
        at org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)

Only "NullPointerException null" is logged to CLI...