[HIVE-13601] Deadline is not registered at some places in HiveMetastore before the ObjectStore/MetaStoreDirectSql APIs are invoked

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Hari Sankar Sivarama Subramaniyan
• Created: 24/Apr/16 19:59
• Updated: 01/Oct/19 22:07

Description

HIVE-13542 exposed this issue as shown below when making Metastore API calls via CBO code path.

16-04-21T11:45:55,201 DEBUG [f2995fd1-e4fc-4783-86cb-9264f715e540 main[]]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(864)) - Direct SQL query in 14.825303ms + 0.086336ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALU\
E", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED"  from "TAB_COL_STATS"  where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)]
2016-04-21T11:45:55,242 WARN  [f2995fd1-e4fc-4783-86cb-9264f715e540 main[]]: metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(2602)) - Direct SQL failed, falling back to ORM
org.apache.hadoop.hive.metastore.api.MetaException: The threadlocal Deadline is null, please register it first.
        at org.apache.hadoop.hive.metastore.Deadline.newMetaException(Deadline.java:178) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.Deadline.checkTimeout(Deadline.java:149) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.makeColumnStats(MetaStoreDirectSql.java:1717) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getTableStats(MetaStoreDirectSql.java:1201) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$7.getSqlResult(ObjectStore.java:6849) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$7.getSqlResult(ObjectStore.java:6846) ~[hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2567) [hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatisticsInternal(ObjectStore.java:6845) [hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatistics(ObjectStore.java:6839) [hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_statistics_req(HiveMetaStore.java:4361) [hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableColumnStatistics(HiveMetaStoreClient.java:1613) [hive-metastore-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.getTableColumnStatistics(SessionHiveMetaStoreClient.java:347) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:3317) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.stats.StatsUtils.getTableColumnStats(StatsUtils.java:758) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.updateColStats(RelOptHiveTable.java:302) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:417) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:410) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan.getColStat(HiveTableScan.java:158) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:77) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:64) [hive-exec-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
        at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]

Issue Links

• is related to: HIVE-13542 Missing stats for tables in TPCDS performance regression suite