Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
3.2.0
-
None
-
HIVE-19424: Fixing NPE In MetaDataFormatters at Branch-3
Description
Overview
According to the Hive Schema definition, a table's INPUT_FORMAT class can be set to NULL. However, there are places in the code where we do not account for this NULL value, in particular the MetaDataFormatters classes TextMetaDataFormatter and JsonMetaDataFormatter. In addition, there is no debug level logging in the MetaDataFormatters classes to tell me which table in particular is causing the problem.
hive-schema-2.2.0.mysql.sql
CREATE TABLE IF NOT EXISTS `SDS` ( `SD_ID` bigint(20) NOT NULL, `CD_ID` bigint(20) DEFAULT NULL, `INPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL, `IS_COMPRESSED` bit(1) NOT NULL, ...
TextMetaDataFormatter.java
// Not checking for a null return from getInputFormatClass
inputFormattCls = par.getInputFormatClass().getName();
outputFormattCls = par.getOutputFormatClass().getName();
Reproduction
-- MySQL Backend update SDS SET INPUT_FORMAT=NULL WHERE SD_ID=XXX;
// Hive SHOW TABLE EXTENDED FROM default LIKE '*'; // HS2 Logs [HiveServer2-Background-Pool: Thread-464]: Error running hive query: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Exception while processing show table status at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89) at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Exception while processing show table status at org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3025) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:405) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2052) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1748) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1501) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1285) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1280) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) ... 11 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.formatting.TextMetaDataFormatter.showTableStatus(TextMetaDataFormatter.java:202) at org.apache.hadoop.hive.ql.exec.DDLTask.showTableStatus(DDLTask.java:3020) ... 20 more
Attachments
Attachments
Issue Links
- is related to
-
IMPALA-6985 Impala View Does Not Populate Table Input/Output Format Class
- Open