Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15778

DROP INDEX (non-existent) throws NPE when using DbNotificationListener

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.3.0
    • Component/s: Metastore
    • Labels:
      None

      Description

      Trying to execute a DROP INDEX operation on a non-existant index throws NPE.

      0: jdbc:hive2://nightly-unsecure-1.gce.cloude> DROP INDEX IF EXISTS vamsee1 ON sample_07;
      INFO  : Compiling command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4): DROP INDEX IF EXISTS vamsee1 ON sample_07
      INFO  : Semantic Analysis Completed
      INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
      INFO  : Completed compiling command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4); Time taken: 0.238 seconds
      INFO  : Executing command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4): DROP INDEX IF EXISTS vamsee1 ON sample_07
      INFO  : Starting task [Stage-0:DDL] in serial mode
      ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
      INFO  : Completed executing command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4); Time taken: 0.061 seconds
      Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException (state=08S01,code=1)
      

      HMS log:

      2017-01-31 16:27:29,421 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-3]: MetaException(message:java.lang.NullPointerException)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:5823)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rethrowException(HiveMetaStore.java:4892)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name(HiveMetaStore.java:4403)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
      	at com.sun.proxy.$Proxy16.drop_index_by_name(Unknown Source)
      	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_index_by_name.getResult(ThriftHiveMetastore.java:10803)
      	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_index_by_name.getResult(ThriftHiveMetastore.java:10787)
      	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
      	at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
      	at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
      	at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
      	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.NullPointerException
      	at org.apache.hive.hcatalog.messaging.json.JSONDropIndexMessage.<init>(JSONDropIndexMessage.java:46)
      	at org.apache.hive.hcatalog.messaging.json.JSONMessageFactory.buildDropIndexMessage(JSONMessageFactory.java:159)
      	at org.apache.hive.hcatalog.listener.DbNotificationListener.onDropIndex(DbNotificationListener.java:280)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name_core(HiveMetaStore.java:4469)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name(HiveMetaStore.java:4396)
      	... 20 more
      

      Looks like if an exception is raised at HiveMetaStore#4572 (which gets triggered if we try to drop an index which doesn't exist) , the control directly jumps to finally block where we are trying to call drop index event on various metastore event listeners HiveMetaStore#4619.

      If one of the event listeners is DbNotificationListener, then it calls the code under JSONDropIndexMessage.java and this fails with NPE during instantiation at JSONDropIndexMessage.java#46 as the exception raised in HiveMetaStore.java wouldn't set the index variable.

        Attachments

        1. HIVE-15778.v0.patch
          1 kB
          Vamsee K. Yarlagadda

          Activity

            People

            • Assignee:
              vamsee Vamsee K. Yarlagadda
              Reporter:
              vamsee Vamsee K. Yarlagadda
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: