Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19130

NPE is thrown when REPL LOAD applied drop partition event.

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      During incremental replication, if we split the events batch as follows, then the REPL LOAD on second batch throws NPE.

      Batch-1: CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)

      Batch-2: DROP_TABLE(t1) ->  CREATE_TABLE(t1) -> ADD_PARTITION(t1.p1) -> DROP_PARTITION (t1.p1)

      2018-04-05 16:20:36,531 ERROR [HiveServer2-Background-Pool: Thread-107044]: metadata.Hive (Hive.java:getTable(1219)) - Table catalog_sales_new not found: new5_tpcds_real_bin_partitioned_orc_1000.catalog_sales_new table not found
      2018-04-05 16:20:36,538 ERROR [HiveServer2-Background-Pool: Thread-107044]: exec.DDLTask (DDLTask.java:failed(540)) - org.apache.hadoop.hive.ql.metadata.HiveException
              at org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4016)
              at org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3983)
              at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:341)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
              at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
              at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
              at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
              at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NullPointerException
              at org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByExpr(Hive.java:2613)
              at org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:4008)
              ... 23 more
      

      Attachments

        1. HIVE-19130.01.patch
          12 kB
          Sankar Hariappan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sankarh Sankar Hariappan Assign to me
            sankarh Sankar Hariappan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment