Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17326

Insert into HBase tables fails if hive.llap.execution.mode is set to only

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.0
    • None
    • llap
    • None

    • HDP 2.6.x

    Description

      Inserting into a table created using HBase storage handler errors out if hive.llap.execution.mode=only. Works if the hive.llap.execution.mode value is none or auto or with MR execution engine.
      Simple repro script
      CREATE TABLE hbase_table_sai(id int, name string)
      STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
      WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name")
      TBLPROPERTIES ("hbase.table.name" = "sai");

      create table hive_tab1(c1 int, c2 string);
      insert into hive_tab1 values(1,'abc');
      0: jdbc:hive2://localhost:10500/default> insert overwrite table hbase_table_sai select * from hive_tab1;
      INFO  : Compiling command(queryId=hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a): insert overwrite table hbase_table_sai select * from hive_tab1
      INFO  : We are setting the hadoop caller context from HIVE_SSN_ID:7114abad-2ba2-410d-ad73-40d473a647af to hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a
      INFO  : Semantic Analysis Completed
      INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:hive_tab1.c1, type:int, comment:null), FieldSchema(name:hive_tab1.c2, type:string, comment:null)], properties:null)
      INFO  : Completed compiling command(queryId=hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a); Time taken: 0.36 seconds
      INFO  : We are resetting the hadoop caller context to HIVE_SSN_ID:7114abad-2ba2-410d-ad73-40d473a647af
      INFO  : Concurrency mode is disabled, not creating a lock manager
      INFO  : Setting caller context to query id hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a
      INFO  : Executing command(queryId=hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a): insert overwrite table hbase_table_sai select * from hive_tab1
      INFO  : Query ID = hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a
      INFO  : Total jobs = 1
      INFO  : Starting task [Stage-0:DDL] in serial mode
      INFO  : Starting task [Stage-1:DDL] in serial mode
      INFO  : Launching Job 1 out of 1
      INFO  : Starting task [Stage-3:MAPRED] in serial mode
      INFO  : Session is already open
      INFO  : Tez session missing resources, adding additional necessary resources
      INFO  : Dag name: insert overwrite table hbase_tab...hive_tab1(Stage-3)
      INFO  : Dag submit failed due to There is conflicting local resource (guava-14.0.1.jar) between dag local resource and vertex Map 1 local resource.
      Resource of dag : resource

      { scheme: "hdfs" host: "ulcer1" port: 8020 file: "/tmp/hive/hive/7114abad-2ba2-410d-ad73-40d473a647af/hive_2017-08-08_12-54-31_225_8109820757632121978-7/hive/_tez_scratch_dir/guava-14.0.1.jar" }

      size: 2189117 timestamp: 1502222072247 type: FILE visibility: PRIVATE
      Resource of vertex: resource

      { scheme: "hdfs" host: "ulcer1" port: 8020 file: "/tmp/hive/hive/_tez_session_dir/8a93f7fd-b925-4684-a6b1-6561b5c8e344/guava-14.0.1.jar" }

      size: 2189117 timestamp: 1502211657919 type: FILE visibility: PRIVATE stack trace: [org.apache.tez.dag.api.DAG.verify(DAG.java:695), org.apache.tez.dag.api.DAG.createDag(DAG.java:796), org.apache.tez.client.TezClientUtils.prepareAndCreateDAGPlan(TezClientUtils.java:718), org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:555), org.apache.tez.client.TezClient.submitDAG(TezClient.java:522), org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:506), org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:188), org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197), org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100), org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1905), org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1607), org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1354), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1123), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116), org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242), org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91), org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334), java.security.AccessController.doPrivileged(Native Method), javax.security.auth.Subject.doAs(Subject.java:422), org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866), org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:348), java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511), java.util.concurrent.FutureTask.run(FutureTask.java:266), java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511), java.util.concurrent.FutureTask.run(FutureTask.java:266), java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149), java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624), java.lang.Thread.run(Thread.java:748)] retrying...
      ERROR : Failed to execute tez graph.
      java.lang.IllegalStateException: There is conflicting local resource (guava-14.0.1.jar) between dag local resource and vertex Map 1 local resource.
      Resource of dag : resource

      { scheme: "hdfs" host: "ulcer1" port: 8020 file: "/tmp/hive/hive/7114abad-2ba2-410d-ad73-40d473a647af/hive_2017-08-08_12-54-31_225_8109820757632121978-7/hive/_tez_scratch_dir/guava-14.0.1.jar" }

      size: 2189117 timestamp: 1502222072247 type: FILE visibility: PRIVATE
      Resource of vertex: resource

      { scheme: "hdfs" host: "ulcer1" port: 8020 file: "/tmp/hive/hive/_tez_session_dir/8a93f7fd-b925-4684-a6b1-6561b5c8e344/guava-14.0.1.jar" }

      size: 2189117 timestamp: 1502211657919 type: FILE visibility: PRIVATE
              at org.apache.tez.dag.api.DAG.verify(DAG.java:695) ~[tez-api-0.8.4.2.6.1.0-129.jar:0.8.4.2.6.1.0-129]
              at org.apache.tez.dag.api.DAG.createDag(DAG.java:796) ~[tez-api-0.8.4.2.6.1.0-129.jar:0.8.4.2.6.1.0-129]
              at org.apache.tez.client.TezClientUtils.prepareAndCreateDAGPlan(TezClientUtils.java:718) ~[tez-api-0.8.4.2.6.1.0-129.jar:0.8.4.2.6.1.0-129]
              at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:555) ~[tez-api-0.8.4.2.6.1.0-129.jar:0.8.4.2.6.1.0-129]
              at org.apache.tez.client.TezClient.submitDAG(TezClient.java:522) ~[tez-api-0.8.4.2.6.1.0-129.jar:0.8.4.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:526) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:188) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1905) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1607) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1354) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1123) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116) [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_141]
              at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_141]
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) [hadoop-common-2.7.3.2.6.1.0-129.jar:?]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:348) [hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
              at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
      ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
      INFO  : Resetting the caller context to HIVE_SSN_ID:7114abad-2ba2-410d-ad73-40d473a647af
      INFO  : Completed executing command(queryId=hive_20170808125431_652dbcde-96d5-4afd-9359-bd71bfd6b01a); Time taken: 11.141 seconds
      Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1)

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. AMBARI-21730 Default of hive.llap.execution.mode=only on Hive Interactive breaks StorageHandler tables.

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              sailajanch Sailaja Navvluru
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: