Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23604

LLAP does not have correct version of guava after HIVE-22126

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: llap
    • Labels:
      None

      Description

      This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in hive-exec. There is a issue that LLAP is also using this guava, so after shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar` which will cause following error

      Status: Running (Executing on YARN cluster with App id application_1591081923777_0005)
       
      Map 1: -/-	Reducer 2: 0/1	
      Map 1: 0/11	Reducer 2: 0/1	
      Map 1: 0(+11,-11)/11	Reducer 2: 0/1	
      Map 1: 0(+0,-32)/11	Reducer 2: 0/1	
      Status: Failed
      Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00, diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002, diagnostics=[TaskAttempt 0 failed, info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError): com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
      	at org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
      	at org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
      	at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
      	at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
      	at org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
      	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
      	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
      

      This can be solved by adding `com.google.common.base.Stopwatch.class` to this section in hive 3.1.2 releaseĀ https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409 and probably to this file for master branch https://github.com/apache/hive/blob/2385361ef33ff81760115cfee4de31ec18afe2f3/llap-server/src/java/org/apache/hadoop/hive/llap/cli/service/AsyncTaskCopyLocalJars.java

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              luyuanhao Yuanhao Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: