Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8777

Beeline timeout to connect to impalad in the minicluster

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: Not Applicable
    • Component/s: None
    • Labels:
      None
    • Epic Color:
      ghx-label-5

      Description

      In the minicluster, I was able to use beeline to connect to HiveServer2:

      beeline -u "jdbc:hive2://localhost:11050"

      However, connecting to Impalad via beeline will hang forever:

      beeline -u "jdbc:hive2://localhost:21050"

      jstack result of the beeline client:

      2019-07-21 14:07:48
      Full thread dump OpenJDK 64-Bit Server VM (25.212-b03 mixed mode):
      
      "Attach Listener" #16 daemon prio=9 os_prio=0 tid=0x00007f870c2c7000 nid=0x7a11 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "NonBlockingInputStreamThread" #15 daemon prio=5 os_prio=0 tid=0x00007f8729d13800 nid=0x3dd7 in Object.wait() [0x00007f870b4bf000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000fbce17d0> (a jline.internal.NonBlockingInputStream)
      	at jline.internal.NonBlockingInputStream.run(NonBlockingInputStream.java:278)
      	- locked <0x00000000fbce17d0> (a jline.internal.NonBlockingInputStream)
      	at java.lang.Thread.run(Thread.java:748)
      
      "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x00007f87280f9800 nid=0x3dc6 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C1 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x00007f87280ee800 nid=0x3dc5 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007f87280ec800 nid=0x3dc4 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f87280ea800 nid=0x3dc3 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f87280e7800 nid=0x3dc2 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f87280e6000 nid=0x3dc1 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f87280b3800 nid=0x3dc0 in Object.wait() [0x00007f87127e6000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000ff589210> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
      	- locked <0x00000000ff589210> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
      
      "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f87280b1000 nid=0x3dbf in Object.wait() [0x00007f87128e7000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000ff5893c8> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:502)
      	at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
      	- locked <0x00000000ff5893c8> (a java.lang.ref.Reference$Lock)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
      
      "main" #1 prio=5 os_prio=0 tid=0x00007f872802d800 nid=0x3db3 runnable [0x00007f8731696000]
         java.lang.Thread.State: RUNNABLE
      	at java.net.SocketInputStream.socketRead0(Native Method)
      	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
      	at java.net.SocketInputStream.read(SocketInputStream.java:171)
      	at java.net.SocketInputStream.read(SocketInputStream.java:141)
      	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
      	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
      	- locked <0x00000000fcf31708> (a java.io.BufferedInputStream)
      	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
      	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
      	at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
      	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:305)
      	at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
      	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:229)
      	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:184)
      	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
      	at java.sql.DriverManager.getConnection(DriverManager.java:664)
      	at java.sql.DriverManager.getConnection(DriverManager.java:208)
      	at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145)
      	at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209)
      	at org.apache.hive.beeline.Commands.connect(Commands.java:1617)
      	at org.apache.hive.beeline.Commands.connect(Commands.java:1512)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56)
      	at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1290)
      	at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1329)
      	at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:864)
      	at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:768)
      	at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1004)
      	at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:526)
      	at org.apache.hive.beeline.BeeLine.main(BeeLine.java:508)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
      
      "VM Thread" os_prio=0 tid=0x00007f87280a7000 nid=0x3dbe runnable 
      
      "GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f8728042800 nid=0x3db4 runnable 
      
      "GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f8728044000 nid=0x3db5 runnable 
      
      "GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007f8728046000 nid=0x3db6 runnable 
      
      "GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007f8728047800 nid=0x3db7 runnable 
      
      "GC task thread#4 (ParallelGC)" os_prio=0 tid=0x00007f8728049800 nid=0x3db8 runnable 
      
      "GC task thread#5 (ParallelGC)" os_prio=0 tid=0x00007f872804b000 nid=0x3db9 runnable 
      
      "GC task thread#6 (ParallelGC)" os_prio=0 tid=0x00007f872804d000 nid=0x3dba runnable 
      
      "GC task thread#7 (ParallelGC)" os_prio=0 tid=0x00007f872804e800 nid=0x3dbb runnable 
      
      "GC task thread#8 (ParallelGC)" os_prio=0 tid=0x00007f8728050800 nid=0x3dbc runnable 
      
      "GC task thread#9 (ParallelGC)" os_prio=0 tid=0x00007f8728052000 nid=0x3dbd runnable 
      
      "VM Periodic Task Thread" os_prio=0 tid=0x00007f87280fc000 nid=0x3dc7 waiting on condition 
      
      JNI global references: 331
      

      Looks like it's waiting for something to read. However, in the logs of Impalad, it seems the connection is established successfully:

      I0721 12:06:56.736927 15220 TAcceptQueueServer.cpp:338] New connection to server hiveserver2-frontend from client <Host: ::ffff:127.0.0.1 Port: 34340>
      I0721 12:06:56.736991 15228 TAcceptQueueServer.cpp:225] TAcceptQueueServer: hiveserver2-frontend started connection setup for client <Host: ::ffff:127.0.0.1 Port: 34340>
      I0721 12:06:56.737013 15228 TAcceptQueueServer.cpp:243] TAcceptQueueServer: hiveserver2-frontend finished connection setup for client <Host: ::ffff:127.0.0.1 Port: 34340>
      

      In this impalad's /sessions page, no sessions are shown.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              stigahuang Quanlong Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: