Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
1.19.0
-
None
-
None
Description
When using streaming HTTP connection for queries that return varchar arrays within multiple batches, Drillbit may crash with SIGSEGV fatal error:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00000001166a7593, pid=66979, tid=0x0000000000007603
#
# JRE version: Java(TM) SE Runtime Environment (8.0_241-b07) (build 1.8.0_241-b07)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.241-b07 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# J 9781 C2 io.netty.buffer.DrillBuf.getInt(I)I (15 bytes) @ 0x00000001166a7593 [0x00000001166a7560+0x33]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x00007f95ddbbb800): JavaThread "1eee85ef-2a8d-7c8e-65e4-aaba7bc5c4c6:frag:0:0" daemon [_thread_in_Java, id=30211, stack(0x000070000e21e000,0x000070000e31e000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007f95a20bbcc4
Registers:
RAX=0x000000015e91e3c0, RBX=0x000000015e91e3c0, RCX=0x000000077a95df10, RDX=0x00000000c0e0ccc4
RSP=0x000070000e31c4a0, RBP=0x000070000e31c518, RSI=0x000000077a95df10, RDI=0x0000000000000128
R8 =0x0000000000000000, R9 =0x0000000000000000, R10=0x00007f95e12af000, R11=0xffffffffc0e0ccc4
R12=0x0000000000000000, R13=0x000070000e31c4c8, R14=0x000070000e31c568, R15=0x00007f95ddbbb800
RIP=0x00000001166a7593, EFLAGS=0x0000000000010206, ERR=0x0000000000000004
TRAPNO=0x000000000000000e
Top of Stack: (sp=0x000070000e31c4a0)
0x000070000e31c4a0: 0000000000000000 0000000162da1160
0x000070000e31c4b0: 000070000e31c518 0000000114807da0
0x000070000e31c4c0: 0000000114807da0 00000000c0e0ccc4
0x000070000e31c4d0: 000000077a95df10 000070000e31c4d8
0x000070000e31c4e0: 0000000162d9b028 000070000e31c568
0x000070000e31c4f0: 0000000162d9b3d8 0000000000000000
0x000070000e31c500: 0000000162d9b0b0 000070000e31c4c8
0x000070000e31c510: 000070000e31c568 000070000e31c5b0
0x000070000e31c520: 0000000114807e70 0000000000000000
0x000070000e31c530: 0000000000000000 0000000000000000
0x000070000e31c540: 0000000000000000 0000000000000000
0x000070000e31c550: 0000000000000000 00000000c0e0ccc4
0x000070000e31c560: 000000077a95df10 00000007704ebb20
0x000070000e31c570: 000070000e31c570 0000000162d95e36
0x000070000e31c580: 000070000e31c5d8 0000000162d98e48
0x000070000e31c590: 0000000000000000 0000000162d95e78
0x000070000e31c5a0: 000070000e31c568 000070000e31c5d8
0x000070000e31c5b0: 000070000e31c630 00000001148080e0
0x000070000e31c5c0: 0000000000000000 0000000000000000
0x000070000e31c5d0: 000000077a95dfe0 00000007704db008
0x000070000e31c5e0: 000000077a98b638 000000077a98b638
0x000070000e31c5f0: 000070000e31c5f0 0000000162d95f05
0x000070000e31c600: 000070000e31c640 0000000162d98e48
0x000070000e31c610: 0000000000000000 0000000162d95f28
0x000070000e31c620: 000070000e31c5d8 000070000e31c640
0x000070000e31c630: 000070000e31c690 00000001148081b4
0x000070000e31c640: 00000007704db008 00000006c1fc6020
0x000070000e31c650: 000070000e31c650 000000016104cd0c
0x000070000e31c660: 000070000e31c6a8 000000016104f9f0
0x000070000e31c670: 0000000000000000 000000016104cd90
0x000070000e31c680: 000070000e31c640 000070000e31c6a0
0x000070000e31c690: 000070000e31c6f0 000000011480835d
Instructions: (pc=0x00000001166a7593)
0x00000001166a7573: 90 0f 1f 84 00 00 00 00 00 66 66 66 90 48 81 ec
0x00000001166a7583: 18 00 00 00 48 89 6c 24 10 4c 8b 56 28 4c 63 da
0x00000001166a7593: 43 8b 04 1a 48 83 c4 10 5d 85 05 5e 7a be f0 c3
0x00000001166a75a3: f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4 f4
Register to memory mapping:
RAX={method} {0x000000015e91e3c0} 'getInt' '(I)I' in 'io/netty/buffer/DrillBuf'
RBX={method} {0x000000015e91e3c0} 'getInt' '(I)I' in 'io/netty/buffer/DrillBuf'
RCX=0x000000077a95df10 is an oop
io.netty.buffer.DrillBuf
- klass: 'io/netty/buffer/DrillBuf'
RDX=0x00000000c0e0ccc4 is an unknown value
RSP=0x000070000e31c4a0 is pointing into the stack for thread: 0x00007f95ddbbb800
RBP=0x000070000e31c518 is pointing into the stack for thread: 0x00007f95ddbbb800
RSI=0x000000077a95df10 is an oop
io.netty.buffer.DrillBuf
- klass: 'io/netty/buffer/DrillBuf'
RDI=0x0000000000000128 is an unknown value
R8 =0x0000000000000000 is an unknown value
R9 =0x0000000000000000 is an unknown value
R10=0x00007f95e12af000 is an unknown value
R11=0xffffffffc0e0ccc4 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x000070000e31c4c8 is pointing into the stack for thread: 0x00007f95ddbbb800
R14=0x000070000e31c568 is pointing into the stack for thread: 0x00007f95ddbbb800
R15=0x00007f95ddbbb800 is a thread
Stack: [0x000070000e21e000,0x000070000e31e000], sp=0x000070000e31c4a0, free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 9781 C2 io.netty.buffer.DrillBuf.getInt(I)I (15 bytes) @ 0x00000001166a7593 [0x00000001166a7560+0x33]
j org.apache.drill.exec.vector.accessor.reader.OffsetVectorReader.getEntry()J+24
j org.apache.drill.exec.vector.accessor.ColumnAccessors$VarCharColumnReader.getBytes()[B+14
j org.apache.drill.exec.vector.accessor.ColumnAccessors$VarCharColumnReader.getString()Ljava/lang/String;+5
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeScalar(Lorg/apache/drill/exec/vector/accessor/ScalarReader;)V+260
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeValue(Lorg/apache/drill/exec/vector/accessor/ObjectReader;)V+84
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeArray(Lorg/apache/drill/exec/vector/accessor/ArrayReader;)V+33
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeValue(Lorg/apache/drill/exec/vector/accessor/ObjectReader;)V+71
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeColumn(Lorg/apache/drill/exec/vector/accessor/ObjectReader;)V+22
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeObject(Lorg/apache/drill/exec/vector/accessor/TupleReader;)V+34
j org.apache.drill.exec.physical.resultSet.util.JsonWriter.writeRow(Lorg/apache/drill/exec/vector/accessor/TupleReader;)V+2
j org.apache.drill.exec.server.rest.stream.StreamingHttpConnection.emitRows(Lorg/apache/drill/exec/physical/rowSet/RowSetReader;)V+14
j org.apache.drill.exec.server.rest.stream.StreamingHttpConnection.emitBatch(Lorg/apache/drill/exec/physical/rowSet/RowSetReader;)V+37
j org.apache.drill.exec.server.rest.stream.StreamingHttpConnection.sendData(Lorg/apache/drill/exec/rpc/RpcOutcomeListener;Lorg/apache/drill/exec/physical/impl/materialize/QueryDataPackage;)V+69
j org.apache.drill.exec.ops.AccountingUserConnection.sendData(Lorg/apache/drill/exec/physical/impl/materialize/QueryDataPackage;)V+16
j org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext()Z+194
j org.apache.drill.exec.physical.impl.BaseRootExec.next()Z+43
j org.apache.drill.exec.work.fragment.FragmentExecutor$1.run()Ljava/lang/Void;+77
j org.apache.drill.exec.work.fragment.FragmentExecutor$1.run()Ljava/lang/Object;+1
v ~StubRoutines::call_stub
V [libjvm.dylib+0x2f1b06]
V [libjvm.dylib+0x3523f6]
J 183 java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object; (0 bytes) @ 0x0000000114b2db95 [0x0000000114b2dac0+0xd5]
j javax.security.auth.Subject.doAs(Ljavax/security/auth/Subject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+42
j org.apache.hadoop.security.UserGroupInformation.doAs(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+14
j org.apache.drill.exec.work.fragment.FragmentExecutor.run()V+302
j org.apache.drill.common.SelfCleaningRunnable.run()V+4
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [libjvm.dylib+0x2f1b06]
V [libjvm.dylib+0x2f22aa]
V [libjvm.dylib+0x2f2456]
V [libjvm.dylib+0x34cd45]
V [libjvm.dylib+0x576b93]
V [libjvm.dylib+0x57828e]
V [libjvm.dylib+0x490b86]
C [libsystem_pthread.dylib+0x68fc] _pthread_start+0xe0
C [libsystem_pthread.dylib+0x2443] thread_start+0xf
C 0x0000000000000000