[IMPALA-135] Impalad crashes with gdb attached.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 0.6
• Fix Version/s: Impala 1.0.1
• Environment: Xubuntu 12.04
• Assignee: Lenni Kuff
• Reporter: Alexander Behm
• Created: 14/Mar/13 16:48
• Updated: 16/May/13 21:38
• Resolved: 16/May/13 21:38

Description

I can pretty reliably crash a single impalad when gdb is attached to it with the following query (Nong has verified the crash on his machine):
select * from alltypes limit 10

Steps to reproduce:
1. Start a single impalad
./be/build/debug/service/impalad -use_statestore=false
2. Attach gdb
3. Set a breakpoint, e.g., in coordinator.cc:279
4. Start bin/impala-shell.sh
5. connect and use "functional"
6. Run "select * from alltypes limit 10"

Note that other queries may work, e.g., select * from alltypessmall.

The impalad log reports the following message, but typically the messages vary:

1. A fatal error has been detected by the Java Runtime Environment:
   #
2. SIGSEGV (0xb) at pc=0x00007fa7a4d5209c, pid=8193, tid=140357734848256
   #
3. JRE version: 6.0_26-b03
4. Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode linux-amd64 compressed oops)
5. Problematic frame:
6. J java.lang.String.substring(II)Ljava/lang/String;
   #
7. An error report file with more information is saved as:
8. /home/abehm/impala/hs_err_pid8193.log
   #
9. If you would like to submit a bug report, please visit:
10. http://java.sun.com/webapps/bugreport/crash.jsp
    #

When I backtrace in gdb I get this (also typically different for each attempt but always something with JVM_handle_linux_signal ()):
#0 0x00007fa7aae3a425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fa7aae3db8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007fa7aa1d2637 in os::abort(bool) () from /home/abehm/jdk1.6.0_26/jre/lib/amd64/server/libjvm.so
#3 0x00007fa7aa325cf8 in VMError::report_and_die() () from /home/abehm/jdk1.6.0_26/jre/lib/amd64/server/libjvm.so
#4 0x00007fa7aa1d8fe5 in JVM_handle_linux_signal () from /home/abehm/jdk1.6.0_26/jre/lib/amd64/server/libjvm.so
#5 0x00007fa7aa1d530e in signalHandler(int, siginfo*, void*) () from /home/abehm/jdk1.6.0_26/jre/lib/amd64/server/libjvm.so
#6 <signal handler called>
#7 0x00007fa7a4d5209c in ?? ()
#8 0x00007fa7a4cdd929 in ?? ()
#9 0x00007fa7a4cdd929 in ?? ()
#10 0x0000000000000004 in ?? ()
#11 0x0000000000000000 in ?? ()