[IMPALA-296] Query Fragments sometimes go rogue, and the report thread is stuck in an infinite loop.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: Impala 1.0
• Fix Version/s: Impala 1.0
• Assignee: Alan Choi
• Reporter: Ishaan Joshi
• Created: 20/Apr/13 00:30
• Updated: 25/Apr/13 00:13
• Resolved: 25/Apr/13 00:13

Description

Report thread:
This has been repeating for at least 4 hours.
top shows that impalad is spinning.

E0419 17:07:49.216120   527 impala-server.cc:1628] unknown query id: 9f32d18598974ad1:832a8602560787c5
E0419 17:07:54.265277   955 impala-server.cc:1628] unknown query id: 8533a91702104f67:b120a3fb6d6b6894
E0419 17:08:27.103909   423 impala-server.cc:1628] unknown query id: 7194ff755ac64b37:b46509b5afe87edf
E0419 17:08:28.345149   949 impala-server.cc:1628] unknown query id: 17e34fa05f984817:a263b0fff2ce74c9
E0419 17:08:28.993016 13162 impala-server.cc:1628] unknown query id: 13bc38909e92492e:bcfcef763dc2b09e
E0419 17:08:29.573935   763 impala-server.cc:1628] unknown query id: 97cd4b66976f4e45:a4eadd25dfd0f553
E0419 17:08:30.282666   811 impala-server.cc:1628] unknown query id: 26228eff81e747bb:9e86bc3e475a6fe0

Breakpoint 1, impala::ImpalaServer::ReportExecStatus

(gdb) bt
#0  impala::ImpalaServer::ReportExecStatus (this=0x5f93c00, return_val=..., params=...) at /data/1/jenkins/workspace/impala-master-64bit/repos/Impala/be/src/service/impala-server.cc:1614
#1  0x0000000000d7782b in impala::ImpalaInternalServiceProcessor::process_ReportExecStatus (this=0x519a780, seqid=0, iprot=0x3c7ff1c0, oprot=0x3c7ff180, callContext=0x3c9de568)
    at /data/1/jenkins/workspace/impala-master-64bit/repos/Impala/be/generated-sources/gen-cpp/ImpalaInternalService.cpp:1003
#2  0x0000000000d76da4 in impala::ImpalaInternalServiceProcessor::dispatchCall (this=0x519a780, iprot=0x3c7ff1c0, oprot=0x3c7ff180, fname="ReportExecStatus", seqid=0, callContext=0x3c9de568)
    at /data/1/jenkins/workspace/impala-master-64bit/repos/Impala/be/generated-sources/gen-cpp/ImpalaInternalService.cpp:922
#3  0x0000000000b2fd12 in apache::thrift::TDispatchProcessor::process (this=0x519a780, in=..., out=..., connectionContext=0x3c9de568)
    at /data/1/jenkins/workspace/impala-master-64bit/repos/Impala/thirdparty/thrift-0.9.0/build/include/thrift/TDispatchProcessor.h:121
#4  0x00000000017f6f09 in apache::thrift::server::TThreadedServer::Task::run (this=0x3c848720) at src/thrift/server/TThreadedServer.cpp:70
#5  0x00000000017f98b2 in apache::thrift::concurrency::PthreadThread::threadMain (arg=0x3c593aa0) at src/thrift/concurrency/PosixThreadFactory.cpp:208
#6  0x0000003e3ec077f1 in start_thread () from /lib64/libpthread.so.0
#7  0x0000003e3e4e5ccd in clone () from /lib64/libc.so.6