[IMPALA-13040] SIGSEGV in QueryState::UpdateFilterFromRemote

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: Impala 4.5.0, Impala 4.4.1
• Component/s: Backend
• Target Version: Impala 4.5.0
• Epic Color: ghx-label-1
• Assignee: Riza Suminto
• Reporter: Csaba Ringhofer
• Created: 26/Apr/24 09:32
• Updated: 14/Aug/24 00:57
• Resolved: 14/May/24 23:26

Description

Crash reason:  SIGSEGV /SEGV_MAPERR
Crash address: 0x48
Process uptime: not available

Thread 114 (crashed)
 0  libpthread.so.0 + 0x9d00
    rax = 0x000000019e57ad00   rdx = 0x000000002a656720
    rcx = 0x00000000059a9860   rbx = 0x0000000000000000
    rsi = 0x000000019e57ad00   rdi = 0x0000000000000038
    rbp = 0x00007f6233d544e0   rsp = 0x00007f6233d544a8
     r8 = 0x0000000006a53540    r9 = 0x0000000000000039
    r10 = 0x0000000000000000   r11 = 0x000000000000000a
    r12 = 0x000000019e57ad00   r13 = 0x00007f62a2f997d0
    r14 = 0x00007f6233d544f8   r15 = 0x000000001632c0f0
    rip = 0x00007f62a2f96d00
    Found by: given as instruction pointer in context
 1  impalad!impala::QueryState::UpdateFilterFromRemote(impala::UpdateFilterParamsPB const&, kudu::rpc::RpcContext*) [query-state.cc : 1033 + 0x5]
    rbp = 0x00007f6233d54520   rsp = 0x00007f6233d544f0
    rip = 0x00000000015c0837
    Found by: previous frame's frame pointer
 2  impalad!impala::DataStreamService::UpdateFilterFromRemote(impala::UpdateFilterParamsPB const*, impala::UpdateFilterResultPB*, kudu::rpc::RpcContext*) [data-stream-service.cc : 134 + 0xb]
    rbp = 0x00007f6233d54640   rsp = 0x00007f6233d54530
    rip = 0x00000000017c05de
    Found by: previous frame's frame pointer

The line that crashes is https://github.com/apache/impala/blob/b39cd79ae84c415e0aebec2c2b4d7690d2a0cc7a/be/src/runtime/query-state.cc#L1033
My guess is that inside the actual segfault is within WaitForPrepare() but it was inlined. Not sure if a remote filter can arrive even before QueryState::Init is finished - that would explain the issue, as instances_prepared_barrier_ is not yet created at that point.

Issue Links

• is caused by: IMPALA-3825 Distribute runtime filter aggregation across cluster
• is duplicated by: IMPALA-13127 custom_cluster/test_runtime_filter_aggregation.py is failing on ASAN jobs