[QPID-8512] QPID threads hang on shutdown

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++ Client

Environment

QPID 0.8 on SuSE Linux Enterprise Server 15 (SLES), Service Pack 2 (SP2) (SLES 15 SP2)

• Flags: Important
• Assignee: Unassigned
• Reporter: Karthik
• Created: 18/Mar/21 10:19
• Updated: 07/Nov/21 20:22

Description

QPID Threads in the client hang on shutdown. This issue seems to specifically occur in SLES 15 SP2 (SuSE Linux Service Pack 2). It was working fine on SLES 15 SP1 and older versions like SLES11, 12 etc. 

QPID Client/Broker Version: 0.8

On debugging into the QPID code, observed that the hang occurs in this function and in the ::epoll_wait call.

Code-Reference:
qpid/sys/epoll/EpollPoller.cpp

Poller::Event Poller::wait(Duration timeout) {

           ...

      int rc = ::epoll_wait(impl->epollFd, &epe, 1, timeoutMs);

      ...

}

 

Thread backtrace on shutdown which hang:
Thread 8 (Thread 0x7fffe27fc700 (LWP 29944)):
#0  0x00007ffff2dff126 in epoll_pwait () from /lib64/libc.so.6
#1  0x00007ffff5abc461 in qpid::sys::Poller::wait (this=0x6eb230, timeout=...) at qpid/sys/epoll/EpollPoller.cpp:568
#2  0x00007ffff5abcf31 in qpid::sys::Poller::run (this=0x6eb230) at qpid/sys/epoll/EpollPoller.cpp:518
#3  0x00007ffff5ab3caa in qpid::sys::(anonymous namespace)::runRunnable (p=0x6) at qpid/sys/posix/Thread.cpp:35
#4  0x00007ffff77b44f9 in start_thread () from /lib64/libpthread.so.0
#5  0x00007ffff2dfefbf in clone () from /lib64/libc.so.6
 
Thread 3 (Thread 0x7ffff1681700 (LWP 29939)):
#0  0x00007ffff2dff126 in epoll_pwait () from /lib64/libc.so.6
#1  0x00007ffff5abc461 in qpid::sys::Poller::wait (this=0x6eb230, timeout=...) at qpid/sys/epoll/EpollPoller.cpp:568
#2  0x00007ffff5abcf31 in qpid::sys::Poller::run (this=0x6eb230) at qpid/sys/epoll/EpollPoller.cpp:518
#3  0x00007ffff5ab3caa in qpid::sys::(anonymous namespace)::runRunnable (p=0x6) at qpid/sys/posix/Thread.cpp:35
#4  0x00007ffff77b44f9 in start_thread () from /lib64/libpthread.so.0
#5  0x00007ffff2dfefbf in clone () from /lib64/libc.so.6
 
Thread 1 (Thread 0x7ffff7fb97c0 (LWP 29842)):
#0  0x00007ffff77b58bd in pthread_join () from /lib64/libpthread.so.0
#1  0x00007ffff5ab3db3 in qpid::sys::Thread::join (this=<optimized out>) at qpid/sys/posix/Thread.cpp:70
#2  0x00007ffff5716b09 in qpid::client::(anonymous namespace)::IOThread::~IOThread (this=0x7ffff597b6a0 <qpid::client::(anonymous namespace)::theIO()::io>, __in_chrg=<optimized out>) at qpid/client/ConnectionImpl.cpp:140
#3  0x00007ffff2d3f138 in __run_exit_handlers () from /lib64/libc.so.6
#4  0x00007ffff2d3f18a in exit () from /lib64/libc.so.6
#5  0x00007ffff2d27351 in __libc_start_main () from /lib64/libc.so.6
#6  0x000000000040f179 in _start () at ../sysdeps/x86_64/elf/start.S:113