[IMPALA-9638] Don't create unnecessary threads in executor-only impalads

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Epic Color: ghx-label-14
• Assignee: Unassigned
• Reporter: Sahil Takiar
• Created: 09/Apr/20 22:35
• Updated: 09/Apr/20 22:35

Description

Looking through the code for ImpalaServer, it looks like there are several threads / thread-pools that are started on executor-only impalads, but don't actually do anything:

• The "cancellation-worker" threadpool (ImpalaServer::CancelFromThreadPool)
• The "session-maintenance" thread (ImpalaServer::SessionMaintenance)
• The "query-expirer" thread (ImpalaServer::ExpireQueries)
• The "unresponsive-backend-thread" thread (ImpalaServer::UnresponsiveBackendThread)
• The code to start ImpalaInternalService might be dead code, but maybe we should just delete it since the ImpalaInternalService doesn't exist anymore

Confirmed this my creating a cluster with dedicated coordinator and getting a thread dump of an executor, which showed the following:

Thread 16 (Thread 0x7fe96fe57700 (LWP 8721)):
#0  0x00007fea1ad20360 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1  0x0000000001ce4423 in impala::ConditionVariable::Wait (this=0x110b3370, lock=...) at /home/stakiar/Impala/be/src/util/condition-variable.h:49
        mutex = 0x110b3348
#2  0x0000000002469f6a in impala::ImpalaServer::SessionMaintenance (this=0x110b3200) at /home/stakiar/Impala/be/src/service/impala-server.cc:2055
        timeout_lock = {_M_device = 0x110b3348, _M_owns = true}
        now = 140640581412928
        expired_cnt = 0 

Issue Links

• is related to: IMPALA-9609 Minimize Frontend activity in executor only Impalas