[IMPALA-7802] Implement support for closing idle sessions

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.0, Impala 2.12.0
• Fix Version/s: Impala 3.3.0
• Component/s: Clients
• Labels: supportability
• Target Version: Impala 3.3.0
• Epic Color: ghx-label-9
• Assignee: Michael Ho
• Reporter: Michael Ho
• Created: 02/Nov/18 02:56
• Updated: 15/Mar/24 17:51
• Resolved: 24/Jun/19 17:51

Description

Currently, the query option idle_session_timeout specifies a timeout in seconds after which all running queries of that idle session will be cancelled and no new queries can be issued to it. However, the idle session will remain open and it needs to be closed explicitly. Please see the documentation for details.

This behavior may be undesirable as each session still consumes an Impala frontend service thread. The number of frontend service threads is bound by the flag fe_service_threads. So, in a multi-tenant environment, an Impala server can have a lot of idle sessions but they still consume against the quota of fe_service_threads. If the number of sessions established reaches fe_service_threads, all new session creations will block until some of the existing sessions exit. There may be no time bound on when these zombie idle sessions will be closed and it's at the mercy of the client implementation to close them. In some sense, leaving many idle sessions open is a way to launch a denial of service attack on Impala.

To fix this situation, we should have an option to forcefully close a session when it's considered idle so it won't unnecessarily consume the limited number of frontend service threads. cc'ing zoram

Issue Links

• breaks: IMPALA-8744 TestSessionExpiration.test_closing_idle_connection fails on Centos 6 due to Python 2.6 incompatibility
• is duplicated by: IMPALA-10567 Failed close open session in flight on impalad web UI
• is duplicated by: IMPALA-10575 Expired sessions not closed in Impala
• is related to: IMPALA-9170 close idle connections without an associated session
• is related to: IMPALA-8701 Document idle session management after IMPALA-7802
• is required by: IMPALA-12911 Set defaults for idle session / query timeouts