[IMPALA-9919] Bad Impala Performance after a period of time

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.10.0
• Component/s: Backend
• Labels: performance
• Environment: OS: CentOS 6.9
• Flags: Important
• Epic Color: ghx-label-8
• Assignee: Unassigned
• Reporter: Vagelis Nomikos
• Created: 06/Jul/20 14:01
• Updated: 16/Jul/20 11:41

Description

Our cluster is consisting of about 60 Impala nodes. After a period of time and after executing some "heavy" queries the performance of the cluster becomes bad and the Impala is not responding after a period of time. We observed that day after day the Impala Resident memory and the running threads of the machine keep growing even if we do not run queries. Everytime we perform an Impala restart everything seems to work fine for a period of time.

Issue Links

• relates to: IMPALA-2567 KRPC milestone 1
• relates to: IMPALA-2990 Coordinator should timeout and cancel queries with unresponsive / stuck executors
• relates to: IMPALA-6792 Appears to be a memory leak in orphaned fragments
• relates to: IMPALA-7239 Mitigate ParseSmaps() overhead
• relates to: IMPALA-7984 Port UpdateFilter() and PublishFilter() to KRPC