[IMPALA-6414] Impalad binary failed to start with SIGSEGV with GPerfTools 2.6.3 on certain platforms

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.12.0
• Fix Version/s: Impala 2.12.0
• Component/s: Backend
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-3
• Assignee: Michael Ho
• Reporter: Michael Ho
• Created: 18/Jan/18 03:16
• Updated: 21/Feb/18 07:38
• Resolved: 19/Jan/18 00:05

Description

A recent commit upgraded GPerfTools to version 2.6.3. Apparently, this mysteriously caused Impala to fail to start on certain platforms (e.g. Oracle Enterprise Linux 6.9). There is no core file or anything found when this failed.

$ /opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.176/lib/impala/sbin-retail/impalad
Segmentation fault
$ strace /opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.176/lib/impala/sbin-retail/impalad
execve("/opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.176/lib/impala/sbin-retail/impalad", ["/opt/cloudera/parcels/CDH-5.15.0"...], [/* 23 vars */]) = -1 EINVAL (Invalid argument)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} ---
+++ killed by SIGSEGV +++
Segmentation fault

It was verified that downgrading GPerfTools to previous version (2.5) made the problem go away.

Issue Links

• is broken by: IMPALA-5528 tcmalloc contention much higher with concurrency after KRPC patch