[IMPALA-4019] Crash when HdfsTableSink::table_desc_->partition_descriptors() is missing the default partition

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.7.0
• Component/s: Backend
• Labels: crash
• Assignee: Tim Armstrong
• Reporter: Juan Yu
• Created: 25/Aug/16 04:10
• Updated: 02/Sep/16 20:55
• Resolved: 02/Sep/16 20:55

Description

Impalad crashed during running a simple insert query

insert into testcase1 partition(part) select 'AAA', 'BBB', cast(rand() * 1000 + 20000 as int) from smaller limit 5

Wrote minidump to /var/log/impala-minidumps/impalad/634cf038-5cb7-85ca-7da86373-7297c956.dmp
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000000d041f8, pid=5116, tid=140625151235840
#
# JRE version: Java(TM) SE Runtime Environment (7.0_67-b01) (build 1.7.0_67-b01)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [impalad+0x9041f8]  impala::HdfsTableSink::BuildHdfsFileNames(impala::HdfsPartitionDescriptor const&, impala::OutputPartition*)+0x238
#
# Core dump written. Default location: /var/log/impalad/core or core.5116
#
# An error report file with more information is saved as:
# /var/log/impalad/hs_err_pid5116.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#

The table has 20K partitions and 1.5 million files.
Please note the same query works fine on a much smaller table.

Issue Links

• relates to: IMPALA-1702 "invalidate metadata" can cause duplicate TableIds