[IMPALA-10426] Impala crashes when it tries to write invalid timestamp value with INT64 Parquet timestamp type

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: crash newbie ramp-up
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-6
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 07/Jan/21 10:13
• Updated: 22/Mar/22 07:51
• Resolved: 15/Jan/21 08:56

Description

Reproduction

set parquet_timestamp_type=INT64_MICROS;
create table ts_test (ts timestamp) stored as parquet;
insert into ts_test values ('21-01-07 10:53:47.224885000');

Stack trace

#0 0x00007ff56e752438 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ff56e75403a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ff571486789 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#3 0x00007ff57164b2c0 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#4 0x00007ff57149062f in JVM_handle_linux_signal () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#5 0x00007ff571483488 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
#6 <signal handler called>
#7 0x0000000001daaf36 in boost::date_time::date<boost::gregorian::date, boost::gregorian::gregorian_calendar, boost::gregorian::date_duration>::is_not_a_date (this=0x8)
 at /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:117
#8 0x0000000001daaec0 in boost::date_time::date<boost::gregorian::date, boost::gregorian::gregorian_calendar, boost::gregorian::date_duration>::is_special (this=0x8)
 at /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:112
#9 0x0000000001daa36a in impala::TimestampValue::HasDate (this=0x0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:182
#10 0x0000000001daa3a6 in impala::TimestampValue::HasDateAndTime (this=0x0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:184
#11 0x0000000002bbf180 in impala::TimestampValue::FloorUtcToUnixTimeMicros (this=0x0, unix_time_micros=0x11b711c0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.inline.h:137
#12 0x0000000002eae9d3 in impala::HdfsParquetTableWriter::Int64MicroTimestampColumnWriter::ConvertTimestamp (this=0x11b70f80, ts=..., result=0x11b711c0)
 at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:642
#13 0x0000000002eae870 in impala::HdfsParquetTableWriter::Int64TimestampColumnWriterBase::ConvertValue (this=0x11b70f80, value=0x0)
 at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:601
#14 0x0000000002eaeacd in impala::HdfsParquetTableWriter::BaseColumnWriter::AppendRow (this=0x11b70f80, row=0xcdc4000) at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:664
#15 0x0000000002ea6d00 in impala::HdfsParquetTableWriter::AppendRows (this=0xe1f2e00, batch=0x10518700, row_group_indices=..., new_file=0x7ff4bf5a301f)
...

At frame #14 the timestamp value in the row is NULL, so ConvertValue() is invoked with 'value' being NULL which is not handled well.