Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10426

Impala crashes when it tries to write invalid timestamp value with INT64 Parquet timestamp type

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Reproduction

      set parquet_timestamp_type=INT64_MICROS;
      create table ts_test (ts timestamp) stored as parquet;
      insert into ts_test values ('21-01-07 10:53:47.224885000');
      

      Stack trace

      #0 0x00007ff56e752438 in raise () from /lib/x86_64-linux-gnu/libc.so.6
      #1 0x00007ff56e75403a in abort () from /lib/x86_64-linux-gnu/libc.so.6
      #2 0x00007ff571486789 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
      #3 0x00007ff57164b2c0 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
      #4 0x00007ff57149062f in JVM_handle_linux_signal () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
      #5 0x00007ff571483488 in ?? () from /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
      #6 <signal handler called>
      #7 0x0000000001daaf36 in boost::date_time::date<boost::gregorian::date, boost::gregorian::gregorian_calendar, boost::gregorian::date_duration>::is_not_a_date (this=0x8)
       at /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:117
      #8 0x0000000001daaec0 in boost::date_time::date<boost::gregorian::date, boost::gregorian::gregorian_calendar, boost::gregorian::date_duration>::is_special (this=0x8)
       at /opt/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/date_time/date.hpp:112
      #9 0x0000000001daa36a in impala::TimestampValue::HasDate (this=0x0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:182
      #10 0x0000000001daa3a6 in impala::TimestampValue::HasDateAndTime (this=0x0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.h:184
      #11 0x0000000002bbf180 in impala::TimestampValue::FloorUtcToUnixTimeMicros (this=0x0, unix_time_micros=0x11b711c0) at /home/boroknagyz/Impala/be/src/runtime/timestamp-value.inline.h:137
      #12 0x0000000002eae9d3 in impala::HdfsParquetTableWriter::Int64MicroTimestampColumnWriter::ConvertTimestamp (this=0x11b70f80, ts=..., result=0x11b711c0)
       at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:642
      #13 0x0000000002eae870 in impala::HdfsParquetTableWriter::Int64TimestampColumnWriterBase::ConvertValue (this=0x11b70f80, value=0x0)
       at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:601
      #14 0x0000000002eaeacd in impala::HdfsParquetTableWriter::BaseColumnWriter::AppendRow (this=0x11b70f80, row=0xcdc4000) at /home/boroknagyz/Impala/be/src/exec/parquet/hdfs-parquet-table-writer.cc:664
      #15 0x0000000002ea6d00 in impala::HdfsParquetTableWriter::AppendRows (this=0xe1f2e00, batch=0x10518700, row_group_indices=..., new_file=0x7ff4bf5a301f)
      ...

      At frame #14 the timestamp value in the row is NULL, so ConvertValue() is invoked with 'value' being NULL which is not handled well.

       

        Attachments

          Activity

            People

            • Assignee:
              boroknagyz Zoltán Borók-Nagy
              Reporter:
              boroknagyz Zoltán Borók-Nagy

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment