[IMPALA-499] Querying Parquet table that contains a different number of columns than the data file causes impala to coredump in impala::HdfsParquetScanner::Close () [hdfs-parquet-scanner.cc:203]

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.1
• Fix Version/s: Impala 1.1.1
• Assignee: Nong Li
• Reporter: Lenni Kuff
• Created: 31/Jul/13 04:23
• Updated: 20/Dec/15 00:05
• Resolved: 15/Aug/13 18:16

Description

Querying Parquet table that contains a different number of columns than the data file causes impala to segfault in impala::HdfsParquetScanner::Close () [hdfs-parquet-scanner.cc:203].

Repro:
1) Create parquet format table w/ 2+ columns and insert some data
2) Remove/Add a column using ALTER TABLE FOO DROP/ADD COLUMN
3) SELECT * FROM <TABLE>

Stack Trace

#7  0x0000000000d9f0c6 in impala::AtomicInt<int>::operator+= (this=0x224, delta=0)
    at /home/lskuff/dev/Impala/be/src/common/atomic.h:56
#8  0x0000000000da66e3 in impala::DiskIoMgr::ScanRange::CleanupQueuedBuffers (this=0x4c05d40)
    at /home/lskuff/dev/Impala/be/src/runtime/disk-io-mgr-scan-range.cc:158
#9  0x0000000000da6555 in impala::DiskIoMgr::ScanRange::Cancel (this=0x4c05d40, status=...)
    at /home/lskuff/dev/Impala/be/src/runtime/disk-io-mgr-scan-range.cc:150
#10 0x0000000000f3e6bd in impala::ScannerContext::Stream::AttachCompletedResources (this=0x55fc100, 
    batch=0x4e61400, done=true) at /home/lskuff/dev/Impala/be/src/exec/scanner-context.cc:98
#11 0x0000000000f3e1ee in impala::ScannerContext::AttachCompletedResources (this=0x6259cb0, batch=0x4e61400, 
    done=true) at /home/lskuff/dev/Impala/be/src/exec/scanner-context.cc:50
#12 0x0000000000f5639c in impala::HdfsScanner::AddFinalRowBatch (this=0x6f78000)
    at /home/lskuff/dev/Impala/be/src/exec/hdfs-scanner.cc:154
#13 0x0000000000f1e388 in impala::HdfsParquetScanner::Close (this=0x6f78000)
    at /home/lskuff/dev/Impala/be/src/exec/hdfs-parquet-scanner.cc:203
#14 0x0000000000eeca43 in impala::HdfsScanNode::ScannerThreadHelper (this=0x5da3c00)
    at /home/lskuff/dev/Impala/be/src/exec/hdfs-scan-node.cc:694
#15 0x0000000000eea75e in impala::HdfsScanNode::ScannerThread (this=0x5da3c00)
    at /home/lskuff/dev/Impala/be/src/exec/hdfs-scan-node.cc:740
#16 0x0000000000ef96f6 in boost::_mfi::mf0<void, impala::HdfsScanNode>::operator() (this=0x71b79e8, 
    p=0x5da3c00) at /usr/include/boost/bind/mem_fn_template.hpp:49
#17 0x0000000000ef95a2 in boost::_bi::list1<boost::_bi::value<impala::HdfsScanNode*> >::operator()<boost::_mfi::mf0<void, impala::HdfsScanNode>, boost::_bi::list0> (this=0x71b79f8, f=..., a=...)
    at /usr/include/boost/bind/bind.hpp:253
#18 0x0000000000ef94ed in boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::HdfsScanNode>, boost::_bi::list1<boost::_bi::value<impala::HdfsScanNode*> > >::operator() (this=0x71b79e8)
    at /usr/include/boost/bind/bind_template.hpp:20
#19 0x0000000000ef9050 in boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::HdfsScanNode>, boost::_bi::list1<boost::_bi::value<impala::HdfsScanNode*> > > >::run (this=0x71b7860)
    at /usr/include/boost/thread/detail/thread.hpp:61