[ORC-317] C++ reader crash on reading corrupt types

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0, 1.1.2, 1.2.3, 1.3.4, 1.4.3
• Fix Version/s: 1.5.0
• Component/s: C++, Java
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 07/Mar/18 03:19
• Updated: 14/May/18 22:24
• Resolved: 23/Mar/18 16:58

Description

The c++ reader crash on reading the attached corrupt orc file. Looks like an infinite loop:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000860813 in std::__copy_move<true, true, std::random_access_iterator_tag>::__copy_m<orc::Type*> (__first=0x0, __last=0x0, __result=0x0) at /usr/include/c++/4.8/bits/stl_algobase.h:368
368             __copy_m(const _Tp* __first, const _Tp* __last, _Tp* __result)
(gdb) bt
#0  0x0000000000860813 in std::__copy_move<true, true, std::random_access_iterator_tag>::__copy_m<orc::Type*> (__first=0x0, __last=0x0, __result=0x0) at /usr/include/c++/4.8/bits/stl_algobase.h:368
#1  0x0000000000860779 in std::__copy_move_a<true, orc::Type**, orc::Type**> (__first=0x0, __last=0x0, __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_algobase.h:390
#2  0x000000000086067c in std::__copy_move_a2<true, orc::Type**, orc::Type**> (__first=0x0, __last=0x0, __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_algobase.h:428
#3  0x000000000086050b in std::copy<std::move_iterator<orc::Type**>, orc::Type**> (__first=..., __last=..., __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_algobase.h:460
#4  0x000000000086035b in std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<orc::Type**>, orc::Type**> (__first=..., __last=..., __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_uninitialized.h:93
#5  0x0000000000860138 in std::uninitialized_copy<std::move_iterator<orc::Type**>, orc::Type**> (__first=..., __last=..., __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_uninitialized.h:117
#6  0x000000000085fcfe in std::__uninitialized_copy_a<std::move_iterator<orc::Type**>, orc::Type**, orc::Type*> (__first=..., __last=..., __result=0x38e5c30) at /usr/include/c++/4.8/bits/stl_uninitialized.h:258
#7  0x000000000085f6d7 in std::__uninitialized_move_if_noexcept_a<orc::Type**, orc::Type**, std::allocator<orc::Type*> > (__first=0x0, __last=0x0, __result=0x38e5c30, __alloc=...) at /usr/include/c++/4.8/bits/stl_uninitialized.h:281
#8  0x000000000085ed23 in std::vector<orc::Type*, std::allocator<orc::Type*> >::_M_emplace_back_aux<orc::Type*>(orc::Type*&&) (this=0x38e5af8) at /usr/include/c++/4.8/bits/vector.tcc:412
#9  0x000000000085e803 in std::vector<orc::Type*, std::allocator<orc::Type*> >::emplace_back<orc::Type*>(orc::Type*&&) (this=0x38e5af8) at /usr/include/c++/4.8/bits/vector.tcc:101
#10 0x000000000085de02 in std::vector<orc::Type*, std::allocator<orc::Type*> >::push_back(orc::Type*&&) (this=0x38e5af8, __x=<unknown type in /mnt/volume1/orc/build/tools/src/orc-scan, CU 0xaddb7, DIE 0xc38ee>) at /usr/include/c++/4.8/bits/stl_vector.h:920
#11 0x0000000000859c00 in orc::TypeImpl::addChildType (this=0x38e5ad0, childType=std::unique_ptr<orc::Type> containing 0x0) at /mnt/volume1/orc/c++/src/TypeImpl.cc:138
#12 0x0000000000859c70 in orc::TypeImpl::addStructField (this=0x38e5ad0, fieldName="_col0", fieldType=std::unique_ptr<orc::Type> containing 0x0) at /mnt/volume1/orc/c++/src/TypeImpl.cc:147
#13 0x000000000085b8f7 in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#14 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#15 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#16 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#17 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#18 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#19 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#20 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#21 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#22 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
#23 0x000000000085b8cd in orc::convertType (type=..., footer=...) at /mnt/volume1/orc/c++/src/TypeImpl.cc:399
......

The java reader can't survive from the same problem:

Exception in thread "main" java.lang.StackOverflowError
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        at org.apache.orc.OrcUtils.convertTypeFromProtobuf(OrcUtils.java:515)
        ...... 

Issue Links

• blocks: ORC-315 More robust on reading random corrupt files
• relates to: ORC-325 Java reader should protect against malformed types in ORC footers
• links to: GitHub Pull Request #231