[IMPALA-4731] Sorter crash Impalad instance

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: crash
• Environment: centos 6
• Target Version: Impala 2.9.0
• Assignee: Thomas Tauber-Marshall
• Reporter: fishing
• Created: 05/Jan/17 15:09
• Updated: 27/Apr/17 18:22
• Resolved: 27/Apr/17 18:22

Description

log.WARNING.20170105-180851.32662:F0105 19:26:32.863729 1907 sorter.cc:1085] Check failed: index_ < run->num_tuples() (7486 vs. 7486) Can only advance one past end of run

log.ERROR.20170105-202624.26390:F0105 21:27:56.650339 25273 sorter.cc:1103] Check failed: index_ >= 0 (-1 vs. 0) Can only advance one before start of run

(gdb) bt
#0 0x000000314b832635 in raise () from /lib64/libc.so.6
#1 0x000000314b833e15 in abort () from /lib64/libc.so.6
#2 0x00007fd8667a67b9 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x00007fd86679e5bd in google::LogMessage::Fail () at src/logging.cc:1315
#4 0x00007fd8667a030d in google::LogMessage::SendToLog (this=<optimized out>) at src/logging.cc:1269
#5 0x00007fd86679e0ca in google::LogMessage::Flush (this=this@entry=0x7fd6c8389ef0) at src/logging.cc:1138
#6 0x00007fd8667a0dce in google::LogMessageFatal::~LogMessageFatal (this=0x7fd6c8389ef0, __in_chrg=<optimized out>) at src/logging.cc:1836
#7 0x0000000001f4878e in impala::Sorter::TupleIterator::Next (this=0x7fd6c8389f40, run=0x7fcfb74a0100, tuple_size=263)
at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1085
#8 0x0000000001f443f2 in impala::Sorter::TupleSorter::Partition (this=0x7fd5190b6000, begin=..., end=..., pivot=0x7fd2fd7c7418, cut=0x7fd6c838a0b0)
at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1219
#9 0x0000000001f44753 in impala::Sorter::TupleSorter::SortHelper (this=0x7fd5190b6000, begin=..., end=...) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1248
#10 0x0000000001f44943 in impala::Sorter::TupleSorter::SortHelper (this=0x7fd5190b6000, begin=..., end=...) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1257
#11 0x0000000001f44943 in impala::Sorter::TupleSorter::SortHelper (this=0x7fd5190b6000, begin=..., end=...) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1257
#12 0x0000000001f44943 in impala::Sorter::TupleSorter::SortHelper (this=0x7fd5190b6000, begin=..., end=...) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1257
#13 0x0000000001f43c88 in impala::Sorter::TupleSorter::Sort (this=0x7fd5190b6000, run=0x7fcfb74a0100) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1150
#14 0x0000000001f46b5f in impala::Sorter::SortCurrentInputRun (this=0x3f3b3700) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1477
#15 0x0000000001f46267 in impala::Sorter::InputDone (this=0x3f3b3700) at /export/ldb/online/impala_gitlab/be/src/runtime/sorter.cc:1412
#16 0x0000000001c85fee in impala::SortNode::SortInput (this=0xd11f660, state=0x951f0800) at /export/ldb/online/impala_gitlab/be/src/exec/sort-node.cc:166
#17 0x0000000001c85202 in impala::SortNode::Open (this=0xd11f660, state=0x951f0800) at /export/ldb/online/impala_gitlab/be/src/exec/sort-node.cc:81
#18 0x0000000001f38f99 in impala::PlanFragmentExecutor::OpenInternal (this=0x7fd550c0d3f0) at /export/ldb/online/impala_gitlab/be/src/runtime/plan-fragment-executor.cc:321
#19 0x0000000001f38a8c in impala::PlanFragmentExecutor::Open (this=0x7fd550c0d3f0) at /export/ldb/online/impala_gitlab/be/src/runtime/plan-fragment-executor.cc:293
#20 0x00000000019e2b84 in impala::FragmentMgr::FragmentExecState::Exec (this=0x7fd550c0d000) at /export/ldb/online/impala_gitlab/be/src/service/fragment-exec-state.cc:61
#21 0x00000000019dc112 in impala::FragmentMgr::FragmentThread (this=0xa324880, fragment_instance_id=...) at /export/ldb/online/impala_gitlab/be/src/service/fragment-mgr.cc:90
#22 0x00000000019dfeba in boost::_mfi::mf1<void, impala::FragmentMgr, impala::TUniqueId>::operator() (this=0x88fd8960, p=0xa324880, a1=...)
at /usr/local/include/boost/bind/mem_fn_template.hpp:165
#23 0x00000000019dfc77 in boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::TUniqueId> >::operator()<boost::_mfi::mf1<void, impala::FragmentMgr, impala::TUniqueId>, boost::_bi::list0> (this=0x88fd8970, f=..., a=...) at /usr/local/include/boost/bind/bind.hpp:313
#24 0x00000000019df5a1 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::TUniqueId>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::TUniqueId> > >::operator() (this=0x88fd8960) at /usr/local/include/boost/bind/bind_template.hpp:20
#25 0x00000000019def34 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::TUniqueId>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::TUniqueId> > >, void>::invoke (function_obj_ptr=...)
at /usr/local/include/boost/function/function_template.hpp:153
#26 0x00000000017e8504 in boost::function0<void>::operator() (this=0x7fd6c838ba40) at /usr/local/include/boost/function/function_template.hpp:767
#27 0x0000000001aa0ad3 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=...,
functor=..., thread_started=0x7fd80c62ab00) at /export/ldb/online/impala_gitlab/be/src/util/thread.cc:317

Issue Links

• relates to: IMPALA-397 ORDER BY rand() does not work.
• relates to: IMPALA-4728 materialize expressions for window sorts vs lazy expression evaluation