Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.2
-
None
Description
The query generator found a bug with FIRST_VALUE when there is a window like "ROWS X PRECEDING Y PRECEDING", such that X < Y and (X > the size of a partition).
The query generator's reproduction is very complicated, but I have the following, simpler example:
select id, date_string_col, first_value(id) over (partition by date_string_col order by id rows between 10 preceding and 3 preceding) from alltypes where id < 15
This is somewhat of a corner case because it requires having a relatively strange window and data. On debug builds we get a DCHECK failure, but in release builds there may be incorrect results.
#0 0x00007f5d1abd30d5 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x00007f5d1abd30d5 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f5d1abd683b in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x0000000002091e49 in google::DumpStackTraceAndExit () at src/utilities.cc:147 #3 0x000000000208ad1d in google::LogMessage::Fail () at src/logging.cc:1315 #4 0x000000000208d4bf in google::LogMessage::SendToLog (this=0x7f5c6912e9b0) at src/logging.cc:1269 #5 0x000000000208a887 in google::LogMessage::Flush (this=0x7f5c6912e9b0) at src/logging.cc:1138 #6 0x000000000208dd4d in google::LogMessageFatal::~LogMessageFatal (this=0x7f5c6912e9b0, __in_chrg=<optimized out>) at src/logging.cc:1836 #7 0x00000000015a99d8 in impala::AnalyticEvalNode::InitNextPartition (this=0xcc77500, stream_idx=10) at /opt/Impala/be/src/exec/analytic-eval-node.cc:469 #8 0x00000000015a5763 in impala::AnalyticEvalNode::ProcessChildBatch (this=0xcc77500, state=0xba3a000) at /opt/Impala/be/src/exec/analytic-eval-node.cc:598 #9 0x00000000015a515e in impala::AnalyticEvalNode::ProcessChildBatches (this=0xcc77500, state=0xba3a000) at /opt/Impala/be/src/exec/analytic-eval-node.cc:531 #10 0x00000000015a6897 in impala::AnalyticEvalNode::GetNext (this=0xcc77500, state=0xba3a000, row_batch=0x7f5c6912ed20, eos=0x7f5c6912edff) at /opt/Impala/be/src/exec/analytic-eval-node.cc:707 #11 0x00000000015900f3 in impala::SortNode::SortInput (this=0x8615400, state=0xba3a000) at /opt/Impala/be/src/exec/sort-node.cc:143 #12 0x000000000158f668 in impala::SortNode::Open (this=0x8615400, state=0xba3a000) at /opt/Impala/be/src/exec/sort-node.cc:71 #13 0x0000000001481106 in impala::PlanFragmentExecutor::OpenInternal (this=0xb99d810) at /opt/Impala/be/src/runtime/plan-fragment-executor.cc:331 #14 0x0000000001480f10 in impala::PlanFragmentExecutor::Open (this=0xb99d810) at /opt/Impala/be/src/runtime/plan-fragment-executor.cc:317 #15 0x0000000001105a5c in impala::FragmentMgr::FragmentExecState::Exec (this=0xb99d600) at /opt/Impala/be/src/service/fragment-exec-state.cc:49 #16 0x00000000010fcc45 in impala::FragmentMgr::FragmentExecThread (this=0xee35d40, exec_state=0xb99d600) at /opt/Impala/be/src/service/fragment-mgr.cc:70 #17 0x0000000001101665 in boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>::operator() (this=0x7bf76a0, p=0xee35d40, a1=0xb99d600) at /usr/include/boost/bind/mem_fn_template.hpp:165 #18 0x0000000001101473 in boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> >::operator()<boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list0> (this=0x7bf76b0, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313 #19 0x0000000001100cb9 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >::operator() (this=0x7bf76a0) at /usr/include/boost/bind/bind_template.hpp:20 #20 0x000000000110076a in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153 #21 0x0000000000f9cbc0 in boost::function0<void>::operator() (this=0x7f5c6912fdb0) at /usr/include/boost/function/function_template.hpp:1013 #22 0x00000000011dcc6a in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=..., functor=..., thread_started=0x7f5c6a130d70) at /opt/Impala/be/src/util/thread.cc:311 #23 0x00000000011e48f6 in boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&, int) (this=0xd4eb810, f=@0xd4eb808: 0xceeb1b8, a=...) at /usr/include/boost/bind/bind.hpp:457 #24 0x00000000011e483f in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0xd4eb808) at /usr/include/boost/bind/bind_template.hpp:20 #25 0x00000000011e47d2 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0xd4eb680) at /usr/include/boost/thread/detail/thread.hpp:61 #26 0x00007f5d1d347ce9 in thread_proxy () from /usr/lib/libboost_thread.so.1.46.1 #27 0x00007f5d1d125e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #28 0x00007f5d1ac908bd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #29 0x0000000000000000 in ?? ()
From .FATAL file:
F0316 11:12:58.511064 10696 analytic-eval-node.cc:469] Check failed: last_result_idx_ == curr_partition_idx_ - 1 (6 vs. 9)
DB: Functional