Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1888

FIRST_VALUE produces incorrect results with some strange windows

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2
    • Fix Version/s: Impala 2.2
    • Component/s: None
    • Labels:

      Description

      The query generator found a bug with FIRST_VALUE when there is a window like "ROWS X PRECEDING Y PRECEDING", such that X < Y and (X > the size of a partition).

      The query generator's reproduction is very complicated, but I have the following, simpler example:

      select id, date_string_col,
      first_value(id) over (partition by date_string_col order by id rows between 10 preceding and 3 preceding)
      from alltypes where id < 15
      

      This is somewhat of a corner case because it requires having a relatively strange window and data. On debug builds we get a DCHECK failure, but in release builds there may be incorrect results.

      #0  0x00007f5d1abd30d5 in raise () from /lib/x86_64-linux-gnu/libc.so.6
      #0  0x00007f5d1abd30d5 in raise () from /lib/x86_64-linux-gnu/libc.so.6
      #1  0x00007f5d1abd683b in abort () from /lib/x86_64-linux-gnu/libc.so.6
      #2  0x0000000002091e49 in google::DumpStackTraceAndExit () at src/utilities.cc:147
      #3  0x000000000208ad1d in google::LogMessage::Fail () at src/logging.cc:1315
      #4  0x000000000208d4bf in google::LogMessage::SendToLog (this=0x7f5c6912e9b0) at src/logging.cc:1269
      #5  0x000000000208a887 in google::LogMessage::Flush (this=0x7f5c6912e9b0) at src/logging.cc:1138
      #6  0x000000000208dd4d in google::LogMessageFatal::~LogMessageFatal (this=0x7f5c6912e9b0, __in_chrg=<optimized out>) at src/logging.cc:1836
      #7  0x00000000015a99d8 in impala::AnalyticEvalNode::InitNextPartition (this=0xcc77500, stream_idx=10) at /opt/Impala/be/src/exec/analytic-eval-node.cc:469
      #8  0x00000000015a5763 in impala::AnalyticEvalNode::ProcessChildBatch (this=0xcc77500, state=0xba3a000) at /opt/Impala/be/src/exec/analytic-eval-node.cc:598
      #9  0x00000000015a515e in impala::AnalyticEvalNode::ProcessChildBatches (this=0xcc77500, state=0xba3a000) at /opt/Impala/be/src/exec/analytic-eval-node.cc:531
      #10 0x00000000015a6897 in impala::AnalyticEvalNode::GetNext (this=0xcc77500, state=0xba3a000, row_batch=0x7f5c6912ed20, eos=0x7f5c6912edff) at /opt/Impala/be/src/exec/analytic-eval-node.cc:707
      #11 0x00000000015900f3 in impala::SortNode::SortInput (this=0x8615400, state=0xba3a000) at /opt/Impala/be/src/exec/sort-node.cc:143
      #12 0x000000000158f668 in impala::SortNode::Open (this=0x8615400, state=0xba3a000) at /opt/Impala/be/src/exec/sort-node.cc:71
      #13 0x0000000001481106 in impala::PlanFragmentExecutor::OpenInternal (this=0xb99d810) at /opt/Impala/be/src/runtime/plan-fragment-executor.cc:331
      #14 0x0000000001480f10 in impala::PlanFragmentExecutor::Open (this=0xb99d810) at /opt/Impala/be/src/runtime/plan-fragment-executor.cc:317
      #15 0x0000000001105a5c in impala::FragmentMgr::FragmentExecState::Exec (this=0xb99d600) at /opt/Impala/be/src/service/fragment-exec-state.cc:49
      #16 0x00000000010fcc45 in impala::FragmentMgr::FragmentExecThread (this=0xee35d40, exec_state=0xb99d600) at /opt/Impala/be/src/service/fragment-mgr.cc:70
      #17 0x0000000001101665 in boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>::operator() (this=0x7bf76a0, p=0xee35d40, a1=0xb99d600) at /usr/include/boost/bind/mem_fn_template.hpp:165
      #18 0x0000000001101473 in boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> >::operator()<boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list0> (this=0x7bf76b0, f=..., a=...) at /usr/include/boost/bind/bind.hpp:313
      #19 0x0000000001100cb9 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >::operator() (this=0x7bf76a0) at /usr/include/boost/bind/bind_template.hpp:20
      #20 0x000000000110076a in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
      #21 0x0000000000f9cbc0 in boost::function0<void>::operator() (this=0x7f5c6912fdb0) at /usr/include/boost/function/function_template.hpp:1013
      #22 0x00000000011dcc6a in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=..., functor=..., thread_started=0x7f5c6a130d70) at /opt/Impala/be/src/util/thread.cc:311
      #23 0x00000000011e48f6 in boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&, int) (this=0xd4eb810, f=@0xd4eb808: 0xceeb1b8, a=...) at /usr/include/boost/bind/bind.hpp:457
      #24 0x00000000011e483f in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0xd4eb808) at /usr/include/boost/bind/bind_template.hpp:20
      #25 0x00000000011e47d2 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0xd4eb680) at /usr/include/boost/thread/detail/thread.hpp:61
      #26 0x00007f5d1d347ce9 in thread_proxy () from /usr/lib/libboost_thread.so.1.46.1
      #27 0x00007f5d1d125e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
      #28 0x00007f5d1ac908bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
      #29 0x0000000000000000 in ?? ()
      

      From .FATAL file:

      F0316 11:12:58.511064 10696 analytic-eval-node.cc:469] Check failed: last_result_idx_ == curr_partition_idx_ - 1 (6 vs. 9)
      

      DB: Functional

        Attachments

          Activity

            People

            • Assignee:
              mjacobs Matthew Jacobs
              Reporter:
              tarasbob Taras Bobrovytsky
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: