Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4997

crash when using sortby hint on a very large table

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Backend
    • Labels:

      Description

      I have a 28.8 billion row table that I'm trying to sort into another table:

      insert into partsupp_sorted /*+ sortby(ps_availqty) */ select * from partsupp;

      This caused impalad to crash.
      Stack trace:

      #0  0x000000305cc32625 in raise () from /lib64/libc.so.6
      #1  0x000000305cc33e05 in abort () from /lib64/libc.so.6
      #2  0x00007f37b6a47c55 in os::abort(bool) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
      #3  0x00007f37b6bc9cd7 in VMError::report_and_die() () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
      #4  0x00007f37b6a4cb6f in JVM_handle_linux_signal () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
      #5  <signal handler called>
      #6  0x00007f37602580db in Compare ()
      #7  0x0000000000dbb9d1 in Compare (this=0xa633200, lhs=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/util/tuple-row-compare.h:99
      #8  Less (this=0xa633200, lhs=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/util/tuple-row-compare.h:108
      #9  impala::Sorter::TupleSorter::Less (this=0xa633200, lhs=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1144
      #10 0x0000000000dbbeae in impala::Sorter::TupleSorter::Partition (this=0xa633200, begin=..., end=..., pivot=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1222
      #11 0x0000000000dbe00d in impala::Sorter::TupleSorter::SortHelper (this=0xa633200, begin=..., end=...) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1249
      #12 0x0000000000dbe253 in impala::Sorter::TupleSorter::Sort (this=0xa633200, run=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1151
      #13 0x0000000000dbe334 in impala::Sorter::SortCurrentInputRun (this=0xa76aaa0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1478
      #14 0x0000000000dbf20b in impala::Sorter::InputDone (this=0xa76aaa0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/sorter.cc:1413
      #15 0x0000000000c9a8a2 in impala::SortNode::SortInput (this=0x214a7c00, state=0xa50bc00) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/sort-node.cc:166
      #16 0x0000000000c9b55e in impala::SortNode::Open (this=0x214a7c00, state=0xa50bc00) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/sort-node.cc:81
      #17 0x0000000000db3671 in impala::PlanFragmentExecutor::OpenInternal (this=0xa50b7b0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/plan-fragment-executor.cc:324
      #18 0x0000000000db4db0 in impala::PlanFragmentExecutor::Open (this=0xa50b7b0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/plan-fragment-executor.cc:296
      #19 0x0000000000daddd0 in impala::FragmentInstanceState::Exec (this=0xa50b500) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/fragment-instance-state.cc:65
      #20 0x0000000000db710f in impala::QueryExecMgr::ExecFInstance (this=0xad0e4e0, fis=0xa50b500) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/query-exec-mgr.cc:109
      #21 0x0000000000bbc374 in operator() (name=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:767
      #22 impala::Thread::SuperviseThread (name=Unhandled dwarf expression opcode 0xf3
      ) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/util/thread.cc:317
      #23 0x0000000000bbcd54 in operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (this=0xa9a1400)
          at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:457
      #24 operator() (this=0xa9a1400) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
      #25 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (this=0xa9a1400)
          at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/thread/detail/thread.hpp:116
      #26 0x0000000000e08dca in thread_proxy ()
      #27 0x000000305d0079d1 in start_thread () from /lib64/libpthread.so.0
      #28 0x000000305cce88fd in clone () from /lib64/libc.so.6

        Attachments

          Activity

            People

            • Assignee:
              lv Lars Volker
              Reporter:
              mmulder Matthew Mulder
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: