[IMPALA-4995] crash when limit clause > MAX_INT

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: crash
• Assignee: Lars Volker
• Reporter: Matthew Mulder
• Created: 25/Feb/17 01:42
• Updated: 05/Mar/17 00:42
• Resolved: 01/Mar/17 19:59

Description

I wanted to sort my table of 28.8 billion rows, so I ran

create table partsupp_sorted stored as parquet as select * from partsupp order by ps_availqty;

Impala eventually finished but warned

WARNINGS: Ignoring ORDER BY clause without LIMIT or OFFSET: ORDER BY ps_availqty ASC.
An ORDER BY appearing in a view, subquery, union operand, or an insert/ctas statement has no effect on the query result unless a LIMIT and/or OFFSET is used in conjunction with the ORDER BY.

The table was not sorted. So I did the logical thing: I dropped the table and reran my query with the limit clause.

create table partsupp_sorted stored as parquet as select * from partsupp order by ps_availqty limit 28800000000;

After working for about half an hour impalad crashed.
It looks like be/src/exec/topn-node.cc

int index = sorted_top_n_.size() - 1;

is the culprit. An int doesn't go up to 28800000000, so it probably wound up as a negative value which caused the SIGABRT when trying to set the array value in

sorted_top_n_[index] = tuple;

Stack trace

#0  0x000000305cc32625 in raise () from /lib64/libc.so.6
#1  0x000000305cc33e05 in abort () from /lib64/libc.so.6
#2  0x00007f634b192c55 in os::abort(bool) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#3  0x00007f634b314cd7 in VMError::report_and_die() () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#4  0x00007f634b31525e in crash_handler(int, siginfo*, void*) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#5  0x00007f634b191df2 in os::Linux::chained_handler(int, siginfo*, void*) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#6  0x00007f634b197ad6 in JVM_handle_linux_signal () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#7  <signal handler called>
#8  0x00007f634b1896f1 in os::is_first_C_frame(frame*) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#9  0x00007f634b3133cd in VMError::report(outputStream*) () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#10 0x00007f634b3148da in VMError::report_and_die() () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#11 0x00007f634b197b6f in JVM_handle_linux_signal () from /usr/java/jdk1.7.0_75-cloudera/jre/lib/amd64/server/libjvm.so
#12 <signal handler called>
#13 0x0000000000c9fd01 in impala::TopNNode::PrepareForOutput (this=0x3778dcc0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/topn-node.cc:229
#14 0x0000000000ca0b48 in impala::TopNNode::Open (this=0x3778dcc0, state=0x45230a00) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/topn-node.cc:167
#15 0x0000000000db3671 in impala::PlanFragmentExecutor::OpenInternal (this=0x45231ab0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/plan-fragment-executor.cc:324
#16 0x0000000000db4db0 in impala::PlanFragmentExecutor::Open (this=0x45231ab0) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/plan-fragment-executor.cc:296
#17 0x0000000000daddd0 in impala::FragmentInstanceState::Exec (this=0x45231800) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/fragment-instance-state.cc:65
#18 0x0000000000db710f in impala::QueryExecMgr::ExecFInstance (this=0x99d00c0, fis=0x45231800) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/runtime/query-exec-mgr.cc:109
#19 0x0000000000bbc374 in operator() (name=Unhandled dwarf expression opcode 0xf3
) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:767
#20 impala::Thread::SuperviseThread (name=Unhandled dwarf expression opcode 0xf3
) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/util/thread.cc:317
#21 0x0000000000bbcd54 in operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (this=0x31e4e400)
    at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:457
#22 operator() (this=0x31e4e400) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
#23 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (
    this=0x31e4e400) at /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p1/include/boost/thread/detail/thread.hpp:116
#24 0x0000000000e08dca in thread_proxy ()
#25 0x000000305d0079d1 in start_thread () from /lib64/libpthread.so.0
#26 0x000000305cce88fd in clone () from /lib64/libc.so.6

Issue Links

• is related to: IMPALA-5004 Switch to sorting node for large TopN queries
• relates to: IMPALA-3471 TopN should be able to spill