[IMPALA-1817] Codegen-related crash in PAGG::Open() when running TPC-H Q16

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Works for Me
• Affects Version/s: Impala 2.1
• Fix Version/s: Impala 2.2
• Assignee: Skye Wanderman-Milne
• Reporter: Ippokratis Pandis
• Created: 25/Feb/15 08:39
• Updated: 20/Dec/15 00:05
• Resolved: 05/Mar/15 08:09

Description

TPC-H Q16 is crashing in PAGG::Open() when I set mem_limit. When codegen is disabled it is running ok. To repro:

[localhost:21000] > use tpch_parquet;
Query: use tpch_parquet
[localhost:21000] > set mem_limit=1954m;
MEM_LIMIT set to 1954m
[localhost:21000] > set disable_codegen=true;
DISABLE_CODEGEN set to true
[localhost:21000] > select
...
Fetched 18314 row(s) in 2.82s
[localhost:21000] > set disable_codegen=false;
DISABLE_CODEGEN set to false
[localhost:21000] > select
                  >   p_brand,
                  >   p_type,
                  >   p_size,
                  >   count(distinct ps_suppkey) as supplier_cnt
                  > from
                  >   partsupp,
                  >   part
                  > where
                  >   p_partkey = ps_partkey
                  >   and p_brand <> 'Brand#45'
                  >   and p_type not like 'MEDIUM POLISHED%'
                  >   and p_size in (49, 14, 23, 45, 19, 3, 36, 9)
                  >   and ps_suppkey not in (
                  >     select
                  >       s_suppkey
                  >     from
                  >       supplier
                  >     where
                  >       s_comment like '%Customer%Complaints%'
                  >   )
                  > group by
                  >   p_brand,
                  >   p_type,
                  >   p_size
                  > order by
                  >   supplier_cnt desc,
                  >   p_brand,
                  >   p_type,
                  >   p_size;
Query: select p_brand,
...
Socket error 104: Connection reset by peer

hs_err.log has the following:

RAX=0x0000000000000000 is an unknown value
RBX=0x000000000cf65030 is an unknown value
RCX=0x0000000008ed8000 is an unknown value
RDX=0x0000000000000000 is an unknown value
RSP=0x00007f9f390fb8f0 is an unknown value
RBP=0x00007f9f390fbba0 is an unknown value
C  [impalad+0x1139148]  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x724
C  [impalad+0x105f6b0]  impala::PlanFragmentExecutor::OpenInternal()+0x90
C  [impalad+0x105f495]  impala::PlanFragmentExecutor::Open()+0x2df
C  [impalad+0xce9ef8]  impala::FragmentMgr::FragmentExecState::Exec()+0x26
C  [impalad+0xce117f]  impala::FragmentMgr::FragmentExecThread(impala::FragmentMgr::FragmentExecState*)+0x5f
C  [impalad+0xce5aff]  boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>::operator()(impala::FragmentMgr*, impala::FragmentMgr::FragmentExecState*) const+0x63
C  [impalad+0xce590d]  void boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> >::operator()<boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>&, boost::_bi::list0&, int)+0x69
C  [impalad+0xce5153]  boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >::operator()()+0x39
C  [impalad+0xce4c04]  boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::FragmentMgr, impala::FragmentMgr::FragmentExecState*>, boost::_bi::list2<boost::_bi::value<impala::FragmentMgr*>, boost::_bi::value<impala::FragmentMgr::FragmentExecState*> > >, void>::invoke(boost::detail::function::function_buffer&)+0x23
C  [impalad+0xb84db6]  boost::function0<void>::operator()() const+0x52
C  [impalad+0xdbd442]  impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()()>, impala::Promise<long>*)+0x2de
C  [impalad+0xdc50ce]  void boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()()>, impala::Promise<long>*), boost::_bi::list0&, int)+0xb0
C  [impalad+0xdc5017]  boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()()+0x39
C  [impalad+0xdc4faa]  boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()()> >, boost::_bi::value<impala::Promise<long>*> > > >::run()+0x1e