Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.11.0
    • Fix Version/s: Impala 2.11.0
    • Component/s: Backend
    • Labels:
      None
    • Epic Color:
      ghx-label-5

      Description

      I'm seeing the following in exhaustive tests running on release builds:

      $cat data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs_static/logs/backtraces.txt
      CORE: ./core.1512548491.30596.impalad
      BINARY: ./be/build/latest/service/impalad
      Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/bu'.
      Program terminated with signal 6, Aborted.
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      To enable execution of this file add
              add-auto-load-safe-path /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      To completely disable this security protection add
              set auto-load safe-path /
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      For more information about this security protection see the
      "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
              info "(gdb)Auto-loading safe path"
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      #1  0x0000003e902340c5 in abort () from /lib64/libc.so.6
      #2  0x00007f5707ff01a5 in os::abort(bool) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #3  0x00007f5708180843 in VMError::report_and_die() () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #4  0x00007f5707ff5562 in JVM_handle_linux_signal () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #5  0x00007f5707fec4f3 in signalHandler(int, siginfo*, void*) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #6  <signal handler called>
      #7  0x000000000098eff1 in impala::AggregateFunctions::DecimalAvgAddOrRemove (ctx=0x953ba48, src=..., dst=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/aggregate-functions-ir.cc:426
      #8  0x000000000102ad1d in impala::AggFnEvaluator::Update (this=0x1e3fd2c60, row=0x6ea05d20, dst=0x17b90049, fn=0xfffffffffe9ab2ac) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.cc:323
      #9  0x0000000000fd2b5b in Add (this=0x11f42a00, agg_fn_evals=0x6ea05a10, tuple=0x17b90049, row=0x105a34030, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.h:257
      #10 impala::PartitionedAggregationNode::UpdateTuple (this=0x11f42a00, agg_fn_evals=0x6ea05a10, tuple=0x17b90049, row=0x105a34030, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:976
      #11 0x0000000000fde94c in impala::PartitionedAggregationNode::TryAddToHashTable (this=0x11f42a00, ht_ctx=0xaeb8300, partition=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:245
      #12 0x0000000000fdebd5 in impala::PartitionedAggregationNode::ProcessBatchStreaming (this=0x105a34030, needs_serialize=3, prefetch_mode=4271551148, in_batch=0x0, out_batch=0x16f4ff00, ht_ctx=0xaeb8300, remaining_capacity=0x7f5669ba1730) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:189
      #13 0x0000000000fd6730 in impala::PartitionedAggregationNode::GetRowsStreaming (this=0x11f42a00, state=0x7c9d380, out_batch=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:515
      #14 0x0000000000fd7381 in impala::PartitionedAggregationNode::GetNext (this=0x11f42a00, state=0x7c9d380, row_batch=0x16f4ff00, eos=0x7f5669ba185e) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:369
      #15 0x0000000000b71112 in impala::FragmentInstanceState::ExecInternal (this=0xacc894a0) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:270
      #16 0x0000000000b73a82 in impala::FragmentInstanceState::Exec (this=0xacc894a0) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:89
      #17 0x0000000000b636e8 in impala::QueryState::ExecFInstance (this=0x8115600, fis=0xacc894a0) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/query-state.cc:383
      #18 0x0000000000d12243 in operator() (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
      #19 impala::Thread::SuperviseThread (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/util/thread.cc:352
      #20 0x0000000000d12984 in operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (this=0x1516a400) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
      #21 operator() (this=0x1516a400) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
      #22 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (this=0x1516a400) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
      #23 0x000000000128c78a in thread_proxy ()
      #24 0x0000003e90607851 in start_thread () from /lib64/libpthread.so.0
      #25 0x0000003e902e894d in clone () from /lib64/libc.so.6
      
      CORE: ./core.1512548491.30527.impalad
      BINARY: ./be/build/latest/service/impalad
      Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/bu'.
      Program terminated with signal 6, Aborted.
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      To enable execution of this file add
              add-auto-load-safe-path /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      To completely disable this security protection add
              set auto-load safe-path /
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      For more information about this security protection see the
      "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
              info "(gdb)Auto-loading safe path"
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      #1  0x0000003e902340c5 in abort () from /lib64/libc.so.6
      #2  0x00007f23aa4771a5 in os::abort(bool) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #3  0x00007f23aa607843 in VMError::report_and_die() () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #4  0x00007f23aa47c562 in JVM_handle_linux_signal () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #5  0x00007f23aa4734f3 in signalHandler(int, siginfo*, void*) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #6  <signal handler called>
      #7  0x000000000098eff1 in impala::AggregateFunctions::DecimalAvgAddOrRemove (ctx=0xa5a3e88, src=..., dst=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/aggregate-functions-ir.cc:426
      #8  0x000000000102ad1d in impala::AggFnEvaluator::Update (this=0x13b9f320, row=0x12b303a50, dst=0x139ae049, fn=0xfffffffffe9ab2ac) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.cc:323
      #9  0x0000000000fd2b5b in Add (this=0x9c26300, agg_fn_evals=0x12b302ac0, tuple=0x139ae049, row=0x832e010, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.h:257
      #10 impala::PartitionedAggregationNode::UpdateTuple (this=0x9c26300, agg_fn_evals=0x12b302ac0, tuple=0x139ae049, row=0x832e010, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:976
      #11 0x0000000000fde94c in impala::PartitionedAggregationNode::TryAddToHashTable (this=0x9c26300, ht_ctx=0x8ee5480, partition=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:245
      #12 0x0000000000fdebd5 in impala::PartitionedAggregationNode::ProcessBatchStreaming (this=0x832e010, needs_serialize=3, prefetch_mode=4271551148, in_batch=0x0, out_batch=0x15720750, ht_ctx=0x8ee5480, remaining_capacity=0x7f2308446730) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:189
      #13 0x0000000000fd6730 in impala::PartitionedAggregationNode::GetRowsStreaming (this=0x9c26300, state=0x7991600, out_batch=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:515
      #14 0x0000000000fd7381 in impala::PartitionedAggregationNode::GetNext (this=0x9c26300, state=0x7991600, row_batch=0x15720750, eos=0x7f230844685e) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:369
      #15 0x0000000000b71112 in impala::FragmentInstanceState::ExecInternal (this=0xade7a40) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:270
      #16 0x0000000000b73a82 in impala::FragmentInstanceState::Exec (this=0xade7a40) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:89
      #17 0x0000000000b636e8 in impala::QueryState::ExecFInstance (this=0x15d3c000, fis=0xade7a40) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/query-state.cc:383
      #18 0x0000000000d12243 in operator() (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
      #19 impala::Thread::SuperviseThread (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/util/thread.cc:352
      #20 0x0000000000d12984 in operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (this=0x185d2a00) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
      #21 operator() (this=0x185d2a00) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
      #22 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (this=0x185d2a00) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
      #23 0x000000000128c78a in thread_proxy ()
      #24 0x0000003e90607851 in start_thread () from /lib64/libpthread.so.0
      #25 0x0000003e902e894d in clone () from /lib64/libc.so.6
      
      CORE: ./core.1512548491.30514.impalad
      BINARY: ./be/build/latest/service/impalad
      Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/bu'.
      Program terminated with signal 6, Aborted.
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      To enable execution of this file add
              add-auto-load-safe-path /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      To completely disable this security protection add
              set auto-load safe-path /
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      For more information about this security protection see the
      "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
              info "(gdb)Auto-loading safe path"
      #0  0x0000003e902328e5 in raise () from /lib64/libc.so.6
      #1  0x0000003e902340c5 in abort () from /lib64/libc.so.6
      #2  0x00007f773fa481a5 in os::abort(bool) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #3  0x00007f773fbd8843 in VMError::report_and_die() () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #4  0x00007f773fa4d562 in JVM_handle_linux_signal () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #5  0x00007f773fa444f3 in signalHandler(int, siginfo*, void*) () from /opt/toolchain/sun-jdk-64bit-1.8.0.05/jre/lib/amd64/server/libjvm.so
      #6  <signal handler called>
      #7  0x000000000098eff1 in impala::AggregateFunctions::DecimalAvgAddOrRemove (ctx=0x12f0ca560, src=..., dst=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/aggregate-functions-ir.cc:426
      #8  0x000000000102ad1d in impala::AggFnEvaluator::Update (this=0xc3d5120, row=0x15e9fe0a0, dst=0x1490ec049, fn=0xfffffffffe9ab2ac) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.cc:323
      #9  0x0000000000fd2b5b in Add (this=0xc940500, agg_fn_evals=0x15e9fe3e0, tuple=0x1490ec049, row=0xa504008, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exprs/agg-fn-evaluator.h:257
      #10 impala::PartitionedAggregationNode::UpdateTuple (this=0xc940500, agg_fn_evals=0x15e9fe3e0, tuple=0x1490ec049, row=0xa504008, is_merge=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:976
      #11 0x0000000000fde94c in impala::PartitionedAggregationNode::TryAddToHashTable (this=0xc940500, ht_ctx=0x15ffea80, partition=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:245
      #12 0x0000000000fdebd5 in impala::PartitionedAggregationNode::ProcessBatchStreaming (this=0xa504008, needs_serialize=3, prefetch_mode=4271551148, in_batch=0x0, out_batch=0x95bb560, ht_ctx=0x15ffea80, remaining_capacity=0x7f769fec0730) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node-ir.cc:189
      #13 0x0000000000fd6730 in impala::PartitionedAggregationNode::GetRowsStreaming (this=0xc940500, state=0xde3fd900, out_batch=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:515
      #14 0x0000000000fd7381 in impala::PartitionedAggregationNode::GetNext (this=0xc940500, state=0xde3fd900, row_batch=0x95bb560, eos=0x7f769fec085e) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/exec/partitioned-aggregation-node.cc:369
      #15 0x0000000000b71112 in impala::FragmentInstanceState::ExecInternal (this=0xb8d9840) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:270
      #16 0x0000000000b73a82 in impala::FragmentInstanceState::Exec (this=0xb8d9840) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/fragment-instance-state.cc:89
      #17 0x0000000000b636e8 in impala::QueryState::ExecFInstance (this=0xae2b200, fis=0xb8d9840) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/runtime/query-state.cc:383
      #18 0x0000000000d12243 in operator() (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
      #19 impala::Thread::SuperviseThread (name=) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/be/src/util/thread.cc:352
      #20 0x0000000000d12984 in operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (this=0xcb8e000) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
      #21 operator() (this=0xcb8e000) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
      #22 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (this=0xcb8e000) at /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
      #23 0x000000000128c78a in thread_proxy ()
      #24 0x0000003e90607851 in start_thread () from /lib64/libpthread.so.0
      #25 0x0000003e902e894d in clone () from /lib64/libc.so.6
      

      Tim Armstrong suggested that this may be caused by http://gerrit.cloudera.org:8080/8404, hence assignign to Taras Bobrovytsky

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tarasbob Taras Bobrovytsky
                Reporter:
                philip Philip Zeyliger
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: