Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5386

disk-io-mgr-handle-cache.inline.h:124] Check failed: release_it != range.second

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Backend
    • Labels:

      Description

      Running latest bit on asf-master hits the following DCHECK on Isilon hardware. Joe McDonnell, can you please take a look ?

      F0529 10:00:13.853025 15540 disk-io-mgr-handle-cache.inline.h:124] Check failed: release_it != range.second
      
      CORE: ./core.1496077215.15169.impalad
      BINARY: ./be/build/latest/service/impalad
      Core was generated by `/data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/b'.
      Program terminated with signal 6, Aborted.
      #0  0x0000003069e328e5 in raise () from /lib64/libc.so.6
      To enable execution of this file add
              add-auto-load-safe-path /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/gcc-4.9.2/lib64/libstdc++.so.6.0.20-gdb.py
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      To completely disable this security protection add
              set auto-load safe-path /
      line to your configuration file "/var/lib/jenkins/.gdbinit".
      For more information about this security protection see the
      "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
              info "(gdb)Auto-loading safe path"
      #0  0x0000003069e328e5 in raise () from /lib64/libc.so.6
      #1  0x0000003069e340c5 in abort () from /lib64/libc.so.6
      #2  0x0000000002946f24 in google::DumpStackTraceAndExit() ()
      #3  0x000000000293d99d in google::LogMessage::Fail() ()
      #4  0x000000000293f242 in google::LogMessage::SendToLog() ()
      #5  0x000000000293d377 in google::LogMessage::Flush() ()
      #6  0x000000000294093e in google::LogMessageFatal::~LogMessageFatal() ()
      #7  0x000000000143ea49 in impala::FileHandleCache<16ul>::ReleaseFileHandle (this=0x7ef6b40, fname=0xa8cf9a8, fh=0xa758200, destroy_handle=false) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr-handle-cache.inline.h:124
      #8  0x000000000143ad1c in impala::DiskIoMgr::ReleaseCachedHdfsFileHandle (this=0x7ef6a00, fname=0xa8cf9a8, fid=0xa758200) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr.cc:1266
      #9  0x000000000144dcbc in impala::DiskIoMgr::ScanRange::Close (this=0xa8cf980) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr-scan-range.cc:352
      #10 0x000000000144bca3 in impala::DiskIoMgr::ScanRange::Cancel (this=0xa8cf980, status=...) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr-scan-range.cc:169
      #11 0x0000000001438ac3 in impala::DiskIoMgr::HandleReadFinished (this=0x7ef6a00, disk_queue=0x8d8cf80, reader=0xa01a3c0, buffer=0xf288df0) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr.cc:1000
      #12 0x0000000001439478 in impala::DiskIoMgr::ReadRange (this=0x7ef6a00, disk_queue=0x8d8cf80, reader=0xa01a3c0, range=0xa8cf980) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr.cc:1105
      #13 0x0000000001438e05 in impala::DiskIoMgr::WorkLoop (this=0x7ef6a00, disk_queue=0x8d8cf80) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/runtime/disk-io-mgr.cc:1047
      #14 0x00000000014451be in boost::_mfi::mf1<void, impala::DiskIoMgr, impala::DiskIoMgr::DiskQueue*>::operator() (this=0xa034380, p=0x7ef6a00, a1=0x8d8cf80) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/mem_fn_template.hpp:165
      #15 0x0000000001444c61 in boost::_bi::list2<boost::_bi::value<impala::DiskIoMgr*>, boost::_bi::value<impala::DiskIoMgr::DiskQueue*> >::operator()<boost::_mfi::mf1<void, impala::DiskIoMgr, impala::DiskIoMgr::DiskQueue*>, boost::_bi::list0> (this=0xa034390, f=..., a=...) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:313
      #16 0x0000000001444409 in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::DiskIoMgr, impala::DiskIoMgr::DiskQueue*>, boost::_bi::list2<boost::_bi::value<impala::DiskIoMgr*>, boost::_bi::value<impala::DiskIoMgr::DiskQueue*> > >::operator() (this=0xa034380) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
      #17 0x00000000014438c8 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::DiskIoMgr, impala::DiskIoMgr::DiskQueue*>, boost::_bi::list2<boost::_bi::value<impala::DiskIoMgr*>, boost::_bi::value<impala::DiskIoMgr::DiskQueue*> > >, void>::invoke (function_obj_ptr=...) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:153
      #18 0x00000000013ba85c in boost::function0<void>::operator() (this=0x7ffc087eec20) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/function/function_template.hpp:767
      #19 0x00000000016862bf in impala::Thread::SuperviseThread (name=..., category=..., functor=..., thread_started=0x7fff2e6ebc10) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/be/src/util/thread.cc:325
      #20 0x000000000168ec42 in boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> >::operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> > &, const std::basic_string<char, std::char_traits<char>, std::allocator<char> > &, boost::function<void()>, impala::Promise<long> *), boost::_bi::list0 &, int) (this=0x8a7a3c0, f=@0x8a7a3b8, a=...) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind.hpp:457
      #21 0x000000000168eb85 in boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > >::operator()(void) (this=0x8a7a3b8) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/bind/bind_template.hpp:20
      #22 0x000000000168eb48 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> > > >::run(void) (this=0x8a7a200) at /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/Impala-Toolchain/boost-1.57.0-p1/include/boost/thread/detail/thread.hpp:116
      #23 0x0000000001b6e38a in thread_proxy ()
      #24 0x000000306a207851 in start_thread () from /lib64/libpthread.so.0
      #25 0x0000003069ee894d in clone () from /lib64/libc.so.6
      

        Attachments

          Activity

            People

            • Assignee:
              joemcdonnell Joe McDonnell
              Reporter:
              kwho Michael Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: