Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4749

Hit DCHECK in sorter for spilling query with scratch limit

    Details

      Description

      I hit a DCHECK when running the following query on the test minicluster:

      set max_block_mgr_memory=64m;
      set scratch_limit=24m;
      select o_orderdate, o_custkey, o_comment
      from tpch.orders
      order by o_orderdate;
      
      F0109 16:56:25.251039 27202 exec-node.cc:204] Check failed: mem_tracker()->consumption() == 0 (8388608 vs. 0) Leaked memory.
      Fragment 784ce2a2e1ebd300:31263e8000000002: Total=8.00 MB Peak=115.47 MB
        SORT_NODE (id=1): Total=8.00 MB Peak=72.13 MB
        HDFS_SCAN_NODE (id=0): Total=0 Peak=43.49 MB
        CodeGen: Total=4.36 KB Peak=450.00 KB
      *** Check failure stack trace: ***
          @          0x282ae6d  google::LogMessage::Fail()
          @          0x282d796  google::LogMessage::SendToLog()
          @          0x282a98d  google::LogMessage::Flush()
          @          0x282e23e  google::LogMessageFatal::~LogMessageFatal()
          @          0x165e469  impala::ExecNode::Close()
          @          0x17a0ddf  impala::SortNode::Close()
          @          0x19f00a5  impala::PlanFragmentExecutor::Close()
          @          0x19e7b65  impala::FragmentInstanceState::Exec()
          @          0x19f31df  impala::QueryExecMgr::ExecFInstance()
          @          0x19f672a  boost::_mfi::mf1<>::operator()()
          @          0x19f65cd  boost::_bi::list2<>::operator()<>()
          @          0x19f6291  boost::_bi::bind_t<>::operator()()
          @          0x19f5d68  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x134c6ae  boost::function0<>::operator()()
          @          0x15f5d23  impala::Thread::SuperviseThread()
          @          0x15fccfc  boost::_bi::list4<>::operator()<>()
          @          0x15fcc3f  boost::_bi::bind_t<>::operator()()
          @          0x15fcb9a  boost::detail::thread_data<>::run()
          @          0x1a5c45a  thread_proxy
          @     0x7f1ccc53170a  start_thread
          @     0x7f1ccc26782d  (unknown)
      Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,address=30000,server=y,suspend=n 
      Wrote minidump to /home/tarmstrong/Impala/incubator-impala/logs/cluster/minidumps/impalad/13b81910-07d0-3765-4f496d56-3ed53e84.dmp
      

      This only affects debug builds - on a release build the memory should eventually be cleaned up.

        Attachments

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: