Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4749

Hit DCHECK in sorter for spilling query with scratch limit

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      I hit a DCHECK when running the following query on the test minicluster:

      set max_block_mgr_memory=64m;
      set scratch_limit=24m;
      select o_orderdate, o_custkey, o_comment
      from tpch.orders
      order by o_orderdate;
      
      F0109 16:56:25.251039 27202 exec-node.cc:204] Check failed: mem_tracker()->consumption() == 0 (8388608 vs. 0) Leaked memory.
      Fragment 784ce2a2e1ebd300:31263e8000000002: Total=8.00 MB Peak=115.47 MB
        SORT_NODE (id=1): Total=8.00 MB Peak=72.13 MB
        HDFS_SCAN_NODE (id=0): Total=0 Peak=43.49 MB
        CodeGen: Total=4.36 KB Peak=450.00 KB
      *** Check failure stack trace: ***
          @          0x282ae6d  google::LogMessage::Fail()
          @          0x282d796  google::LogMessage::SendToLog()
          @          0x282a98d  google::LogMessage::Flush()
          @          0x282e23e  google::LogMessageFatal::~LogMessageFatal()
          @          0x165e469  impala::ExecNode::Close()
          @          0x17a0ddf  impala::SortNode::Close()
          @          0x19f00a5  impala::PlanFragmentExecutor::Close()
          @          0x19e7b65  impala::FragmentInstanceState::Exec()
          @          0x19f31df  impala::QueryExecMgr::ExecFInstance()
          @          0x19f672a  boost::_mfi::mf1<>::operator()()
          @          0x19f65cd  boost::_bi::list2<>::operator()<>()
          @          0x19f6291  boost::_bi::bind_t<>::operator()()
          @          0x19f5d68  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x134c6ae  boost::function0<>::operator()()
          @          0x15f5d23  impala::Thread::SuperviseThread()
          @          0x15fccfc  boost::_bi::list4<>::operator()<>()
          @          0x15fcc3f  boost::_bi::bind_t<>::operator()()
          @          0x15fcb9a  boost::detail::thread_data<>::run()
          @          0x1a5c45a  thread_proxy
          @     0x7f1ccc53170a  start_thread
          @     0x7f1ccc26782d  (unknown)
      Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,address=30000,server=y,suspend=n 
      Wrote minidump to /home/tarmstrong/Impala/incubator-impala/logs/cluster/minidumps/impalad/13b81910-07d0-3765-4f496d56-3ed53e84.dmp
      

      This only affects debug builds - on a release build the memory should eventually be cleaned up.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tarmstrong Tim Armstrong
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment