Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3670

Block is not deleted by sorter in some cases when spilling is disabled

    Details

      Description

      Start an impala cluster with no scratch:

      ./bin/start-impala-cluster.py --impalad_args=--scratch_dirs=""
      

      Run this query:

      set max_block_mgr_memory=64m;
      select o_orderdate, o_custkey, o_comment
            from tpch.orders
            order by o_orderdate;
      

      It fails with:

      F0603 12:10:04.079107  9045 exec-node.cc:193] Check failed: mem_tracker()->consumption() == 0 (8388608 vs. 0) Leaked memory.
      Fragment 984bd41202d209d7:a8fc5a9ca891d96: Consumption=8.00 MB
        SORT_NODE (id=1): Consumption=8.00 MB
        HDFS_SCAN_NODE (id=0): Consumption=0
      *** Check failure stack trace: ***
          @          0x27d789d  google::LogMessage::Fail()
          @          0x27da1c6  google::LogMessage::SendToLog()
          @          0x27d73bd  google::LogMessage::Flush()
          @          0x27dac6e  google::LogMessageFatal::~LogMessageFatal()
          @          0x162e715  impala::ExecNode::Close()
          @          0x1771407  impala::SortNode::Close()
          @          0x195137a  impala::PlanFragmentExecutor::Close()
          @          0x151a7bc  impala::FragmentMgr::FragmentExecState::Exec()
          @          0x1511f85  impala::FragmentMgr::FragmentThread()
          @          0x1515b32  boost::_mfi::mf1<>::operator()()
          @          0x15158ef  boost::_bi::list2<>::operator()<>()
          @          0x1515219  boost::_bi::bind_t<>::operator()()
          @          0x1514b82  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x1338664  boost::function0<>::operator()()
          @          0x15cfc29  impala::Thread::SuperviseThread()
          @          0x15d672a  boost::_bi::list4<>::operator()<>()
          @          0x15d666d  boost::_bi::bind_t<>::operator()()
          @          0x15d6630  boost::detail::thread_data<>::run()
          @          0x1a082aa  thread_proxy
          @     0x7f4cd26c86aa  start_thread
          @     0x7f4ccf8a2e9d  (unknown)
      

      I did some digging and it looks like sorted_var_len_blocks in UnpinAllBlocks() are not always deleted in the failure case.

      Status Sorter::Run::UnpinAllBlocks() {
      ...
        vector<BufferedBlockMgr::Block*> sorted_var_len_blocks; // These can be leaked!
        ...
                RETURN_IF_ERROR(TryAddBlock(UNPIN_PREV, &sorted_var_len_blocks, &added)); // Leak happens if this fails, because a pinned block is left in the list.
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tarmstrong Tim Armstrong
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: