Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2099

BufferedBlockMgr DCHECK client->num_tmp_reserved_buffers_ == 0 failure w/ RM enabled

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.2
    • Fix Version/s: None
    • Component/s: None

      Description

      On a 6 node cluster with RM (Yarn+Llama) enabled, running tpcds q46 results in a DCHECK failure. The query completes fine in the same cluster with RM disabled. (profile attached).

      F0624 14:10:18.447474  2245 buffered-block-mgr.cc:259] Check failed: client->num_tmp_reserved_buffers_ == 0 (3 vs. 0) 
      

      The core dump shows the following stack (missing symbols so not much else):

      …
      #6  0x0000000001fd3ecd in google::LogMessageFatal::~LogMessageFatal() ()
      #7  0x00000000011d72da in impala::BufferedBlockMgr::TryAcquireTmpReservation(impala::BufferedBlockMgr::Client*, int) ()
      #8  0x0000000001519597 in impala::BufferedTupleStream::PinStream(bool, bool*) ()
      #9  0x000000000160f65b in impala::Status impala::PartitionedHashJoinNode::Partition::BuildHashTableInternal<false>(impala::RuntimeState*, bool*) ()
      #10 0x0000000001601c92 in impala::PartitionedHashJoinNode::Partition::BuildHashTable(impala::RuntimeState*, bool*, bool) ()
      #11 0x0000000001608a68 in impala::PartitionedHashJoinNode::BuildHashTables(impala::RuntimeState*) ()
      #12 0x000000000160402f in impala::PartitionedHashJoinNode::ProcessBuildInput(impala::RuntimeState*, int) ()
      #13 0x0000000001602cc6 in impala::PartitionedHashJoinNode::ConstructBuildSide(impala::RuntimeState*) ()
      #14 0x00000000016412f5 in impala::BlockingJoinNode::BuildSideThread(impala::RuntimeState*, impala::Promise<impala::Status>*) ()
      #15 0x0000000001642ff2 in boost::_mfi::mf2<void, impala::BlockingJoinNode, impala::RuntimeState*, impala::Promise<impala::Status>*>::operator()(impala::BlockingJoinNode*, impala::RuntimeState*, impala::Promise<impala::Status>*) const ()
      …
      

        Attachments

        1. profile_q46_no-RM_success.txt
          227 kB
          Matthew Jacobs
        2. q46-debug-crash.zip
          4.82 MB
          Matthew Jacobs

          Issue Links

            Activity

              People

              • Assignee:
                mjacobs Matthew Jacobs
                Reporter:
                mjacobs Matthew Jacobs
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: