Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3946

Hit DCHECK in MemPool::CheckIntegrity()

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.7.0
    • Fix Version/s: Impala 2.7.0
    • Component/s: Backend
    • Labels:

      Description

      When running the scanner fuzzing, I hit this DCHECK in the mempool integrity check. The issue seems to be connected with making large allocations. Looking at the code, I think there are at least two separate issues that could lead to this being hit with ReturnPartialAllocation() and in FindChunk() if the memory can't be allocated.

      bool MemPool::CheckIntegrity(bool current_chunk_empty) {
        DCHECK_EQ(zero_length_region_, MEM_POOL_POISON);
      
        // Without pooling, there are way too many chunks and this takes too long.
        if (FLAGS_disable_mem_pools) return true;
      
        // check that current_chunk_idx_ points to the last chunk with allocated data
        DCHECK_LT(current_chunk_idx_, static_cast<int>(chunks_.size()));
        int64_t total_allocated = 0;
        for (int i = 0; i < chunks_.size(); ++i) {
          DCHECK_GT(chunks_[i].size, 0);
          if (i < current_chunk_idx_) {
            DCHECK_GT(chunks_[i].allocated_bytes, 0); <=== HERE
          } else if (i == current_chunk_idx_) {
            if (current_chunk_empty) {
              DCHECK_EQ(chunks_[i].allocated_bytes, 0);
            } else {
              DCHECK_GT(chunks_[i].allocated_bytes, 0);
            }
          } else {
            DCHECK_EQ(chunks_[i].allocated_bytes, 0);
          }
          total_allocated += chunks_[i].allocated_bytes;
        }
        DCHECK_EQ(total_allocated, total_allocated_bytes_);
        return true;
      }
      

        Attachments

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: