Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1836

Crash at PAGG::Partition::Close() in low mem situation

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.0, Impala 2.1
    • Fix Version/s: Impala 2.2
    • Component/s: None
    • Labels:

      Description

      After fixing some of the low mem / spilling issues, the simple low mem stress test found another problem: In the infrequent case where at Partition::hash_tbl::Init() couldn't consume 8KB to allocate the buckets array it would return false. But then, PAGG::Partition::Close() would try to clean up the hash table iterating the buckets. even though it didn't have any buckets. That would cause a dcheck() in HT::Begin().

      The HashTable::Init() code would return set num_buckets_ == 0 and return false if it couldn't consume 8KB.

      bool HashTable::Init() {
        int64_t buckets_byte_size = num_buckets_ * sizeof(Bucket);
        if (block_mgr_client_ != NULL &&
            !state_->block_mgr()->ConsumeMemory(block_mgr_client_, buckets_byte_size)) {
          num_buckets_ = 0; <-- couldn't consume 8KB, set num_buckets_ == 0.
          return false;
        }
        buckets_ = reinterpret_cast<Bucket*>(malloc(buckets_byte_size));
        memset(buckets_, 0, buckets_byte_size);
        return GrowNodeArray();
      }
      

      The PAGG::Partition::Close() would try to iterate the hash_tbl even though it didn't have any buckets.

      void PartitionedAggregationNode::Partition::Close(bool finalize_rows) {
        if (is_closed) return;
        is_closed = true;
        if (aggregated_row_stream.get() != NULL) {
          if (finalize_rows && hash_tbl.get() != NULL) {
            // We need to walk all the rows and Finalize them here so the UDA gets a chance
            // to cleanup. If the hash table is gone (meaning this was spilled), the rows
            // should have been finalized/serialized in Spill().
            parent->CleanupHashTbl(agg_fn_ctxs, hash_tbl->Begin(parent->ht_ctx_.get())); <--- the hash_tbl->Begin() call will DCHECK if num_buckets_ == 0
          }
          aggregated_row_stream->Close();
        }
      

        Attachments

          Activity

            People

            • Assignee:
              ippokratis Ippokratis Pandis
              Reporter:
              ippokratis Ippokratis Pandis
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: