[IMPALA-1919] partitioned-hash-join-node-ir.cc:46: Check failed: !out_batch->AtCapacity()

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.1
• Fix Version/s: Impala 2.3.0
• Assignee: Ippokratis Pandis
• Reporter: Daniel Hecht
• Created: 25/Mar/15 20:48
• Updated: 05/May/15 20:35
• Resolved: 04/May/15 19:59

Description

A couple times we have hit this DCHECK in the random stress:

template<int const JoinOp>
int PartitionedHashJoinNode::ProcessProbeBatch(
    RowBatch* out_batch, HashTableCtx* ht_ctx, Status* status) {
<snip>
  DCHECK(!out_batch->AtCapacity());
<snip>

Reading through the code, this block in PartitionedHashJoinNode::GetNext() looks suspicious to me: out_batch could reach capacity at the same time that output_build_partitions_ becomes empty, though I haven't looked at the core to verify this is the case we've hit.

    if ((join_op_ == TJoinOp::RIGHT_OUTER_JOIN || join_op_ == TJoinOp::RIGHT_ANTI_JOIN ||
         join_op_ == TJoinOp::FULL_OUTER_JOIN) &&
        !output_build_partitions_.empty())  {
      // In case of right-outer, right-anti and full-outer joins, flush the remaining
      // unmatched build rows of any partition we are done processing, before processing
      // the next batch.
      RETURN_IF_ERROR(OutputUnmatchedBuild(out_batch));
      if (!output_build_partitions_.empty()) break;

      // Finished to output unmatched build rows, move to next partition.
      DCHECK(hash_partitions_.empty());
      RETURN_IF_ERROR(PrepareNextPartition(state));
      if (input_partition_ == NULL) {
        *eos = true;
        break;
      }
    }