Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1919

partitioned-hash-join-node-ir.cc:46: Check failed: !out_batch->AtCapacity()

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.1
    • Fix Version/s: Impala 2.3.0
    • Component/s: None
    • Labels:
      None

      Description

      A couple times we have hit this DCHECK in the random stress:

      template<int const JoinOp>
      int PartitionedHashJoinNode::ProcessProbeBatch(
          RowBatch* out_batch, HashTableCtx* ht_ctx, Status* status) {
      <snip>
        DCHECK(!out_batch->AtCapacity());
      <snip>
      

      Reading through the code, this block in PartitionedHashJoinNode::GetNext() looks suspicious to me: out_batch could reach capacity at the same time that output_build_partitions_ becomes empty, though I haven't looked at the core to verify this is the case we've hit.

          if ((join_op_ == TJoinOp::RIGHT_OUTER_JOIN || join_op_ == TJoinOp::RIGHT_ANTI_JOIN ||
               join_op_ == TJoinOp::FULL_OUTER_JOIN) &&
              !output_build_partitions_.empty())  {
            // In case of right-outer, right-anti and full-outer joins, flush the remaining
            // unmatched build rows of any partition we are done processing, before processing
            // the next batch.
            RETURN_IF_ERROR(OutputUnmatchedBuild(out_batch));
            if (!output_build_partitions_.empty()) break;
      
            // Finished to output unmatched build rows, move to next partition.
            DCHECK(hash_partitions_.empty());
            RETURN_IF_ERROR(PrepareNextPartition(state));
            if (input_partition_ == NULL) {
              *eos = true;
              break;
            }
          }
      

        Attachments

          Activity

            People

            • Assignee:
              ippokratis Ippokratis Pandis
              Reporter:
              dhecht Dan Hecht
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: