Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-4995

AssertionError caused by RelFieldTrimmer on SEMI/ANTI join

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      It seems RelFieldTrimmer can cause an AssertionError (or later an ArrayIndexOutOfBoundsException if assertions are disabled) on certain plans involving SEMI/ANTI join (i.e. joins that do NOT project the RHS fields).
      The root cause seems to be the "early return" in RelFieldTrimmer#trimFields(Join join, ImmutableBitSet fieldsUsed, Set<RelDataTypeField> extraFields) when nothing has been trimmed inside join's inputs (so the join itself can be return as it is):

          if (changeCount == 0
              && mapping.isIdentity()) {
            return result(join, Mappings.createIdentity(fieldCount));
          }
      

      The problem is that this fieldCount is an addition of LHS + RHS fields (+ system fields); but in case of a SEMI/ANTI the mappings to be returned must not consider RHS fields (since they are not projected by these join types).

      The problem only happens here (when the trimmer does not trim the join). Notice that, a few lines below, in the "other return scenario" of the method (when something has been trimmed), there is a special treatment of the mapping for ANTI/SEMI, so things will work fine in this case:

          switch (join.getJoinType()) {
          case SEMI:
          case ANTI:
            // For SemiJoins and AntiJoins only map fields from the left-side
            if (join.getJoinType() == JoinRelType.SEMI) {
              relBuilder.semiJoin(newConditionExpr);
            } else {
              relBuilder.antiJoin(newConditionExpr);
            }
            Mapping inputMapping = inputMappings.get(0);
            mapping = Mappings.create(MappingType.INVERSE_SURJECTION,
                join.getRowType().getFieldCount(),
                newSystemFieldCount + inputMapping.getTargetCount());
            for (int i = 0; i < newSystemFieldCount; ++i) {
              mapping.set(i, i);
            }
            offset = systemFieldCount;
            newOffset = newSystemFieldCount;
            for (IntPair pair : inputMapping) {
              mapping.set(pair.source + offset, pair.target + newOffset);
            }
            break;
          default:
            relBuilder.join(join.getJoinType(), newConditionExpr);
          }
          relBuilder.hints(join.getHints());
          return result(relBuilder.build(), mapping);
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rubenql Ruben Q L
            rubenql Ruben Q L
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m

                Slack

                  Issue deployment