Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16047

Blink planner produces wrong aggregate results with state clean up

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      It seems that FLINK-10674 has not been ported to the Blink planner.

      Because state clean up happens in processing time, it might be the case that retractions are arriving after the state has been cleaned up. Before these changes, a new accumulator was created and invalid retraction messages were emitted. This change drops retraction messages for which no accumulator exists.

      These lines are missing in org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction:

      if (null == accumulators) {
            // Don't create a new accumulator for a retraction message. This
            // might happen if the retraction message is the first message for the
            // key or after a state clean up.
            if (!inputC.change) {
              return
            }
            // first accumulate message
            firstRow = true
            accumulators = function.createAccumulators()
          } else {
            firstRow = false
          }
      

      The bug has not been verified. I spotted it only by looking at the code.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jark Jark Wu
            twalthr Timo Walther
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment