Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11136

Fix the logical of merge for DISTINCT aggregates

    XMLWordPrintableJSON

Details

    Description

      The logic of merge for DISTINCT aggregates has bug. For the following query:

      SELECT
        c,
        COUNT(DISTINCT b),
        SUM(DISTINCT b),
        SESSION_END(rowtime, INTERVAL '0.005' SECOND)
      FROM MyTable
      GROUP BY SESSION(rowtime, INTERVAL '0.005' SECOND), c

      the following exception will be thrown:

      Caused by: java.lang.ClassCastException: org.apache.flink.types.Row cannot be cast to java.lang.Integer
      at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
      at scala.math.Numeric$IntIsIntegral$.plus(Numeric.scala:58)
      at org.apache.flink.table.functions.aggfunctions.SumAggFunction.accumulate(SumAggFunction.scala:50)
      at GroupingWindowAggregateHelper$18.mergeAccumulatorsPair(Unknown Source)
      at org.apache.flink.table.runtime.aggregate.AggregateAggFunction.merge(AggregateAggFunction.scala:66)
      at org.apache.flink.table.runtime.aggregate.AggregateAggFunction.merge(AggregateAggFunction.scala:33)
      at org.apache.flink.runtime.state.heap.HeapAggregatingState.mergeState(HeapAggregatingState.java:117)
      at org.apache.flink.runtime.state.heap.AbstractHeapMergingState$MergeTransformation.apply(AbstractHeapMergingState.java:102)
      at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.transform(CopyOnWriteStateTable.java:463)
      at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.transform(CopyOnWriteStateTable.java:341)
      at org.apache.flink.runtime.state.heap.AbstractHeapMergingState.mergeNamespaces(AbstractHeapMergingState.java:91)
      at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator$2.merge(WindowOperator.java:341)
      at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator$2.merge(WindowOperator.java:311)
      at org.apache.flink.streaming.runtime.operators.windowing.MergingWindowSet.addWindow(MergingWindowSet.java:212)
      at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.processElement(WindowOperator.java:311)
      at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
      at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
      at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
      at java.lang.Thread.run(Thread.java:745)
      

      Attachments

        Issue Links

          Activity

            People

              dian.fu Dian Fu
              dian.fu Dian Fu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m