Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-23276

Changelog backend not always updates delegating functions

    XMLWordPrintableJSON

    Details

      Description

      (currently disabled in tests, so doesn't affect build stability)

      E.g. org.apache.flink.table.planner.runtime.stream.table.GroupWindowITCase, or other tests from the same package.

       2021-07-03T20:30:28.0384912Z Jul 03 20:30:28 Caused by: java.lang.NullPointerException
      *2021-07-03T20:30:28.0385566Z Jul 03 20:30:28    at org.apache.flink.util.Preconditions.checkNotNull(Preconditions.java:59)
      *2021-07-03T20:30:28.0386330Z Jul 03 20:30:28    at org.apache.flink.state.changelog.restore.FunctionDelegationHelper$DelegatingReduceFunction.redu*ce(FunctionDelegationHelper.java:138)
      *2021-07-03T20:30:28.0387147Z Jul 03 20:30:28    at org.apache.flink.contrib.streaming.state.RocksDBReducingState.add(RocksDBReducingState.java:95)
      *2021-07-03T20:30:28.0387892Z Jul 03 20:30:28    at org.apache.flink.state.changelog.ChangelogReducingState.add(ChangelogReducingState.java:82)
      *2021-07-03T20:30:28.0388677Z Jul 03 20:30:28    at org.apache.flink.table.runtime.operators.window.triggers.ElementTriggers$CountElement.onElement*(ElementTriggers.java:124)
      *2021-07-03T20:30:28.0389503Z Jul 03 20:30:28    at org.apache.flink.table.runtime.operators.window.WindowOperator$TriggerContext.onElement(WindowO*perator.java:572)
      *2021-07-03T20:30:28.0390296Z Jul 03 20:30:28    at org.apache.flink.table.runtime.operators.window.WindowOperator.processElement(WindowOperator.ja*va:379)
      *2021-07-03T20:30:28.0391107Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(*OneInputStreamTask.java:228)
      *2021-07-03T20:30:28.0391936Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStr*eamTaskNetworkInput.java:134)
      *2021-07-03T20:30:28.0392776Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTas*kNetworkInput.java:105)
      *2021-07-03T20:30:28.0393929Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProces*sor.java:66)
      *2021-07-03T20:30:28.0394611Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:428)
      *2021-07-03T20:30:28.0395288Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcess*or.java:204)
      *2021-07-03T20:30:28.0395966Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:691)
      *2021-07-03T20:30:28.0396892Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:646)
      *2021-07-03T20:30:28.0397601Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:657)
      *2021-07-03T20:30:28.0398398Z Jul 03 20:30:28    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:630)
      *2021-07-03T20:30:28.0399051Z Jul 03 20:30:28    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779)
      *2021-07-03T20:30:28.0399669Z Jul 03 20:30:28    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
       2021-07-03T20:30:28.0400238Z Jul 03 20:30:28    at java.lang.Thread.run(Thread.java:748)
      
      

      The reason is missing functionDelegationHelper.addOrUpdate() call in
      ChangelogKeyedStateBackend.getPartitionedState().

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                roman Roman Khachatryan
                Reporter:
                roman Roman Khachatryan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: