Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-23277

Changelog backend doesn't apply TTL after recovery

    XMLWordPrintableJSON

    Details

      Description

      Upon recovery, changelog backend requests states to apply changes.
      TTL config is not available at this moment, so states are created regardless of TTL config.
      One solution is to serialize TTL config along with metadata (in changelog).

      Note: values are already serialized as TTL values and serializers as TTL seralizers

      Caused by: java.lang.ClassCastException: org.apache.flink.runtime.state.ttl.TtlValue cannot be cast to org.apache.flink.table.data.RowData
         at org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:129)
         at org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:43)
         at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:83)
         at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:228)
         at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
         at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
         at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66)
         at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:428)
         at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:204)
         at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:691)
         at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:646)
         at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:657)
         at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:630)
         at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779)
         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
         at java.lang.Thread.run(Thread.java:748)
      

      (doesn't affect test stability as changelog backend is currently disabled in tests)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                roman Roman Khachatryan
                Reporter:
                roman Roman Khachatryan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: