Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17322

Enable latency tracker would corrupt the broadcast state

    XMLWordPrintableJSON

    Details

      Description

      This bug is reported from user mail list:
      http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Latency-tracking-together-with-broadcast-state-can-cause-job-failure-td34013.html

      Execute BroadcastStateIT#broadcastStateWorksWithLatencyTracking would easily reproduce this problem.

      From current information, the broadcast element would be corrupt once we enable env.getConfig().setLatencyTrackingInterval(2000).
      The exception stack trace would be: (based on current master branch)

      Caused by: java.io.IOException: Corrupt stream, found tag: 84
      	at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:217) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:46) ~[classes/:?]
      	at org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55) ~[classes/:?]
      	at org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:157) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:123) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessor.processInput(StreamTwoInputProcessor.java:181) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:332) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxStep(MailboxProcessor.java:206) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:196) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:505) ~[classes/:?]
      	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:485) ~[classes/:?]
      	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:720) ~[classes/:?]
      	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:544) ~[classes/:?]
      	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_144]
      

        Attachments

          Activity

            People

            • Assignee:
              AHeise Arvid Heise
              Reporter:
              yunta Yun Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: