[FLINK-27031] ChangelogRescalingITCase.test failed due to IllegalStateException

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.15.0, 1.16.0
• Fix Version/s: 1.16.0, 1.15.2
• Component/s: Runtime / Network, Runtime / State Backends
• Labels: pull-request-available
• Assignee: Anton Kalashnikov
• Reporter: Matthias Pohl
• Created: 03/Apr/22 09:02
• Updated: 12/May/23 11:14
• Resolved: 30/Jun/22 12:09

Description

This build failed in ChangelogRescalingITCase.test:

Apr 01 20:26:53 Caused by: java.lang.IllegalArgumentException: Key group 94 is not in KeyGroupRange{startKeyGroup=96, endKeyGroup=127}. Unless you're directly using low level state access APIs, this is most likely caused by non-deterministic shuffle key (hashCode and equals implementation).
Apr 01 20:26:53 	at org.apache.flink.runtime.state.KeyGroupRangeOffsets.newIllegalKeyGroupException(KeyGroupRangeOffsets.java:37)
Apr 01 20:26:53 	at org.apache.flink.runtime.state.heap.StateTable.getMapForKeyGroup(StateTable.java:305)
Apr 01 20:26:53 	at org.apache.flink.runtime.state.heap.StateTable.get(StateTable.java:261)
Apr 01 20:26:53 	at org.apache.flink.runtime.state.heap.StateTable.get(StateTable.java:143)
Apr 01 20:26:53 	at org.apache.flink.runtime.state.heap.HeapListState.add(HeapListState.java:94)
Apr 01 20:26:53 	at org.apache.flink.state.changelog.ChangelogListState.add(ChangelogListState.java:78)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.processElement(WindowOperator.java:404)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:99)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:80)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39)
Apr 01 20:26:53 	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56)
Apr 01 20:26:53 	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29)
Apr 01 20:26:53 	at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:38)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:531)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:227)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:841)
Apr 01 20:26:53 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:767)
Apr 01 20:26:53 	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948)
Apr 01 20:26:53 	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
Apr 01 20:26:53 	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
Apr 01 20:26:53 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
Apr 01 20:26:53 	at java.lang.Thread.run(Thread.java:748)

Issue Links

• relates to: FLINK-31963 java.lang.ArrayIndexOutOfBoundsException when scaling down with unaligned checkpoints
• links to: GitHub Pull Request #19702