[FLINK-28077] Tasks get stuck during cancellation in ChannelStateWriteRequestExecutorImpl

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Runtime / Checkpointing, Tests
• Labels: pull-request-available test-stability
• Assignee: Rui Fan
• Reporter: Matthias Pohl
• Created: 15/Jun/22 09:14
• Updated: 23/Jun/22 12:40
• Resolved: 21/Jun/22 12:24

Description

Build #36209 got stuck in KeyedStateCheckpointingITCase.testWithMemoryBackendSync:

"main" #1 prio=5 os_prio=0 tid=0x00007f849c00b800 nid=0x19c3 waiting on condition [0x00007f84a45b7000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000080074870> (a java.util.concurrent.CompletableFuture$Signaller)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
	at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
	at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1989)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1951)
	at org.apache.flink.test.checkpointing.KeyedStateCheckpointingITCase.testProgramWithBackend(KeyedStateCheckpointingITCase.java:175)
	at org.apache.flink.test.checkpointing.KeyedStateCheckpointingITCase.testWithMemoryBackendSync(KeyedStateCheckpointingITCase.java:104)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[...]

Issue Links

• is a clone of: FLINK-27880 CI runs keep getting stuck
• is caused by: FLINK-27251 Timeout aligned to unaligned checkpoint barrier in the output buffers of an upstream subtask
• is duplicated by: FLINK-28076 StreamFaultToleranceTestBase runs into timeout
• is duplicated by: FLINK-28024 KeyedStateCheckpointingITCase.KeyedStateCheckpointingITCase ends up in infinite failover loop
• is duplicated by: FLINK-27832 SplitAggregateITCase tests failed with Could not acquire the minimum required resources
• relates to: FLINK-27792 InterruptedException thrown by ChannelStateWriterImpl
• supercedes: FLINK-27792 InterruptedException thrown by ChannelStateWriterImpl
• links to: GitHub Pull Request #19993