[FLINK-33323] HybridShuffleITCase fails with produced an uncaught exception in FatalExitExceptionHandler

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.19.0
• Component/s: Runtime / Network
• Labels: pull-request-available test-stability
• Assignee: Wencong Liu
• Reporter: Sergey Nuyanzin
• Created: 20/Oct/23 07:57
• Updated: 09/Nov/23 15:49
• Resolved: 09/Nov/23 15:46

Description

This build https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53853&view=logs&j=a596f69e-60d2-5a4b-7d39-dc69e4cdaed3&t=712ade8c-ca16-5b76-3acd-14df33bc1cb1&l=9166
fails with

01:15:38,516 [blocking-shuffle-io-thread-4] ERROR org.apache.flink.util.FatalExitExceptionHandler              [] - FATAL: Thread 'blocking-shuffle-io-thread-4' produced an uncaught exception. Stopping the process...
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@4275bb45[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@488dd035[Wrapped task = org.apache.fl
ink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler$$Lambda$2561/0x0000000801a2f728@464a3754]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@22747816[Shutting down, pool size = 10, active threads = 9,
 queued tasks = 1, completed tasks = 1]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) ~[?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) ~[?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) ~[?:?]
        at org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler.run(DiskIOScheduler.java:151) ~[flink-runtime-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
        at org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.disk.DiskIOScheduler.lambda$triggerScheduling$0(DiskIOScheduler.java:308) ~[flink-runtime-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]

also logs are attached

Issue Links

• is related to: FLINK-33502 HybridShuffleITCase caused a fatal error
• links to: GitHub Pull Request #23569