[FLINK-34891] RemoteStorageScannerTest causes exit 239

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.20.0
• Component/s: Runtime / Network
• Labels: test-stability
• Assignee: Unassigned
• Reporter: Ryan Skraba
• Created: 20/Mar/24 09:53
• Updated: 21/Mar/24 10:24

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58432&view=logs&j=f0ac5c25-1168-55a5-07ff-0e88223afed9&t=50bf7a25-bdc4-5e56-5478-c7b4511dde53&l=9121

Mar 20 01:22:54 01:22:54.671 [ERROR] Error occurred in starting fork, check output in log
Mar 20 01:22:54 01:22:54.671 [ERROR] Process Exit Code: 239
Mar 20 01:22:54 01:22:54.671 [ERROR] Crashed tests:
Mar 20 01:22:54 01:22:54.671 [ERROR] org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScannerTest
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:643)
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.lambda$runSuitesForkOnceMultiple$3(ForkStarter.java:350)
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Mar 20 01:22:54 01:22:54.671 [ERROR] 	at java.base/java.lang.Thread.run(Thread.java:829)
Mar 20 01:22:54 01:22:54.671 [ERROR] -> [Help 1]

 

The build artifact mvn-1.log the following FATAL error is found.  It appears that the LocalFileSystem being used by the test can return null for listStatus, causing a NullPointerException in RemoteStorageScanner#scanMaxSegmentId

================================================================================
01:16:09,428 [ForkJoinPool-39-worker-51] INFO  org.apache.flink.util.TestLoggerExtension                    [] -
================================================================================
Test org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScannerTest.testWatchSegmentIgnored[testWatchSegmentIgnored()] is running.
--------------------------------------------------------------------------------
01:16:10,142 [ForkJoinPool-39-worker-51] INFO  org.apache.flink.util.TestLoggerExtension                    [] -
--------------------------------------------------------------------------------
Test org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScannerTest.testWatchSegmentIgnored[testWatchSegmentIgnored()] successfully run.
================================================================================
01:16:10,142 [remote storage scanner-thread-1] ERROR org.apache.flink.util.FatalExitExceptionHandler              [] - FATAL: Thread 'remote storage scanner-thread-1' produced an uncaught exception. Stopping the process...
java.lang.NullPointerException: null
        at org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScanner.scanMaxSegmentId(RemoteStorageScanner.java:247) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScanner.run(RemoteStorageScanner.java:200) [classes/:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
01:16:10,144 [ForkJoinPool-39-worker-51] INFO  org.apache.flink.util.TestLoggerExtension                    [] -
================================================================================
Test org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScannerTest.testScanStrategy[testScanStrategy()] is running.
--------------------------------------------------------------------------------
01:16:10,145 [ForkJoinPool-39-worker-51] INFO  org.apache.flink.util.TestLoggerExtension                    [] -
--------------------------------------------------------------------------------
Test org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteStorageScannerTest.testScanStrategy[testScanStrategy()] successfully run.
================================================================================