[FLINK-25139] S3 tests failing due to rotating credentials

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.13.3, 1.15.0
• Component/s: FileSystems
• Labels: test-stability
• Assignee: Unassigned
• Reporter: Yun Gao
• Created: 02/Dec/21 03:09
• Updated: 02/Dec/21 07:46
• Resolved: 02/Dec/21 07:20

Description

Dec 01 19:33:39 	at org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:1232) ~[?:?]
Dec 01 19:33:39 	at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2169) ~[?:?]
Dec 01 19:33:39 	at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2149) ~[?:?]
Dec 01 19:33:39 	at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088) ~[?:?]
Dec 01 19:33:39 	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) ~[?:?]
Dec 01 19:33:39 	at org.apache.hadoop.fs.s3a.S3AFileSystem.exists(S3AFileSystem.java:2970) ~[?:?]
Dec 01 19:33:39 	at org.apache.flink.fs.s3hadoop.common.HadoopFileSystem.exists(HadoopFileSystem.java:165) ~[?:?]
Dec 01 19:33:39 	at org.apache.flink.core.fs.PluginFileSystemFactory$ClassLoaderFixingFileSystem.exists(PluginFileSystemFactory.java:148) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.core.fs.FileSystem.initOutPathDistFS(FileSystem.java:984) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.api.common.io.FileOutputFormat.initializeGlobal(FileOutputFormat.java:299) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:110) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:174) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:107) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:342) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:190) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:122) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:132) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:110) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:340) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:317) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:107) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Dec 01 19:33:39 	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_292]
Dec 01 19:33:39 	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292]
 

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27379&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529&l=23278

Issue Links

• is related to: FLINK-25140 Streaming File Sink s3 end-to-end test failed due to job has not started within a timeout of 10 sec