Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-26224

HAJobRunOnHadoopS3FileSystemITCase fails with UnsupportedFileSystemSchemeException

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 1.15.0
    • None
    • Connectors / FileSystem
    • None

    Description

      We're observing occassional errors in a Flink fork based on master in HAJobRunOnHadoopS3FileSystemITCase where the Hadoop S3 FileSystem wasn't loaded properly causing the UnsupportedFileSystemSchemeException:

      Feb 16 19:02:52 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.391 s <<< FAILURE! - in org.apache.flink.fs.s3hadoop.HAJobRunOnHadoopS3FileSystemITCase
Feb 16 19:02:52 [ERROR] org.apache.flink.fs.s3hadoop.HAJobRunOnHadoopS3FileSystemITCase.testJobExecutionInHaMode  Time elapsed: 1.89 s  <<< ERROR!
Feb 16 19:02:52 java.io.IOException: Could not create FileSystem for highly available storage path (s3://bucket-cqycdk/test-cluster/default)
Feb 16 19:02:52 	at org.apache.flink.runtime.blob.BlobUtils.createFileSystemBlobStore(BlobUtils.java:102)
Feb 16 19:02:52 	at org.apache.flink.runtime.blob.BlobUtils.createBlobStoreFromConfig(BlobUtils.java:86)
Feb 16 19:02:52 	at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createZooKeeperHaServices(HighAvailabilityServicesUtils.java:87)
Feb 16 19:02:52 	at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createAvailableOrEmbeddedServices(HighAvailabilityServicesUtils.java:70)
Feb 16 19:02:52 	at org.apache.flink.runtime.minicluster.MiniCluster.createHighAvailabilityServices(MiniCluster.java:549)
Feb 16 19:02:52 	at org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:397)
Feb 16 19:02:52 	at org.apache.flink.runtime.testutils.MiniClusterResource.startMiniCluster(MiniClusterResource.java:245)
Feb 16 19:02:52 	at org.apache.flink.runtime.testutils.MiniClusterResource.before(MiniClusterResource.java:105)
Feb 16 19:02:52 	at org.apache.flink.runtime.testutils.MiniClusterExtension.before(MiniClusterExtension.java:56)
Feb 16 19:02:52 	at org.apache.flink.core.testutils.EachCallbackWrapper.beforeEach(EachCallbackWrapper.java:45)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachCallbacks$2(TestMethodTestDescriptor.java:163)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$6(TestMethodTestDescriptor.java:199)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:199)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachCallbacks(TestMethodTestDescriptor.java:162)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:129)
Feb 16 19:02:52 	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
Feb 16 19:02:52 	at java.util.ArrayList.forEach(ArrayList.java:1259)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
Feb 16 19:02:52 	at java.util.ArrayList.forEach(ArrayList.java:1259)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
Feb 16 19:02:52 	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
Feb 16 19:02:52 	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
Feb 16 19:02:52 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:188)
Feb 16 19:02:52 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:154)
Feb 16 19:02:52 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
Feb 16 19:02:52 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
Feb 16 19:02:52 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
Feb 16 19:02:52 	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
Feb 16 19:02:52 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Feb 16 19:02:52 Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 's3'. The scheme is directly supported by Flink through the following plugin: flink-s3-fs-presto. Please ensure that each plugin resides within its own subfolder within the plugins directory. See https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for more information. If you want to use a Hadoop file system for that scheme, please add the scheme to the configuration fs.allowed-fallback-filesystems. For a full list of supported file systems, please see https://nightlies.apache.org/flink/flink-docs-stable/ops/filesystems/.
Feb 16 19:02:52 	at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:514)
Feb 16 19:02:52 	at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:408)
Feb 16 19:02:52 	at org.apache.flink.core.fs.Path.getFileSystem(Path.java:274)
Feb 16 19:02:52 	at org.apache.flink.runtime.blob.BlobUtils.createFileSystemBlobStore(BlobUtils.java:99)
Feb 16 19:02:52 	... 63 more

      Attachments

        Issue Links

          is caused by

          Sub-task - The sub-task of the issue FLINK-26015 FileSystemJobResultStore fails to access Minio

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              mapohl Matthias Pohl
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: