[FLINK-29728] TablePlanner prevents Flink from starting is working directory is a symbolic link

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.2
• Fix Version/s: 1.17.0, 1.16.1
• Component/s: Table SQL / Planner
• Assignee: Weijie Guo
• Reporter: Angelo Kastroulis
• Created: 22/Oct/22 03:04
• Updated: 14/Nov/22 14:41
• Resolved: 31/Oct/22 01:37

Description

The Flink runtime throws an exception when using the table API if the working directory is a symbolic link. This is the case when run on AWS EMR with Yarn. There is a similar issue here and I believe the same fix applied there would work.

 

 

Caused by: org.apache.flink.table.api.TableException: Could not initialize the table planner components loader.
    at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:123) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:52) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule$PlannerComponentsHolder.<clinit>(PlannerModule.java:131) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule.getInstance(PlannerModule.java:135) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.DelegateExecutorFactory.<init>(DelegateExecutorFactory.java:34) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_342]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_342]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_342]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_342]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_342]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_342]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_342]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_342]
    at org.apache.flink.table.factories.ServiceLoaderUtil.load(ServiceLoaderUtil.java:42) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:798) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:517) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.api.internal.TableEnvironmentImpl.create(TableEnvironmentImpl.java:276) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.api.TableEnvironment.create(TableEnvironment.java:93) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at com.ballista.Hermes.BCSE$.useLocalCatalog(BCSE.scala:210) ~[?:?]
    at com.ballista.Hermes.BCSE$.main(BCSE.scala:114) ~[?:?]
    at com.ballista.Hermes.BCSE.main(BCSE.scala) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_342]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_342]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_342]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.15.1.jar:1.15.1]
    ... 7 more
Caused by: java.nio.file.FileAlreadyExistsException: /tmp
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88) ~[?:1.8.0_342]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_342]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_342]
    at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_342]
    at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_342]
    at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_342]
    at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_342]
    at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:96) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:52) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule$PlannerComponentsHolder.<clinit>(PlannerModule.java:131) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.PlannerModule.getInstance(PlannerModule.java:135) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at org.apache.flink.table.planner.loader.DelegateExecutorFactory.<init>(DelegateExecutorFactory.java:34) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_342]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_342]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_342]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_342]
    at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_342]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_342]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_342]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_342]
    at org.apache.flink.table.factories.ServiceLoaderUtil.load(ServiceLoaderUtil.java:42) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:798) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:517) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.api.internal.TableEnvironmentImpl.create(TableEnvironmentImpl.java:276) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at org.apache.flink.table.api.TableEnvironment.create(TableEnvironment.java:93) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
    at com.ballista.Hermes.BCSE$.useLocalCatalog(BCSE.scala:210) ~[?:?]
    at com.ballista.Hermes.BCSE$.main(BCSE.scala:114) ~[?:?]
    at com.ballista.Hermes.BCSE.main(BCSE.scala) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_342]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_342]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_342]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.15.1.jar:1.15.1] 

 

 

Issue Links

• duplicates: FLINK-30001 sql-client.sh start failed
• relates to: FLINK-28102 Flink AkkaRpcSystemLoader fails when temporary directory is a symlink
• links to: GitHub Pull Request #21125