Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-29728

TablePlanner prevents Flink from starting is working directory is a symbolic link

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.15.2
    • 1.17.0, 1.16.1
    • Table SQL / Planner
    • None

    Description

      The Flink runtime throws an exception when using the table API if the working directory is a symbolic link. This is the case when run on AWS EMR with Yarn. There is a similar issue here and I believe the same fix applied there would work.

       

       

      Caused by: org.apache.flink.table.api.TableException: Could not initialize the table planner components loader.
          at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:123) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:52) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule$PlannerComponentsHolder.<clinit>(PlannerModule.java:131) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule.getInstance(PlannerModule.java:135) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.DelegateExecutorFactory.<init>(DelegateExecutorFactory.java:34) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_342]
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_342]
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_342]
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_342]
          at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_342]
          at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_342]
          at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_342]
          at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_342]
          at org.apache.flink.table.factories.ServiceLoaderUtil.load(ServiceLoaderUtil.java:42) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:798) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:517) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.api.internal.TableEnvironmentImpl.create(TableEnvironmentImpl.java:276) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.api.TableEnvironment.create(TableEnvironment.java:93) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at com.ballista.Hermes.BCSE$.useLocalCatalog(BCSE.scala:210) ~[?:?]
          at com.ballista.Hermes.BCSE$.main(BCSE.scala:114) ~[?:?]
          at com.ballista.Hermes.BCSE.main(BCSE.scala) ~[?:?]
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_342]
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_342]
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_342]
          at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
          at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.15.1.jar:1.15.1]
          ... 7 more
      Caused by: java.nio.file.FileAlreadyExistsException: /tmp
          at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88) ~[?:1.8.0_342]
          at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_342]
          at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_342]
          at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_342]
          at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_342]
          at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_342]
          at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_342]
          at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:96) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule.<init>(PlannerModule.java:52) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule$PlannerComponentsHolder.<clinit>(PlannerModule.java:131) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.PlannerModule.getInstance(PlannerModule.java:135) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at org.apache.flink.table.planner.loader.DelegateExecutorFactory.<init>(DelegateExecutorFactory.java:34) ~[flink-table-planner-loader-1.15.1.jar:1.15.1]
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_342]
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_342]
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_342]
          at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_342]
          at java.lang.Class.newInstance(Class.java:442) ~[?:1.8.0_342]
          at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[?:1.8.0_342]
          at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[?:1.8.0_342]
          at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[?:1.8.0_342]
          at org.apache.flink.table.factories.ServiceLoaderUtil.load(ServiceLoaderUtil.java:42) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.factories.FactoryUtil.discoverFactories(FactoryUtil.java:798) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.factories.FactoryUtil.discoverFactory(FactoryUtil.java:517) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.api.internal.TableEnvironmentImpl.create(TableEnvironmentImpl.java:276) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at org.apache.flink.table.api.TableEnvironment.create(TableEnvironment.java:93) ~[flink-table-api-java-uber-1.15.1.jar:1.15.1]
          at com.ballista.Hermes.BCSE$.useLocalCatalog(BCSE.scala:210) ~[?:?]
          at com.ballista.Hermes.BCSE$.main(BCSE.scala:114) ~[?:?]
          at com.ballista.Hermes.BCSE.main(BCSE.scala) ~[?:?]
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_342]
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_342]
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_342]
          at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
          at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist-1.15.1.jar:1.15.1] 

       

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Weijie Guo Weijie Guo
            angelok Angelo Kastroulis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment