Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9769

FileUploads may be shared across requests

    XMLWordPrintableJSON

    Details

      Description

      Files uploaded by the FileUploadHandler are passed on to subsequent handlers by storing them in a channel attribute.
      The files are retrieved from said attribute by the AbstractHandler.

      Apparently, since the attribute isn't set to null when retrieving the contained value, it can happen that other handlers still see the value, if the channel is shared across several requests. (This behavior is surprising as i thought that each requests has it's own channel.)
      However, the retrieved files will no longer exist for any handler but the original recipient, because he ensures that the files are cleaned up after processing.

      Note that this issue existed for a quite a while, it just didn't surface as only a single handler ever accessed these attributes.

      
      2018-07-05 21:55:09,297 ERROR org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Request processing failed.
      java.nio.file.NoSuchFileException: C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
      	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
      	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
      	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
      	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
      	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
      	at sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
      	at java.nio.file.Files.readAttributes(Files.java:1737)
      	at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
      	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
      	at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
      	at java.nio.file.Files.walkFileTree(Files.java:2662)
      	at java.nio.file.Files.walkFileTree(Files.java:2742)
      	at org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
      	at org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
      	at org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
      	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
      	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
      	at java.lang.Thread.run(Thread.java:745)
      2018-07-05 21:55:09,485 ERROR org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Request processing failed.
      java.nio.file.NoSuchFileException: C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
      	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
      	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
      	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
      	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
      	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
      	at sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
      	at java.nio.file.Files.readAttributes(Files.java:1737)
      	at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
      	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
      	at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
      	at java.nio.file.Files.walkFileTree(Files.java:2662)
      	at java.nio.file.Files.walkFileTree(Files.java:2742)
      	at org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
      	at org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
      	at org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
      	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
      	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
      	at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chesnay Chesnay Schepler
                Reporter:
                chesnay Chesnay Schepler
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: