[FLINK-9769] FileUploads may be shared across requests

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.5.1, 1.6.0
• Fix Version/s: 1.5.1, 1.6.0
• Component/s: Runtime / Coordination, Runtime / REST, Runtime / Web Frontend
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 05/Jul/18 20:02
• Updated: 28/Feb/19 13:30
• Resolved: 06/Jul/18 09:50

Description

Files uploaded by the FileUploadHandler are passed on to subsequent handlers by storing them in a channel attribute.
The files are retrieved from said attribute by the AbstractHandler.

Apparently, since the attribute isn't set to null when retrieving the contained value, it can happen that other handlers still see the value, if the channel is shared across several requests. (This behavior is surprising as i thought that each requests has it's own channel.)
However, the retrieved files will no longer exist for any handler but the original recipient, because he ensures that the files are cleaned up after processing.

Note that this issue existed for a quite a while, it just didn't surface as only a single handler ever accessed these attributes.

2018-07-05 21:55:09,297 ERROR org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Request processing failed.
java.nio.file.NoSuchFileException: C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
	at java.nio.file.Files.readAttributes(Files.java:1737)
	at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.nio.file.Files.walkFileTree(Files.java:2662)
	at java.nio.file.Files.walkFileTree(Files.java:2742)
	at org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
	at org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
	at org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
	at java.lang.Thread.run(Thread.java:745)
2018-07-05 21:55:09,485 ERROR org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Request processing failed.
java.nio.file.NoSuchFileException: C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
	at java.nio.file.Files.readAttributes(Files.java:1737)
	at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.nio.file.Files.walkFileTree(Files.java:2662)
	at java.nio.file.Files.walkFileTree(Files.java:2742)
	at org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
	at org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
	at org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
	at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
	at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
	at java.lang.Thread.run(Thread.java:745)

Issue Links

• links to: GitHub Pull Request #6270