[FLINK-9793] When submitting a flink job with yarn-cluster, flink-dist*.jar is repeatedly uploaded

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.0, 1.5.0
• Fix Version/s: 1.4.3, 1.5.2, 1.6.0
• Component/s: Deployment / YARN
• Labels: patch pull-request-available
• Assignee: linzhongjun
• Reporter: linzhongjun
• Created: 10/Jul/18 14:27
• Updated: 23/Jul/18 19:00
• Resolved: 23/Jul/18 19:00

Description

We are using flink1.4.2 in our company. When the flink job is submitted to run on yarn with yarn-cluster mode，we actually find that the flink-dist_2.11-1.4.2.jar is uploaded to HDFS. The jars lies in different directories e.g.,

1..flink/application_1525941455002_539197/flink-dist_2.11-1.4.2.jar

2..flink/application_1525941455002_539197/lib/flink-dist_2.11-1.4.2.jar

Through reviewing source code of flink and having some tests, we suppose that the code below may have a bug in the Linux environment.

java.nio.file.Path file
!(file.getFileName().startsWith("flink-dist") &&file.getFileName().endsWith("jar")) is Always be true

Issue Links

• links to: GitHub Pull Request #6296