Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9793

When submitting a flink job with yarn-cluster, flink-dist*.jar is repeatedly uploaded

    XMLWordPrintableJSON

    Details

      Description

      We are using flink1.4.2 in our company. When the flink job is submitted to run on yarn with yarn-cluster mode,we actually find that the flink-dist_2.11-1.4.2.jar is uploaded to HDFS. The jars lies in different directories e.g.,

      1..flink/application_1525941455002_539197/flink-dist_2.11-1.4.2.jar

      2..flink/application_1525941455002_539197/lib/flink-dist_2.11-1.4.2.jar

      Through reviewing source code of flink and having some tests, we suppose that the code below may have a bug in the Linux environment.

      java.nio.file.Path file
      !(file.getFileName().startsWith("flink-dist") &&file.getFileName().endsWith("jar")) is Always be true

        Attachments

        1. 20180710231518.png
          59 kB
          linzhongjun
        2. 20180710224848.png
          71 kB
          linzhongjun
        3. 20180710224808.png
          95 kB
          linzhongjun

          Issue Links

            Activity

              People

              • Assignee:
                linjun linzhongjun
                Reporter:
                linjun linzhongjun
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: