Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9793

When submitting a flink job with yarn-cluster, flink-dist*.jar is repeatedly uploaded

    XMLWordPrintableJSON

Details

    Description

      We are using flink1.4.2 in our company. When the flink job is submitted to run on yarn with yarn-cluster mode,we actually find that the flink-dist_2.11-1.4.2.jar is uploaded to HDFS. The jars lies in different directories e.g.,

      1..flink/application_1525941455002_539197/flink-dist_2.11-1.4.2.jar

      2..flink/application_1525941455002_539197/lib/flink-dist_2.11-1.4.2.jar

      Through reviewing source code of flink and having some tests, we suppose that the code below may have a bug in the Linux environment.

      java.nio.file.Path file
      !(file.getFileName().startsWith("flink-dist") &&file.getFileName().endsWith("jar")) is Always be true

      Attachments

        1. 20180710224808.png
          95 kB
          linzhongjun
        2. 20180710224848.png
          71 kB
          linzhongjun
        3. 20180710231518.png
          59 kB
          linzhongjun

        Issue Links

          Activity

            People

              linjun linzhongjun
              linjun linzhongjun
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: