Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-18362

Shipping jdk by shipping archive

    XMLWordPrintableJSON

    Details

      Description

      Hello,
      Our team are running flink cluster on YARN, and it works so well 👍

      Functional requirements

      Is there any option to ship archive to YARN applications?

      Backgrounds

      Recently, one of job has been shut down with jdk8 version related issues.
      https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6675699

      It's easy problem if we could set latest jdk on `containerized.taskmanager.env.JAVA_HOME`.
      However, cluster administrator said it's difficult to install the latest jdk on all cluster machines.

      So, we planned to run a job on latest jdk that is shipped via shared resources.
      There's an option `yarn.ship-directories` but it's quite slow because jdk has large number of files.

      If Flink supports to ship archive such as `yarn.ship-archive`, we can ship jdk archive to remote machines and use shipped jdk location as JAVA_HOME (using `yarn.container-start-command-template` )

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gramo gramo lee
                Reporter:
                nobe0716 Noah
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: