Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-26047

Support usrlib in HDFS for YARN application mode

    XMLWordPrintableJSON

Details

    Description

      In YARN Application mode, we currently support using user jar and lib jar from HDFS. For example, we can run commands like:

      ./bin/flink run-application -t yarn-application \
      -Dyarn.provided.lib.dirs="hdfs://myhdfs/my-remote-flink-dist-dir" \
      hdfs://myhdfs/jars/my-application.jar

      For usrlib, we currently only support local directory. I propose to add HDFS support for usrlib to work with CLASSPATH_INCLUDE_USER_JAR better. It can also benefit cases like using notebook to submit flink job.

      Attachments

        Issue Links

          Activity

            People

              bgeng777 Biao Geng
              bgeng777 Biao Geng
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: