Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-27550

Remove checking yarn queues before submitting job to Yarn

    XMLWordPrintableJSON

Details

    Description

      When invoking the method of checkYarnQueues in YarnClusterDescriptor,
      it will check the specified yarnQueue whether exists in the queues gotten by the YarnClient.QueueInfo.

      However when using the capacity-scheduler, the yarn queues path should be retrieved by the api of QueueInfo.getQueuePath
      instead of getQueueName.

      Due to this, it will always print out the yarn all queues log, but it also can be submitted to Yarn successfully.

      The api of getQueuePath is introduced in the latest hadoop version(https://issues.apache.org/jira/browse/YARN-10658), so it's hard to
      solve this problem in the older hadoop cluster.

      According to the above description, the process of checking is unnecessary.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              zuston Junfan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: