Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-25410

Flink CLI should exit when app is accepted with detach mode on Yarn

    XMLWordPrintableJSON

Details

    Description

      Why

      In our internal streaming platform, we will use flink-cli tool to submit Flink streaming application on Yarn.

      However when encountering Hadoop cluster down and then lots of flink apps need to be resubmitted, the submitter of worker in our platform will hang at this time.

      Because the Yarn cluster resources are tight and the scheduling efficiency becomes low when lots of apps needs to be started.

      And flink-cli will not exit until the app status changes to running.

      In addition, I also think there is no need to wait when app status is accepted with detach mode on Yarn.

      How

      When app in accpeted status, flink-cli should exit directly to release submitter worker process resource. The PR could refer to : https://github.com/apache/flink/blob/f191becdb42d6df823a103dc4f787c4737baa8e7/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java#L1224

      Attachments

        Issue Links

          Activity

            People

              zuston Junfan Zhang
              zuston Junfan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: