Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
Why
In our internal streaming platform, we will use flink-cli tool to submit Flink streaming application on Yarn.
However when encountering Hadoop cluster down and then lots of flink apps need to be resubmitted, the submitter of worker in our platform will hang at this time.
Because the Yarn cluster resources are tight and the scheduling efficiency becomes low when lots of apps needs to be started.
And flink-cli will not exit until the app status changes to running.
In addition, I also think there is no need to wait when app status is accepted with detach mode on Yarn.
How
When app in accpeted status, flink-cli should exit directly to release submitter worker process resource. The PR could refer to : https://github.com/apache/flink/blob/f191becdb42d6df823a103dc4f787c4737baa8e7/flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java#L1224
Attachments
Attachments
Issue Links
- links to