[FLINK-19548] Report correct progress to yarn resource manager

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Won't Do
• Affects Version/s: 1.11.2
• Component/s: Deployment / YARN
• Assignee: Unassigned
• Reporter: fa zheng
• Created: 09/Oct/20 03:43
• Updated: 11/Aug/21 12:31
• Resolved: 11/Aug/21 12:31

Description

Currently, we directly report 1 to yarn resource manager. It doesn't make sense for both stream and batch job. For a stream job, it's better to report 0, and for a batch job, it's better to report a meaningful progress. A simple way is calculating totalSubtask and currentFinishedSubTask in scheduler, and return (float)currentFinishedSubTask.get()/totalSubTask in YarnResourceManager.getProgress()