[FLINK-13132] Allow JobClusterEntrypoints use user main method to generate job graph

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 1.8.0, 1.8.1
• Component/s: Deployment / YARN
• Assignee: Unassigned
• Reporter: Zhenqiu Huang
• Created: 07/Jul/19 04:57
• Updated: 11/Aug/21 12:40
• Resolved: 11/Aug/21 12:40

Description

We are building a service that can transparently deploy a job to different cluster management systems, such as Yarn and another internal system. It is very cost to download the jar and generate JobGraph in the client side. Thus, I want to propose an improvement to make Yarn Entrypoints can be configurable to use either FileJobGraphRetriever or ClassPathJobGraphRetriever. It is actually a long asking TODO in AbstractionYarnClusterDescriptor in line 834.

https://github.com/apache/flink/blob/21468e0050dc5f97de5cfe39885e0d3fd648e399/flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java#L834

Issue Links

• duplicates: FLINK-16661 Introduce the YarnApplicationClusterEntrypoint