[FLINK-3134] Make YarnJobManager's allocate call asynchronous

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 0.10.0, 0.10.1, 1.0.0
• Fix Version/s: 1.0.0
• Component/s: Deployment / YARN
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 07/Dec/15 16:05
• Updated: 19/Dec/16 12:47
• Resolved: 17/Dec/15 14:21

Description

The allocate() call is used in the YarnJobManager to send a heartbeat to the YARN resource manager. This call may block the JobManager actor system for arbitrary time, e.g. if retry handlers are set up within the call to allocate.

I propose to use the AMRMClientAsync's callback methods to send heartbeats and update the container information. The API is available for our supported Hadoop versions (2.3.0 and above).

https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/yarn/client/api/async/AMRMClientAsync.html

Issue Links

• duplicates: FLINK-2629 Refactor YARN ApplicationMasterActor to use the async AM<-->RM client