[SAMZA-2444] JobModel save in CoordinatorStreamStore resulting flush for each message

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4
• Assignee: Alan Zhang
• Reporter: Alan Zhang
• Created: 23/Jan/20 00:14
• Updated: 18/Feb/20 21:40
• Resolved: 05/Feb/20 17:59

Description

JobModelManager uses CoordinatorStreamStore.put() to save job model information which does flush for each message.

The flush operation in CoordinatorStreamStore is very heavy, so it may take over a very long time for AM to save the information in a run.

We should use CoordinatorStreamStore's batch put instead of single put.

 

Issue Links

• links to: GitHub Pull Request #1259