[SAMZA-2609] Data model changes to track lifecycle information of job coordinator

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7
• Assignee: Bharath Kumarasubramanian
• Reporter: Bharath Kumarasubramanian
• Created: 02/Dec/20 12:21
• Updated: 08/Dec/20 19:22
• Resolved: 08/Dec/20 19:22

Description

Goal: Implement the data model changes needed to track the lifecycle of AM.

Description: With AM HA, the new AM will regenerate job model, do job planning and rewrite configurations and may have to spin up containers. The new rewritten configurations, job model can change from the existing metadata on which the containers are running. In order to handle this change of metadata, we need a way to identify lifecycle of the AM and store some information on the previous snapshot of the metadata.

Changes: 

1. Epoch identifier of the AM attempt so that we can determine if the new attempt is part of the same lifecycle of AM or a new deployment lifecycle
2. Additional information on the job model, configuration (potentially subset of it) so that we can use this information to decide the course of action if current metadata changed in comparison with the previous snapshot.