[SAMZA-2165] Account for coordinator restarts in calls to status

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2
• Assignee: Jagadish
• Reporter: Jagadish
• Created: 15/Apr/19 23:32
• Updated: 11/Jun/19 22:10
• Resolved: 11/Jun/19 22:10

Description

Currently status of a Samza job is determined by a combination of:
1. Obtaining YARN's status for the job by querying the RM
2. Obtain the AM/coordinator URL for the job
3. If (1) is "Running", Query the job's coordinator URL if all containers have started

YARN may restart the coordinator between (2) and (3) and the old coordinator process may no longer be alive, triggering a ConnectException in (3). This causes the status-call to fail;

A better alternative to handle these retriable errors is to return a "New" status from the API - so that applications can keep polling.

Issue Links

• links to: GitHub Pull Request #995
• links to: GitHub Pull Request #1000