Ideally we should have the state transition itself to go to SUCCEEDED after unregister.
unregister needs to happens when a job is after COMMITTING. If we want unregister to be before SUCCEEDED, we have to insert another state in middle, where unregister can happen. Then, we still need to to map the state in middle to a external state. If we map it to SUCCEEDED, the problem is still there; if we map it to RUNNING, it is equivalent to returning RUNNING when AM is unregistered (in the prior comment).
In addition, the problem is not limited to SUCCEEDED only, is it? FAILED, KILLED and ERROR shouldn't be returned before unregister. In particular, FAILED and KILLED map to two internal states respectively. In this case, I'm afraid it's even more difficult to find state in between to execute unregister.
1) JobImpl go to succeeded/terminal state only after app master serviceStop() has completed?
In addition to the reason above, if serviceStop() is completed, the client is already unable to get the latest state from AM.
2) Change the MRClientProtocol to return job.state as the previous job state when job.state == succeeded. Then when MRClientProtocol.stop() is called it starts returning job.state == succeeded for 5 secs. Then it stops and the app master stops.
+1 for this method. It sounds work with limited touch. Again, we should consider FAILED, KILLED and ERROR as well, right?