You are right Vinod.
Thinking abt this more, I think we should move the unregister call out of the stop() methods. We could make sure that as soon as unregister is called, we send an event back to JobImpl saying done with the RM, and then we can go ahead to delete the staging dir.
So, there'd be a state transition from
RUNNING->SUCCEEDED that would sent an unregister event to RMCommunicator.
SUCCEEDED -> SUCCEEDED in JobImpl, that would happen on an event triggered after unregister is called, and the same for Killed and Failed, which would delete the staging directory.
Do you guys see any issues with that?