[MAPREDUCE-5956] MapReduce AM should not use maxAttempts to determine if this is the last retry

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.6.0
• Component/s: applicationmaster, mrv2
• Target Version/s: 2.6.0
• Hadoop Flags: Reviewed
• Assignee: Wangda Tan
• Reporter: Vinod Kumar Vavilapalli
• Created: 02/Jul/14 02:58
• Updated: 01/Dec/14 03:10
• Resolved: 03/Sep/14 20:21

Description

Found this while reviewing YARN-2074. The problem is that after YARN-2074, we don't count AM preemption towards AM failures on RM side, but MapReduce AM itself checks the attempt id against the max-attempt count to determine if this is the last attempt.

    public void computeIsLastAMRetry() {
      isLastAMRetry = appAttemptID.getAttemptId() >= maxAppAttempts;
    }

This causes issues w.r.t deletion of staging directory etc..

Issue Links

• relates to: YARN-2074 Preemption of AM containers shouldn't count towards AM failures
• relates to: YARN-2261 YARN should have a way to run post-application cleanup