Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.6.0
    • Component/s: applicationmaster, mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Found this while reviewing YARN-2074. The problem is that after YARN-2074, we don't count AM preemption towards AM failures on RM side, but MapReduce AM itself checks the attempt id against the max-attempt count to determine if this is the last attempt.

          public void computeIsLastAMRetry() {
            isLastAMRetry = appAttemptID.getAttemptId() >= maxAppAttempts;
          }
      

      This causes issues w.r.t deletion of staging directory etc..

      1. MR-5956.patch
        14 kB
        Wangda Tan
      2. MR-5956.patch
        20 kB
        Wangda Tan

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Component/s applicationmaster [ 12315625 ]
          Component/s mrv2 [ 12314301 ]
          Component/s performance [ 12316500 ]
          Component/s resourcemanager [ 12315340 ]
          Component/s task [ 12312920 ]
          Vinod Kumar Vavilapalli made changes -
          Summary MapReduce AM should not MapReduce AM should not use maxAttempts to determine if this is the last retry
          Vinod Kumar Vavilapalli made changes -
          Component/s task [ 12312920 ]
          Component/s resourcemanager [ 12315340 ]
          Component/s performance [ 12316500 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue relates to YARN-2074 [ YARN-2074 ]
          Wangda Tan made changes -
          Assignee Vinod Kumar Vavilapalli [ vinodkv ] Wangda Tan [ leftnoteasy ]
          Wangda Tan made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue relates to YARN-2261 [ YARN-2261 ]
          Wangda Tan made changes -
          Attachment MR-5956.patch [ 12654931 ]
          Attachment MR-5956.patch [ 12654932 ]
          Wangda Tan made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Affects Version/s 2.4.0 [ 12326141 ]
          Wangda Tan made changes -
          Attachment MR-5956.patch [ 12654931 ]
          Wangda Tan made changes -
          Attachment MR-5956.patch [ 12655133 ]
          Zhijie Shen made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.6.0 [ 12327180 ]
          Resolution Fixed [ 1 ]
          Karthik Kambatla (Inactive) made changes -
          Target Version/s 2.5.0 [ 12326265 ] 2.5.1 [ 12327747 ]
          Karthik Kambatla (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Karthik Kambatla (Inactive) made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Target Version/s 2.5.1 [ 12327747 ] 2.6.0 [ 12327180 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Wangda Tan
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development