Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1476

committer.needsTaskCommit should not be called for a task cleanup attempt

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup.

      Description

      Currently, Task.done() calls committer.needsTaskCommit() to know whether it needs a commit or not. This need not be called for task cleanup attempt as no commit is required for a cleanup attempt.
      Due to MAPREDUCE-1409, we saw a case where cleanup attempt went into COMMIT_PENDING state.

      1. mr-1476-y20.patch
        10 kB
        Hemanth Yamijala
      2. patch-1476.txt
        7 kB
        Amareshwari Sriramadasu
      3. patch-1476-1.txt
        9 kB
        Hemanth Yamijala
      4. patch-1476-2.txt
        11 kB
        Hemanth Yamijala
      5. patch-1476-ydist.txt
        10 kB
        Amareshwari Sriramadasu

        Activity

        Amareshwari Sriramadasu created issue -
        Amareshwari Sriramadasu made changes -
        Field Original Value New Value
        Attachment patch-1476.txt [ 12435549 ]
        Amareshwari Sriramadasu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hemanth Yamijala made changes -
        Attachment patch-1476-1.txt [ 12435718 ]
        Hemanth Yamijala made changes -
        Attachment patch-1476-2.txt [ 12435846 ]
        Amareshwari Sriramadasu made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Amareshwari Sriramadasu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Amareshwari Sriramadasu made changes -
        Attachment patch-1476-ydist.txt [ 12435855 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Hemanth Yamijala made changes -
        Release Note Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup.
        Hemanth Yamijala made changes -
        Attachment mr-1476-y20.patch [ 12436722 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Amareshwari Sriramadasu
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development