Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1476

committer.needsTaskCommit should not be called for a task cleanup attempt

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup.

      Description

      Currently, Task.done() calls committer.needsTaskCommit() to know whether it needs a commit or not. This need not be called for task cleanup attempt as no commit is required for a cleanup attempt.
      Due to MAPREDUCE-1409, we saw a case where cleanup attempt went into COMMIT_PENDING state.

      1. patch-1476.txt
        7 kB
        Amareshwari Sriramadasu
      2. patch-1476-1.txt
        9 kB
        Hemanth Yamijala
      3. patch-1476-2.txt
        11 kB
        Hemanth Yamijala
      4. patch-1476-ydist.txt
        10 kB
        Amareshwari Sriramadasu
      5. mr-1476-y20.patch
        10 kB
        Hemanth Yamijala

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Amareshwari Sriramadasu
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development