Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1476

committer.needsTaskCommit should not be called for a task cleanup attempt

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup.

      Description

      Currently, Task.done() calls committer.needsTaskCommit() to know whether it needs a commit or not. This need not be called for task cleanup attempt as no commit is required for a cleanup attempt.
      Due to MAPREDUCE-1409, we saw a case where cleanup attempt went into COMMIT_PENDING state.

        Attachments

        1. mr-1476-y20.patch
          10 kB
          Hemanth Yamijala
        2. patch-1476.txt
          7 kB
          Amareshwari Sriramadasu
        3. patch-1476-1.txt
          9 kB
          Hemanth Yamijala
        4. patch-1476-2.txt
          11 kB
          Hemanth Yamijala
        5. patch-1476-ydist.txt
          10 kB
          Amareshwari Sriramadasu

          Activity

            People

            • Assignee:
              amareshwari Amareshwari Sriramadasu
              Reporter:
              amareshwari Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: