Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1476

committer.needsTaskCommit should not be called for a task cleanup attempt

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 0.21.0
    • None
    • None
    • Reviewed
    • Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup.

    Description

      Currently, Task.done() calls committer.needsTaskCommit() to know whether it needs a commit or not. This need not be called for task cleanup attempt as no commit is required for a cleanup attempt.
      Due to MAPREDUCE-1409, we saw a case where cleanup attempt went into COMMIT_PENDING state.

      Attachments

        1. mr-1476-y20.patch
          10 kB
          Hemanth Yamijala
        2. patch-1476.txt
          7 kB
          Amareshwari Sriramadasu
        3. patch-1476-1.txt
          9 kB
          Hemanth Yamijala
        4. patch-1476-2.txt
          11 kB
          Hemanth Yamijala
        5. patch-1476-ydist.txt
          10 kB
          Amareshwari Sriramadasu

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amareshwari Amareshwari Sriramadasu
            amareshwari Amareshwari Sriramadasu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment