Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2350

LocalJobRunner uses "mapred.output.committer.class" configuration property to retrieve the OutputCommitter (regardless of whether the old API is used or the new API)

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      LocalJobRunner uses the "mapred.output.committer.class" configuration property to retrieve the output committer for the job, which can be different from the Output Committer returned from OutputFormat.getOutputCommitter(TaskAttemptContext context). So, two different output committers can be used in the same job.

      See line 324 in org.apache.hadoop.mapred.LocalJobRunner: OutputCommitter outputCommitter = job.getOutputCommitter();

      Need to modify this behavior to check if the new or the old API is used, and then return the correct output committer.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            devaraj Devaraj Kavali Assign to me
            ahmed.radwan Ahmed Radwan
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment