Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5653

DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.23.9, 2.2.0
    • 3.0.0-alpha1
    • distcp
    • None
    • Incompatible change
    • Prior to this change, distcp had hard-coded values for memory usage. Now distcp will honor memory settings in a way compatible with the rest of MapReduce.

    Description

      When a DistCp job is run through Oozie (through a Java action that launches DistCp), one sees that mapred.child.java.opts as set from the caller is honoured by DistCp. But, DistCp doesn't seem to honour any overrides for configs mapreduce.[map,reduce].memory.mb.

      Problem has been identified. I'll post a patch shortly.

      Attachments

        1. MAPREDUCE-5653.trunk.patch
          1 kB
          Mithun Radhakrishnan
        2. MAPREDUCE-5653.trunk.2.patch
          0.7 kB
          Ratandeep Ratti
        3. MAPREDUCE-5653.branch-2.patch
          1 kB
          Mithun Radhakrishnan
        4. MAPREDUCE-5653.branch-0.23.patch
          1 kB
          Mithun Radhakrishnan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rdsr Ratandeep Ratti Assign to me
            mithun Mithun Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment