Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5653

DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.9, 2.2.0
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Prior to this change, distcp had hard-coded values for memory usage. Now distcp will honor memory settings in a way compatible with the rest of MapReduce.

      Description

      When a DistCp job is run through Oozie (through a Java action that launches DistCp), one sees that mapred.child.java.opts as set from the caller is honoured by DistCp. But, DistCp doesn't seem to honour any overrides for configs mapreduce.[map,reduce].memory.mb.

      Problem has been identified. I'll post a patch shortly.

        Attachments

        1. MAPREDUCE-5653.branch-0.23.patch
          1 kB
          Mithun Radhakrishnan
        2. MAPREDUCE-5653.branch-2.patch
          1 kB
          Mithun Radhakrishnan
        3. MAPREDUCE-5653.trunk.2.patch
          0.7 kB
          Ratandeep Ratti
        4. MAPREDUCE-5653.trunk.patch
          1 kB
          Mithun Radhakrishnan

          Activity

            People

            • Assignee:
              rdsr Ratandeep Ratti
              Reporter:
              mithun Mithun Radhakrishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: