[MAPREDUCE-5653] DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.9, 2.2.0
• Fix Version/s: 3.0.0-alpha1
• Component/s: distcp
• Hadoop Flags: Incompatible change
• Release Note: Prior to this change, distcp had hard-coded values for memory usage. Now distcp will honor memory settings in a way compatible with the rest of MapReduce.
• Assignee: Ratandeep Ratti
• Reporter: Mithun Radhakrishnan
• Created: 25/Nov/13 23:11
• Updated: 14/Oct/19 15:37
• Resolved: 01/Mar/15 06:54

Description

When a DistCp job is run through Oozie (through a Java action that launches DistCp), one sees that mapred.child.java.opts as set from the caller is honoured by DistCp. But, DistCp doesn't seem to honour any overrides for configs mapreduce.[map,reduce].memory.mb.

Problem has been identified. I'll post a patch shortly.