In ReduceTask.ReduceCopier.ShuffleRamManager, a memory reservation is done for in-memory shuffle, and that uses Runtime.getRuntime().maxMemory(). The return value of this seems to be machine-dependent.
This should be rigged by TestReduceFetch in mapred.child.java.opts, and match -Xmx128m.
testReduceFromPartialMem is an awkward test to write. Its intent is to configure the reduce so that- presented with a set of crafted map outputs- it will make a particular guess about how to optimize its I/O. If we can't rig the total memory because -Xmx has a machine-dependent interpretation, then writing such a test will be a real pain with our current set of configuration options. We could add a parameter for the memory reservation that defaults to querying the runtime; that would let us be certain of our memory limit, but not burden the user with setting it. I'm really surprised that setting -Xmx doesn't work, though...
The failure for testReduceFromMem suggests this should use the counters added in
HADOOP-2774 rather than the FileSystem counters to validate the result.