vmem.to.pmem.limit.ratio should we vmem-pmem-limit-ratio (better vmem-pmem-ratio)
renamed to vmem-pmem-ratio
Should we default the ratio to 1.0 to be compatible with current CS in 0.20?
I don't think so, since then it will be yet another config that everyone has to set before their Hadoop will work right We have too many of those today, and everyone is going to need to revamp their configs when deploying MR2 anyway. Let's use it as a forcing function to fix what we don't like.
Do you want to make resource.memory-gb as resource.memory-mb i.e. incorporate
The 80% limit on available RAM needs to be more conservative? I shudder to think it probably should be configurable...
This is just a WARN level message, not a true limit. I think on large machines, 80% is reasonable even if not quite advisable. (eg on a 48G machine it leaves 9.6GB free for other processes, which isn't bad)
Might be helpful to add both pmem and vmem in error msgs for both exceptional conditions for users?
I changed the message to look like:
Container [pid=19843,containerID=container_0_0000_01_000000] is running beyond virtual memory limits. Current usage: 1.9mb of 0b physical memory used; 20.2mb of 0b virtual memory used. Killing container.
Dump of the process-tree for container_0_0000_01_000000 :